63. A rare-event classifier has a strong ROC AUC, but most alerts are false alarms. How can both be true?
MediumMachine learning basics
Technical
Show model answer
ROC AUC summarizes ranking across thresholds; it does not guarantee useful precision at the chosen operating point. When negatives greatly outnumber positives, even a modest false-positive rate can produce many more false than true alerts.
Inspect precision-recall behavior, the confusion matrix and alert volume at realistic prevalence and thresholds. Compare operational costs and available review capacity, and avoid selecting a model from an aggregate ranking metric alone.
What interviewers listen for
- Separate ranking from operating-point utility
- Use realistic event prevalence
- Inspect alert counts and precision