Skip to interview prep
Apexed

62. A model ranks cases well, but cases assigned eighty percent risk fail only half the time. What would you investigate?

EasyMachine learning basics

Technical
Show model answer

The ranking may be useful while the probabilities are poorly calibrated. Compare predicted risk bands with observed frequencies on representative data, considering uncertainty and whether conditions have shifted since training.

If probabilities drive decisions, evaluate recalibration using separate suitable data and verify performance afterward. Recheck important subgroups and time periods. A high ranking score alone does not establish that an eighty-percent prediction corresponds to an eighty-percent event frequency.

What interviewers listen for

  • Separate ranking from calibration
  • Compare predictions with observed frequencies
  • Validate recalibration on separate data
A model ranks cases well, but cases assigned eighty percent risk fail only half the time. What would you investigate — Data & analytics interview question — Ape