69. A risk model has similar overall accuracy across regions, but one region has far fewer labeled cases. Is that enough to establish comparable performance?
HardBusiness cases
Case
Show model answer
No. Similar point estimates can hide substantial uncertainty, different error costs and different outcome prevalence. Inspect subgroup sample sizes, relevant error rates and intervals, and examine how labels were obtained.
Identify the decisions affected and work with domain and governance owners on acceptable risks. Seek additional representative evidence or limit use where confidence is inadequate; one equal-looking aggregate metric does not establish fairness or suitability.
What interviewers listen for
- Inspect uncertainty and label coverage
- Compare decision-relevant errors
- Set domain-specific use limits