Skip to interview prep
Apexed

73. An AI workflow passes a task once and fails it on another run. How should the evaluation represent that variability?

EasyMachine learning basics

Technical
Show model answer

Repeat relevant tasks under documented conditions and report a distribution or success rate with uncertainty. Preserve the model, configuration and tool environment so variation is not confused with an unrecorded system change.

Decide whether the product needs one reliable attempt or permits retries, and score that actual policy. Reporting only the best run answers a different question and can hide user-visible instability or extra operating cost.

What interviewers listen for

  • Repeat under recorded conditions
  • Score the deployed retry policy
  • Report variability rather than best runs
An AI workflow passes a task once and fails it on another run. How should the evaluation represent that variability — Data & analytics interview question — Apex