72. A team repeatedly edits a prompt after inspecting every failure in its test set. Can the final score still be described as an untouched test result?
EasyMachine learning basics
Technical
Show model answer
No. Once the test examples guide prompt changes, they have become development data. The final score can describe performance on that familiar set, but it is no longer an independent estimate of generalization.
Keep those examples as useful regressions and evaluate the finalized system on a separate protected set. Record exposure and selection decisions so the distinction between improvement, regression coverage and independent assessment remains clear.
What interviewers listen for
- Recognize test-set reuse
- Retain failures as development regressions
- Evaluate on a protected independent set