67. Why can oversampling rare examples before splitting a dataset inflate validation performance?
EasyMachine learning basics
Technical
Show model answer
Duplicated or closely related examples can land in both training and validation, making the validation task easier than genuinely unseen data. Synthetic examples created using the whole dataset can also transmit evaluation information.
Split first, and apply any learned resampling procedure only within each training partition. Evaluate on a representative untouched validation population, using the deployment class balance unless a different target population is explicitly justified.
What interviewers listen for
- Split before learning or resampling
- Prevent related examples crossing partitions
- Evaluate realistic class prevalence