Skip to interview prep
Apexed

67. Why can oversampling rare examples before splitting a dataset inflate validation performance?

EasyMachine learning basics

Technical
Show model answer

Duplicated or closely related examples can land in both training and validation, making the validation task easier than genuinely unseen data. Synthetic examples created using the whole dataset can also transmit evaluation information.

Split first, and apply any learned resampling procedure only within each training partition. Evaluate on a representative untouched validation population, using the deployment class balance unless a different target population is explicitly justified.

What interviewers listen for

  • Split before learning or resampling
  • Prevent related examples crossing partitions
  • Evaluate realistic class prevalence
Why can oversampling rare examples before splitting a dataset inflate validation performance — Data & analytics interview question — Apexed