60. A model will predict next month's behavior for existing and newly acquired customers. How would you choose validation splits?
MediumMachine learning basics
Technical
Show model answer
Match validation to deployment rather than selecting a convenient random row split. Use later periods to assess future performance and a customer-group separation where needed to test generalization to people not represented in training.
Ensure features reflect information available at prediction time, and report existing-customer and new-customer results separately. Similar aggregate scores can conceal a failure on new customers, especially when repeated records from the same person cross partitions.
What interviewers listen for
- Match splits to deployment conditions
- Respect time and customer groups
- Report new-customer performance separately