57. A few unusually large purchases dominate an experiment's revenue estimate. How would you handle them?
MediumStatistics & experiments
Technical
Show model answer
Verify whether the purchases are real and whether the metric is meant to include them. Removing valid observations because they change the conclusion can alter the target quantity and bias a post-hoc analysis.
Use prespecified robust methods or show transparent sensitivity analyses alongside the original estimate. Explain whether capping values changes the business question, and preserve uncertainty that reflects the heavy-tailed data rather than hiding inconvenient transactions.
What interviewers listen for
- Validate unusual observations
- Preserve the intended estimand
- Disclose robust and sensitivity analyses