46. A planned equal-allocation experiment has a persistent imbalance in recorded groups. What would you investigate before interpreting the treatment effect?
MediumStatistics & experiments
Technical
Show model answer
Compare observed assignment counts with the configured allocation using an appropriate sample-ratio check. Inspect eligibility, assignment, exposure logging and downstream exclusions to locate where the imbalance first appears.
A variant may alter whether users reach a logging event or are filtered from analysis. Repair the cause or clearly withhold an effect conclusion; mechanically reweighting the final groups can conceal a broken experiment rather than restore randomization.
What interviewers listen for
- Locate the first population mismatch
- Inspect treatment-dependent exclusions
- Resolve integrity before effect claims