55. An overall experiment is inconclusive, but one country appears to benefit. How would you assess that subgroup claim?
HardStatistics & experiments
Technical
Show model answer
Check whether the subgroup was specified beforehand, how many subgroups were explored and whether estimates are precise. A significant result in one subgroup and a nonsignificant result in another does not itself establish different treatment effects.
Assess the interaction or heterogeneity directly using a suitable method, account for multiplicity and seek confirmation. Report the selection process and uncertainty instead of using a favorable slice to rescue an unsupported overall conclusion.
What interviewers listen for
- Check subgroup selection and multiplicity
- Test differences between subgroup effects
- Confirm rather than rescue a result