Skip to interview prep
Apexed

55. An overall experiment is inconclusive, but one country appears to benefit. How would you assess that subgroup claim?

HardStatistics & experiments

Technical
Show model answer

Check whether the subgroup was specified beforehand, how many subgroups were explored and whether estimates are precise. A significant result in one subgroup and a nonsignificant result in another does not itself establish different treatment effects.

Assess the interaction or heterogeneity directly using a suitable method, account for multiplicity and seek confirmation. Report the selection process and uncertainty instead of using a favorable slice to rescue an unsupported overall conclusion.

What interviewers listen for

  • Check subgroup selection and multiplicity
  • Test differences between subgroup effects
  • Confirm rather than rescue a result
An overall experiment is inconclusive, but one country appears to benefit. How would you assess that subgroup claim — Data & analytics interview question — Apex