Skip to interview prep
Apexed
Marketing60 / 100

60. A team checks a campaign experiment hourly and stops as soon as the result looks favorable. What is the concern?

MediumAnalytics & budget

Case
Show model answer

Repeated unplanned looks and selective stopping can change the error properties of the analysis. The most favorable moment may reflect random variation rather than a stable improvement, even if the dashboard displays a familiar significance threshold.

Agree a suitable fixed or sequential design before launch. Use predefined stopping and safety rules, disclose deviations, and do not present a selected snapshot as the originally planned procedure.

What interviewers listen for

  • Recognize selection from repeated looks
  • Use an appropriate planned design
  • Disclose stopping rules and deviations
A team checks a campaign experiment hourly and stops as soon as the result looks favorable. What is the concern — Marketing interview question — Apexed