23. A very large study finds a tiny statistically significant effect. What should a scientist ask next?
EasyStatistics & inference
Technical
Show model answer
Ask about the effect’s size, uncertainty and practical or scientific relevance. A large sample can make a small difference statistically detectable even when it does not materially change the intended decision or explanation.
Compare the estimate with a justified meaningful threshold, costs and alternative explanations. Report the actual magnitude rather than using significance alone as a success label, and avoid declaring irrelevance without considering the application.
What interviewers listen for
- Report magnitude and uncertainty
- Assess meaningful impact
- Significance is not importance