49. An experiment compares total revenue divided by total sessions. How would you estimate uncertainty without treating sessions as independent users?
HardStatistics & experiments
Technical
Show model answer
Start with the assignment unit and retain each user's joint revenue and session totals. The numerator and denominator can be correlated, and users with many sessions contribute several dependent observations.
Use an appropriate ratio estimator with uncertainty that respects the assignment structure, such as resampling users and recomputing the ratio. State whether the target is revenue per session or average user-level ratios; those are different quantities.
What interviewers listen for
- Respect the assignment unit
- Preserve numerator-denominator dependence
- Define the target ratio precisely