7. A/B test: lift and 95% confidence interval
HardExperiments
Given conversions and visitors for control (A) and treatment (B), return a dict with rate_a, rate_b, diff (rate_b − rate_a) and ci (the 95% confidence interval for the difference as [low, high], using the unpooled normal approximation and z = 1.96). Round every number to 4 decimals.
Also return significant: true when the interval excludes 0.
Example 1
Input: a: 200/4000, b: 260/4000
Output: {'rate_a': 0.05, 'rate_b': 0.065, 'diff': 0.015, ...}Constraints
Visitors > 0 in both groups