https://datatweets.com/courses/ab-testing/analyzing-proportion-metrics/Recent content in Analyzing Proportion Metrics on DATATWEETSHugoenCopyright (c) 2025 DatatweetsSun, 28 Jun 2026 09:00:00 +0200https://datatweets.com/courses/ab-testing/analyzing-proportion-metrics/lesson-1-the-two-proportion-z-test/Fri, 06 Mar 2026 09:00:00 +0200https://datatweets.com/courses/ab-testing/analyzing-proportion-metrics/lesson-1-the-two-proportion-z-test/Comparing two conversion rates is the most common analysis in A/B testing. This lesson builds the two-proportion z-test step by step — difference, pooled standard error, z, and p-value — and runs it on Lumen’s real seeded experiment, returning z = 3.49 and p = 0.0005.https://datatweets.com/courses/ab-testing/analyzing-proportion-metrics/lesson-2-reading-the-p-value/Fri, 06 Mar 2026 09:00:00 +0200https://datatweets.com/courses/ab-testing/analyzing-proportion-metrics/lesson-2-reading-the-p-value/A p-value is easy to compute and easy to misread. This lesson pins down what it actually means, corrects the four classic misinterpretations, sets the p-versus-α decision rule, and contrasts a significant Lumen result (p = 0.00048) with an underpowered one (p = 0.2857) to show that a big p-value is not proof of no effect.https://datatweets.com/courses/ab-testing/analyzing-proportion-metrics/lesson-3-confidence-intervals-for-the-difference/Fri, 06 Mar 2026 09:00:00 +0200https://datatweets.com/courses/ab-testing/analyzing-proportion-metrics/lesson-3-confidence-intervals-for-the-difference/A p-value tells you an effect is real, but a decision needs its size. This lesson builds the 95% confidence interval for the difference in two proportions using the unpooled standard error, runs it on Lumen’s experiment to get [+0.97, +3.43] points, and shows the duality: the interval excludes 0 exactly when the test is significant.https://datatweets.com/courses/ab-testing/analyzing-proportion-metrics/lesson-4-making-the-ship-decision/Fri, 06 Mar 2026 09:00:00 +0200https://datatweets.com/courses/ab-testing/analyzing-proportion-metrics/lesson-4-making-the-ship-decision/A green p-value doesn’t automatically mean ‘ship it.’ This lesson lays out the four checks a real ship decision needs — statistical significance, practical significance against the design MDE, guardrails, and the full confidence interval — and applies them honestly to Lumen’s borderline result: significant, positive, and best-estimate above the bar, but with a lower CI bound that dips below it.https://datatweets.com/courses/ab-testing/analyzing-proportion-metrics/lesson-5-guided-project-analyze-lumens-signup-experiment/Fri, 06 Mar 2026 09:00:00 +0200https://datatweets.com/courses/ab-testing/analyzing-proportion-metrics/lesson-5-guided-project-analyze-lumens-signup-experiment/The Module 4 capstone: take Lumen’s seeded signup experiment from raw counts to a shipped verdict. You’ll compute the conversion rates, run the two-proportion z-test, build the 95% confidence interval, apply the four-check decision framework, and write the experiment readout — the deliverable that turns a p-value into a decision.