https://datatweets.com/courses/ab-testing/capstone/Recent content in Capstone on DATATWEETSHugoenCopyright (c) 2025 DatatweetsSun, 28 Jun 2026 09:00:00 +0200https://datatweets.com/courses/ab-testing/capstone/lesson-1-the-brief-and-the-design/Fri, 03 Apr 2026 09:00:00 +0200https://datatweets.com/courses/ab-testing/capstone/lesson-1-the-brief-and-the-design/The capstone runs one experiment end to end. This first lesson takes a business brief — does a new onboarding checklist lift 7-day activation? — and turns it into a complete design: a sharp hypothesis, a metric hierarchy, the randomization unit, and a computed sample size of 3,394 users per arm.https://datatweets.com/courses/ab-testing/capstone/lesson-2-running-and-validating-the-experiment/Fri, 03 Apr 2026 09:00:00 +0200https://datatweets.com/courses/ab-testing/capstone/lesson-2-running-and-validating-the-experiment/This lesson executes the design from Lesson 1: simulate a realistic coin-flip assignment of 6,788 users and their 7-day activation outcomes, then — before analyzing the result — run the sample ratio mismatch (SRM) check to confirm the split is fair. Validity comes before the p-value.https://datatweets.com/courses/ab-testing/capstone/lesson-3-analyzing-the-results/Fri, 03 Apr 2026 09:00:00 +0200https://datatweets.com/courses/ab-testing/capstone/lesson-3-analyzing-the-results/The SRM check passed, so the data can be trusted. This lesson analyzes the validated result: control activated 25.00% and treatment 28.32%, a +3.32 point lift. A two-proportion z-test gives z = 3.10, p = 0.00197, and a 95% CI of [+1.22, +5.43] points — significant, though the CI’s lower bound sits below the 3-point bar.https://datatweets.com/courses/ab-testing/capstone/lesson-4-the-cross-check-and-the-decision/Fri, 03 Apr 2026 09:00:00 +0200https://datatweets.com/courses/ab-testing/capstone/lesson-4-the-cross-check-and-the-decision/This lesson gives Lumen’s result a second opinion. A Bayesian readout on the same validated data puts the probability the treatment is truly better at 99.9% with a credible interval that matches the frequentist CI, and then a five-check scorecard — valid, significant, practically meaningful, corroborated, safe — turns it all into a clean decision: ship the onboarding checklist.https://datatweets.com/courses/ab-testing/capstone/lesson-5-the-full-readout-and-course-wrap-up/Fri, 03 Apr 2026 09:00:00 +0200https://datatweets.com/courses/ab-testing/capstone/lesson-5-the-full-readout-and-course-wrap-up/The finale. Every number from Lessons 1-4 is gathered into a single, self-contained experiment readout: the question, the design, the SRM validity check, the significant +3.32-point lift, the Bayesian cross-check, and the SHIP decision with its caveats. Then a celebratory recap of the whole course, from why-randomize to a full capstone.