Module · 5 lessons

Capstone

One experiment, end to end: design a real Lumen test, size it, simulate it, validate it, analyze it two ways, and write the decision readout — putting the whole course together.

At a glance

Level
Intermediate
Lessons
5 lessons
Time to complete
1 week
Cost
Free forever · no sign-up

Welcome to the Capstone, the final module of the course. You’ve learned every piece of the experimentation workflow separately — the logic of randomization, designing and sizing a test, analyzing proportions and means, avoiding the validity traps, and the advanced tools beyond a basic A/B. Now you’ll assemble all of it into a single, complete experiment, run the way a real data scientist runs one: from a one-line business brief to a written ship decision.

The scenario is fresh: Lumen wants to know whether a new onboarding checklist lifts the 7-day activation rate — the share of new users who reach a key milestone in their first week. You’ll write the design (hypothesis, metrics, randomization unit) and compute the sample size. You’ll simulate the experiment and — before looking at the result — run the validity checks that decide whether it can be trusted at all. You’ll analyze it with the two-proportion z-test and a confidence interval, then cross-check with a Bayesian readout that answers “what’s the probability this worked?” directly. And you’ll write the decision readout that ties significance, effect size, and guardrails into a clear recommendation. The module — and the course — closes with that readout and a look back at everything you’ve built.

Every step is real, runnable Python: the sample-size calculation, the simulated data, the SRM check, the z-test and confidence interval, and the Bayesian posterior all compute exactly with numpy and scipy, and the frequentist and Bayesian answers land in the same place. Start with Lesson 1: the brief and the design.

Lessons in this module

Achievement

Complete all 5 lessons to finish the Capstone module.

Start module