Welcome to Designing an Experiment, the second module. In Module 1 you saw why a randomized comparison lets you claim cause and effect. But randomization alone doesn’t make an experiment good — a test aimed at a vague goal, measuring the wrong thing, or too small to see the effect you care about will waste weeks and still leave you guessing. The quality of an experiment is mostly decided before it runs, in its design. This module is about getting that design right.

You’ll start by turning a fuzzy goal into a sharp, testable hypothesis — specific, directional, and falsifiable. You’ll learn to choose metrics: one primary metric the decision hinges on, guardrails that must not get worse, and secondary metrics for context. You’ll pick the randomization unit — user, session, or cluster — that keeps the comparison valid and avoids users seeing both versions. And you’ll set the minimum detectable effect (MDE): the smallest change worth caring about, which directly determines how much data you need. The module ends with a guided project where you write a complete design for one of Lumen’s real decisions — a pricing-page experiment.

Every quantitative claim here is real, runnable Python on seeded data — the example metrics and the MDE-to-sample-size table both compute exact numbers you can reproduce, previewing the power analysis you’ll do in full next module. Start with Lesson 1 on turning a question into a hypothesis.