Welcome to Analyzing Mean Metrics, the fifth module. Not every metric is a rate. Revenue per user, time on task, order value, sessions per week — these are averages of a continuous quantity, and the two-proportion z-test you just learned doesn’t apply to them. They also bring a new complication: money and time metrics are almost always heavily skewed, with a long tail of big spenders or long sessions that can make the average behave in surprising — and misleading — ways. This module gives you the tools to analyze mean metrics honestly.

You’ll start with the two-sample t-test, the mean-metric counterpart to the z-test. You’ll learn why Welch’s t-test — which allows the two groups to have different variances — is the safe default, and watch it overturn a false “significant” result on real data. You’ll build a confidence interval for the difference in means. And you’ll meet the Mann-Whitney U test, a rank-based fallback for when skew makes the average untrustworthy — on a Lumen revenue experiment where the mean went up but the typical user actually spent less. The module ends with a full guided analysis and a decision that the raw average alone would have gotten wrong.

Every result here is real, runnable Python: the t-tests, the confidence interval, and the Mann-Whitney test are computed with numpy and scipy on the same seeded, skewed revenue data, so you see exactly how — and when — the average can deceive you. Start with Lesson 1 on the two-sample t-test.