https://datatweets.com/courses/ab-testing/analyzing-mean-metrics/Recent content in Analyzing Mean Metrics on DATATWEETSHugoenCopyright (c) 2025 DatatweetsSun, 28 Jun 2026 09:00:00 +0200https://datatweets.com/courses/ab-testing/analyzing-mean-metrics/lesson-1-the-two-sample-t-test/Fri, 13 Mar 2026 09:00:00 +0200https://datatweets.com/courses/ab-testing/analyzing-mean-metrics/lesson-1-the-two-sample-t-test/Rates use the z-test; averages use the t-test. This lesson builds the two-sample t-test for a difference in means, runs it (Welch’s version) on Lumen’s revenue data, and finds a $1.22 lift that lands at p = 0.057 — a borderline result the rest of the module will dissect.https://datatweets.com/courses/ab-testing/analyzing-mean-metrics/lesson-2-welchs-t-test-and-unequal-variances/Fri, 13 Mar 2026 09:00:00 +0200https://datatweets.com/courses/ab-testing/analyzing-mean-metrics/lesson-2-welchs-t-test-and-unequal-variances/Student’s t-test pools both groups’ spread into one estimate; Welch’s lets each group keep its own variance. On Lumen’s unequal-variance revenue data the two give opposite answers — Student says p = 0.0063 (significant), Welch says p = 0.0566 (not) — and this lesson shows why Welch is the correct one and the safe default.https://datatweets.com/courses/ab-testing/analyzing-mean-metrics/lesson-3-skewed-metrics-and-the-mann-whitney-test/Fri, 13 Mar 2026 09:00:00 +0200https://datatweets.com/courses/ab-testing/analyzing-mean-metrics/lesson-3-skewed-metrics-and-the-mann-whitney-test/Revenue and time metrics have a long tail of big spenders, and under that skew the mean can move opposite to the typical user. This lesson reveals that on Lumen’s data the treatment mean is higher while its median is lower, introduces the rank-based Mann-Whitney U test, and shows honestly that the mean and the ranks answer different questions — both correct.https://datatweets.com/courses/ab-testing/analyzing-mean-metrics/lesson-4-confidence-intervals-for-the-difference-in-means/Fri, 13 Mar 2026 09:00:00 +0200https://datatweets.com/courses/ab-testing/analyzing-mean-metrics/lesson-4-confidence-intervals-for-the-difference-in-means/The two-sample t-test told you Lumen’s $1.22 revenue lift is borderline. A confidence interval says how big the effect plausibly is, in dollars. This lesson builds the Welch 95% CI for the difference in means, lands at [-$0.03, +$2.47], and shows why an interval that includes $0 is the same verdict as p = 0.0566 — no evidence of a real lift.https://datatweets.com/courses/ab-testing/analyzing-mean-metrics/lesson-5-guided-project-analyze-lumens-revenue-experiment/Fri, 13 Mar 2026 09:00:00 +0200https://datatweets.com/courses/ab-testing/analyzing-mean-metrics/lesson-5-guided-project-analyze-lumens-revenue-experiment/Put the module together on Lumen’s revenue experiment. The average rose $1.22, but a Welch t-test says it’s not significant, the confidence interval includes zero, and a Mann-Whitney check reveals the typical user actually spends less. You’ll write the honest readout and make the ship call the mean alone would have botched.