\( \newcommand{\vect}[1]{\mathbf{#1}} \newcommand{\mat}[1]{\mathbf{#1}} \newcommand{\R}{\mathbb{R}} \newcommand{\E}{\mathbb{E}} \newcommand{\Prob}{\mathbb{P}} \newcommand{\T}{^{\top}} \newcommand{\norm}[1]{\lVert #1 \rVert} \newcommand{\abs}[1]{\lvert #1 \rvert} \newcommand{\ihat}{\mkern1mu\hat{\boldsymbol{\imath}}\mkern2mu} \newcommand{\jhat}{\mkern1mu\hat{\boldsymbol{\jmath}}\mkern2mu} \newcommand{\pd}[2]{\frac{\partial #1}{\partial #2}} \newcommand{\argmax}{\operatorname*{arg\,max}} \newcommand{\argmin}{\operatorname*{arg\,min}} \newcommand{\cdata}[1]{\class{c-data}{#1}} \newcommand{\cmodel}[1]{\class{c-model}{#1}} \newcommand{\ctruth}[1]{\class{c-truth}{#1}} \newcommand{\cgrad}[1]{\class{c-grad}{#1}} \newcommand{\closs}[1]{\class{c-loss}{#1}} \newcommand{\cparam}[1]{\class{c-param}{#1}} \newcommand{\csecond}[1]{\class{c-second}{#1}} \renewcommand{\,}{\mkern3mu} \)

S01.7 · Simpson’s paradox and confounding, a first look

A free, illustrated module of 4 lessons and one interactive in S01 Exploratory Data Analysis: A comparison that reverses: Simpson’s paradox in a table; Simpson’s paradox in a scatter plot; Confounding: a third variable behind both; Split or pool? The numbers alone cannot say.

S01.7 L1 · Core 4 lessons · about 100 min Live · in review

Part of S01 Exploratory Data Analysis in the Statistics and Data Analysis track.

A comparison can hold inside every group and reverse when the groups are pooled. These four lessons find the reversal in a table of kidney-stone operations and in a scatter plot of study hours, explain it exactly as weighted means taken with different weights and as a sum of products split into a part inside the groups and a part between them, name the confounder behind it, compute one fair comparison by direct standardisation, and show why the same counts can call for the split view under one story and the pooled view under another, until a coin flip settles it.

What you need first. The weighted mean of stratum means (S01.2.3), conditional proportions in a two-way table (S01.5.4), and the covariance, correlation and trend line of S01.5. Some pandas: groupby and crosstab (P02.6).

Lessons

A comparison that reverses: Simpson’s paradox in a table S01.7.1 25 min

Open surgery beats a keyhole operation for small kidney stones and for large ones, yet loses overall. See why: an overall rate is a weighted mean of group rates, and the two treatments were averaged with very different weights.
Simpson’s paradox in a scatter plot S01.7.2 25 min

Eight students: more study goes with higher scores inside each course, yet with lower scores when the courses are pooled. Split the sum of products into a part inside the groups and a part between them, and see the same reversal in the iris flowers.
Confounding: a third variable behind both S01.7.3 25 min

Stone size steers both the choice of operation and its chance of success. Name such a confounder, compare like with like by stratifying, compute one fair figure by direct standardisation, and list the rivals to any causal reading of a correlation.
Split or pool? The numbers alone cannot say S01.7.4 25 min

The same counts can call for the split comparison under one story and the pooled one under another. Learn the rule of time order, work Berkeley’s 1973 admissions, and simulate how a coin flip that assigns the treatment makes the reversal disappear.

The interactive

S01.7-X1 · Split the scatter. Thirty students in two courses, each with a rising trend. Slide the courses apart along each axis and watch the pooled trend line turn and flip sign, while the read-out splits the sum of products into its within and between parts. It lives in Simpson’s paradox in a scatter plot.