\( \newcommand{\vect}[1]{\mathbf{#1}} \newcommand{\mat}[1]{\mathbf{#1}} \newcommand{\R}{\mathbb{R}} \newcommand{\E}{\mathbb{E}} \newcommand{\Prob}{\mathbb{P}} \newcommand{\T}{^{\top}} \newcommand{\norm}[1]{\lVert #1 \rVert} \newcommand{\abs}[1]{\lvert #1 \rvert} \newcommand{\ihat}{\mkern1mu\hat{\boldsymbol{\imath}}\mkern2mu} \newcommand{\jhat}{\mkern1mu\hat{\boldsymbol{\jmath}}\mkern2mu} \newcommand{\pd}[2]{\frac{\partial #1}{\partial #2}} \newcommand{\argmax}{\operatorname*{arg\,max}} \newcommand{\argmin}{\operatorname*{arg\,min}} \newcommand{\cdata}[1]{\class{c-data}{#1}} \newcommand{\cmodel}[1]{\class{c-model}{#1}} \newcommand{\ctruth}[1]{\class{c-truth}{#1}} \newcommand{\cgrad}[1]{\class{c-grad}{#1}} \newcommand{\closs}[1]{\class{c-loss}{#1}} \newcommand{\cparam}[1]{\class{c-param}{#1}} \newcommand{\csecond}[1]{\class{c-second}{#1}} \renewcommand{\,}{\mkern3mu} \)

M06.11 · Law of large numbers and central limit theorem

A free, illustrated module of 4 lessons and one interactive in M06 Probability I: The sample mean is a random variable; The law of large numbers; The central limit theorem; Using the normal approximation.

M06.11 L1 · Core 4 lessons · about 100 min Live · in review

Part of M06 Probability I in the Mathematics track.

Average n independent draws and the result is a new random variable, the sample mean. These four lessons follow it as n grows. Its expectation stays at the mean mu while its standard deviation shrinks like sigma over root n; Chebyshev’s inequality turns that into the law of large numbers, the reason a simulation or a test-set score can be trusted; and the central limit theorem says that its shape settles onto the same bell whatever the starting distribution. The module ends by using the bell to approximate probabilities for sums and means, with the continuity correction and the cases where the bell fails.

What you need first. Independent random variables and sums by convolution (M06.5.4), the normal density and Phi (M06.6.4), expectation, variance and standardising (M06.7), and the binomial distribution (M06.8.1). The code uses fractions, NumPy and scipy.stats.

Lessons

1. The sample mean is a random variable M06.11.1 25 min

   Average n independent draws and you get a new random variable, the sample mean. Derive that its expectation is the mean μ and its variance is σ²/n, so its spread shrinks like 1/√n, and see it on dice and on a test-set accuracy.

2. The law of large numbers M06.11.2 25 min

   Why a running average settles on the mean. Prove Markov’s and Chebyshev’s inequalities in a few lines, use them to prove the weak law of large numbers, and see what the law does not promise: no catching up after a streak.

3. The central limit theorem M06.11.3 25 min

   Standardise the mean of n independent draws and its distribution settles onto the same bell, whatever the draws looked like. See it on dice, a skewed delay, two humps and a coin, learn why only μ and σ survive, and meet a case where the bell never comes.

4. Using the normal approximation M06.11.4 25 min

   Turn the central limit theorem into numbers: approximate probabilities for sums and means by standardising and reading Φ, with the continuity correction for counts, the mean ± 2 standard deviations range, and the cases where the bell is a poor guide.

The interactive

M06.11-X1 · Sample-mean machine. Choose a source (a die, a coin with adjustable p, a skewed or a two-humped distribution) and a sample size n from 1 to 50. Each press draws n values and drops their mean into a growing histogram; the read-out compares its centre and spread with mu and sigma over root n, and a toggle lays the matching normal curve on top. It lives in The central limit theorem.