\( \newcommand{\vect}[1]{\mathbf{#1}} \newcommand{\mat}[1]{\mathbf{#1}} \newcommand{\R}{\mathbb{R}} \newcommand{\E}{\mathbb{E}} \newcommand{\Prob}{\mathbb{P}} \newcommand{\T}{^{\top}} \newcommand{\norm}[1]{\lVert #1 \rVert} \newcommand{\abs}[1]{\lvert #1 \rvert} \newcommand{\ihat}{\mkern1mu\hat{\boldsymbol{\imath}}\mkern2mu} \newcommand{\jhat}{\mkern1mu\hat{\boldsymbol{\jmath}}\mkern2mu} \newcommand{\pd}[2]{\frac{\partial #1}{\partial #2}} \newcommand{\argmax}{\operatorname*{arg\,max}} \newcommand{\argmin}{\operatorname*{arg\,min}} \newcommand{\cdata}[1]{\class{c-data}{#1}} \newcommand{\cmodel}[1]{\class{c-model}{#1}} \newcommand{\ctruth}[1]{\class{c-truth}{#1}} \newcommand{\cgrad}[1]{\class{c-grad}{#1}} \newcommand{\closs}[1]{\class{c-loss}{#1}} \newcommand{\cparam}[1]{\class{c-param}{#1}} \newcommand{\csecond}[1]{\class{c-second}{#1}} \renewcommand{\,}{\mkern3mu} \)

M06.7 · Expectation, variance and moments

A free, illustrated module of 4 lessons and one interactive in M06 Probability I: Expectation: the long-run average; Linearity of expectation; Variance and standard deviation; Standardising and higher moments.

M06.7 L1 · Core 4 lessons · about 100 min Live · in review

Part of M06 Probability I in the Mathematics track.

A whole distribution is a lot to carry around. These four lessons boil it down to a few numbers that do most of the work: the expectation, which is where the distribution balances and what a long run of draws averages to; the variance and standard deviation, which say how far it spreads; and the higher moments, skewness and kurtosis, which describe its lean and its tails. Along the way come the rules that make these numbers easy to compute, linearity of expectation and the variance of a sum, and the fact behind every squared-error loss: the mean is the best single guess.

What you need first. Random variables, PMFs and independence of random variables (M06.5), densities and the normal density (M06.6), and integrals, including integration by parts (M04.5 to M04.7). The code uses Python with fractions, NumPy and scipy.stats.

Lessons

1. Expectation: the long-run average M06.7.1 25 min

   The expectation of a random variable is the average of its values weighted by their probabilities: the balance point of its distribution and the number a long run of draws settles on. Learn it for PMFs and densities, and for any function of X.

2. Linearity of expectation M06.7.2 25 min

   The expectation of a sum is the sum of the expectations, with no independence needed. Prove it over the sample space, count with indicators to get the expected heads in n flips and the expected shared birthdays among 23 people, and see where products and nonlinear functions break the pattern.

3. Variance and standard deviation M06.7.3 25 min

   The variance is the expected squared distance from the mean and the standard deviation is its square root, in the units of X. Derive the shortcut E[X²] − (E[X])², how shifts, stretches and independent sums change the spread, and why the spread of a sum grows like √n.

4. Standardising and higher moments M06.7.4 25 min

   Subtract the mean and divide by the standard deviation and any variable gets mean 0 and variance 1. Prove that μ and σ² are the mean and variance of N(μ, σ²), read a distribution’s lean and tails from E[Z³] and E[Z⁴], and see why the mean is the best guess under squared error.

The interactive

M06.7-X1 · Balance beam. A PMF on the values 0 to 6 stands as weights on a beam that balances at its mean, with a band one standard deviation either side. Raise or lower any bar and the rest rescale to keep the total 1; shift and stretch the variable and watch the mean follow while the band ignores the shift. It lives in Variance and standard deviation.