\( \newcommand{\vect}[1]{\mathbf{#1}} \newcommand{\mat}[1]{\mathbf{#1}} \newcommand{\R}{\mathbb{R}} \newcommand{\E}{\mathbb{E}} \newcommand{\Prob}{\mathbb{P}} \newcommand{\T}{^{\top}} \newcommand{\norm}[1]{\lVert #1 \rVert} \newcommand{\abs}[1]{\lvert #1 \rvert} \newcommand{\ihat}{\mkern1mu\hat{\boldsymbol{\imath}}\mkern2mu} \newcommand{\jhat}{\mkern1mu\hat{\boldsymbol{\jmath}}\mkern2mu} \newcommand{\pd}[2]{\frac{\partial #1}{\partial #2}} \newcommand{\argmax}{\operatorname*{arg\,max}} \newcommand{\argmin}{\operatorname*{arg\,min}} \newcommand{\cdata}[1]{\class{c-data}{#1}} \newcommand{\cmodel}[1]{\class{c-model}{#1}} \newcommand{\ctruth}[1]{\class{c-truth}{#1}} \newcommand{\cgrad}[1]{\class{c-grad}{#1}} \newcommand{\closs}[1]{\class{c-loss}{#1}} \newcommand{\cparam}[1]{\class{c-param}{#1}} \newcommand{\csecond}[1]{\class{c-second}{#1}} \renewcommand{\,}{\mkern3mu} \)

M06.4 · Bayes’ rule

A free, illustrated module of 4 lessons and one interactive in M06 Probability I: Turning a conditional around; Base rates and frequency grids; Odds, likelihood ratios and updating twice; Many hypotheses: the Bayes table.

M06.4 L1 · Core 4 lessons · about 100 min Live · in review

Part of M06 Probability I in the Mathematics track.

Knowing how evidence behaves under each hypothesis is not the same as knowing which hypothesis the evidence points to. These four lessons turn a conditional probability around with Bayes’ rule, count it out on a grid of 1,000 people to see why a good test for a rare condition still gives mostly false alarms, recast the rule in odds so that each new piece of evidence multiplies them, and lay many hypotheses out in a Bayes table, which ends as a first spam classifier.

What you need first. Conditional probability, the multiplication rule and probability trees, the law of total probability and independence (M06.3), with fractions and a little Python (P01). The third lesson uses natural logarithms (M01.4).

Lessons

1. Turning a conditional around M06.4.1 25 min

   Knowing how often spam contains ‘free’ is not the same as knowing how often a ‘free’ email is spam. Derive Bayes’ rule in two lines, name its four parts, and turn one conditional probability into the other on the inbox.

2. Base rates and frequency grids M06.4.2 25 min

   A test that finds 90% of cases and wrongly flags 10% of healthy people says you are positive. The chance you have the condition is 1 in 12. Count it out on a grid of 1,000 people and see why the base rate decides the answer.

3. Odds, likelihood ratios and updating twice M06.4.3 25 min

   Write Bayes’ rule in odds and it becomes one multiplication: posterior odds equal prior odds times the likelihood ratio. Use it to update on a second positive test, then take logarithms and watch evidence add up, the way logistic regression does.

4. Many hypotheses: the Bayes table M06.4.4 25 min

   With several hypotheses, Bayes’ rule becomes a table: prior times likelihood in each row, then divide by the column total. Find which die rolled a 5, see through Monty Hall, and build a first naive Bayes spam classifier.

The interactive

M06.4-X1 · Frequency grid. A thousand people as a 25 by 40 grid, coloured by whether they have a condition and filled when they test positive. Move the base rate and the two rates of the test, gather the positives into one block, and read the posterior as true positives over all positives beside the same numbers in Bayes’ rule. It lives in Base rates and frequency grids.