\( \newcommand{\vect}[1]{\mathbf{#1}} \newcommand{\mat}[1]{\mathbf{#1}} \newcommand{\R}{\mathbb{R}} \newcommand{\E}{\mathbb{E}} \newcommand{\Prob}{\mathbb{P}} \newcommand{\T}{^{\top}} \newcommand{\norm}[1]{\lVert #1 \rVert} \newcommand{\abs}[1]{\lvert #1 \rvert} \newcommand{\ihat}{\mkern1mu\hat{\boldsymbol{\imath}}\mkern2mu} \newcommand{\jhat}{\mkern1mu\hat{\boldsymbol{\jmath}}\mkern2mu} \newcommand{\pd}[2]{\frac{\partial #1}{\partial #2}} \newcommand{\argmax}{\operatorname*{arg\,max}} \newcommand{\argmin}{\operatorname*{arg\,min}} \newcommand{\cdata}[1]{\class{c-data}{#1}} \newcommand{\cmodel}[1]{\class{c-model}{#1}} \newcommand{\ctruth}[1]{\class{c-truth}{#1}} \newcommand{\cgrad}[1]{\class{c-grad}{#1}} \newcommand{\closs}[1]{\class{c-loss}{#1}} \newcommand{\cparam}[1]{\class{c-param}{#1}} \newcommand{\csecond}[1]{\class{c-second}{#1}} \renewcommand{\,}{\mkern3mu} \)

ML06 · Statistical Learning Theory

A free, illustrated L4 course of 10 modules in the Machine Learning track. Modules include: The learning problem stated formally; PAC learning; VC dimension.

ML06 L4 · Expert 10 modules · about 40 lessons Planned · Ring 4

Part of the Machine Learning track. Ships in Ring 4 (Expert) of the roadmap.

Modules

1. The learning problem stated formally
2. PAC learning
3. VC dimension
4. Rademacher complexity
5. Uniform convergence and generalization bounds
6. Stability and regularization
7. Online learning and regret
8. Kernel theory
9. Boosting theory and margins
10. Modern puzzles: double descent, benign overfitting, implicit bias

Signature illustrations

The pictures a learner should still see in their head a year later:

Shattering diagrams, plots of how tight each bound is, the double-descent curve with its regimes labeled, regret curves.

Status

This course is planned, not written yet. It ships with Ring 4 (Expert).

Every lesson will follow the same twelve-part blueprint with at least three purpose-made figures. Watch the repository to hear when modules ship, or help write and review it.