\( \newcommand{\vect}[1]{\mathbf{#1}} \newcommand{\mat}[1]{\mathbf{#1}} \newcommand{\R}{\mathbb{R}} \newcommand{\E}{\mathbb{E}} \newcommand{\Prob}{\mathbb{P}} \newcommand{\T}{^{\top}} \newcommand{\norm}[1]{\lVert #1 \rVert} \newcommand{\abs}[1]{\lvert #1 \rvert} \newcommand{\ihat}{\mkern1mu\hat{\boldsymbol{\imath}}\mkern2mu} \newcommand{\jhat}{\mkern1mu\hat{\boldsymbol{\jmath}}\mkern2mu} \newcommand{\pd}[2]{\frac{\partial #1}{\partial #2}} \newcommand{\argmax}{\operatorname*{arg\,max}} \newcommand{\argmin}{\operatorname*{arg\,min}} \newcommand{\cdata}[1]{\class{c-data}{#1}} \newcommand{\cmodel}[1]{\class{c-model}{#1}} \newcommand{\ctruth}[1]{\class{c-truth}{#1}} \newcommand{\cgrad}[1]{\class{c-grad}{#1}} \newcommand{\closs}[1]{\class{c-loss}{#1}} \newcommand{\cparam}[1]{\class{c-param}{#1}} \newcommand{\csecond}[1]{\class{c-second}{#1}} \renewcommand{\,}{\mkern3mu} \)

M04.9 · Differential equations, a first look

A free, illustrated module of 4 lessons and one interactive in M04 Calculus I: Single Variable: Equations about rates, and slope fields; Exponential growth and decay; Separable equations and the logistic curve; Euler’s method: solving by small steps.

M04.9 L1 · Core 4 lessons · about 100 min Live · in review

Part of M04 Calculus I: Single Variable in the Mathematics track.

Many laws of nature, and the training of a model, are rules about rates: how fast a quantity changes, given where it is now. These four lessons read such a rule as a field of slopes, solve the equations of growth, decay and cooling exactly, separate the variables to find the logistic curve, which turns out to be the sigmoid, and follow the slopes in small steps with Euler’s method, which on a loss is gradient descent.

What you need first. Derivatives of exponentials, the product rule and the chain rule (M04.3), gradient descent on the running loss (M04.4.4), antiderivatives and the constant C (M04.6), and substitution (M04.7.1).

Lessons

1. Equations about rates, and slope fields M04.9.1 25 min

   A differential equation says how fast a quantity changes in terms of where it is now, and its answer is a whole function. Check a proposed solution by substitution, pick one curve with a starting value, and draw the equation as a field of slopes that every solution follows.

2. Exponential growth and decay M04.9.2 25 min

   The equation y’ = k y says that a quantity changes at a rate proportional to its size. Prove that y(0) e^(kt) is its only solution, find doubling times and half-lives, shift it to model cooling, and watch gradient flow on the running loss close in on its minimum like e^(-15t).

3. Separable equations and the logistic curve M04.9.3 25 min

   When the rate is a function of t times a function of y, move every y to one side and every t to the other and integrate both. Solve y’ = t y, then logistic growth y’ = y(1 - y), whose solution through (0, 1/2) is exactly the sigmoid, and read stability from the sign of y’.

4. Euler’s method: solving by small steps M04.9.4 25 min

   When no formula solves a differential equation, follow its slope for a short time and look again. Derive Euler’s method from the tangent line, run it by hand on y’ = y, watch the error shrink in step with the step size, and find that on a loss it is gradient descent, with the same step-size limit.

The interactive

M04.9-X1 · Slope field and Euler stepper. Pick an equation, click to set the starting point on its slope field, and compare the exact solution with Euler’s straight steps as you change the step size. The read-out gives the error at the end and says when the steps start to oscillate or blow up. It lives in Euler’s method: solving by small steps.