\( \newcommand{\vect}[1]{\mathbf{#1}} \newcommand{\mat}[1]{\mathbf{#1}} \newcommand{\R}{\mathbb{R}} \newcommand{\E}{\mathbb{E}} \newcommand{\Prob}{\mathbb{P}} \newcommand{\T}{^{\top}} \newcommand{\norm}[1]{\lVert #1 \rVert} \newcommand{\abs}[1]{\lvert #1 \rvert} \newcommand{\ihat}{\mkern1mu\hat{\boldsymbol{\imath}}\mkern2mu} \newcommand{\jhat}{\mkern1mu\hat{\boldsymbol{\jmath}}\mkern2mu} \newcommand{\pd}[2]{\frac{\partial #1}{\partial #2}} \newcommand{\argmax}{\operatorname*{arg\,max}} \newcommand{\argmin}{\operatorname*{arg\,min}} \newcommand{\cdata}[1]{\class{c-data}{#1}} \newcommand{\cmodel}[1]{\class{c-model}{#1}} \newcommand{\ctruth}[1]{\class{c-truth}{#1}} \newcommand{\cgrad}[1]{\class{c-grad}{#1}} \newcommand{\closs}[1]{\class{c-loss}{#1}} \newcommand{\cparam}[1]{\class{c-param}{#1}} \newcommand{\csecond}[1]{\class{c-second}{#1}} \renewcommand{\,}{\mkern3mu} \)

M04.10 · Partial derivatives, a first look

A free, illustrated module of 4 lessons and one interactive in M04 Calculus I: Single Variable: Two inputs, one output: surfaces and contour lines; Partial derivatives: the slope when one input moves; The gradient: the direction of steepest climb; The lowest point of a bowl: the least-squares line.

M04.10 L1 · Core 4 lessons · about 100 min Live · in review

Part of M04 Calculus I: Single Variable in the Mathematics track.

A loss with two parameters is a landscape, not a curve. These four lessons show how to draw a function of two inputs as a surface and as a contour map, take the slope along one input while the other stands still, put the two slopes together into the gradient, the arrow of steepest climb, and find the lowest point of a bowl by making both slopes zero. The last lesson solves the least-squares line of four points that way, the result that Machine Learning Foundations trains towards.

What you need first. The derivative as a slope and the linear approximation (M04.2), the rules of differentiation with the chain rule (M04.3), and critical points and Fermat’s theorem (M04.4). Nothing from M04.5 to M04.9 is used, so this module may be read straight after M04.4. The code uses NumPy and a little SciPy, explaining each call.

Lessons

1. Two inputs, one output: surfaces and contour lines M04.10.1 25 min

   A function of two inputs gives a height at every point of the plane. Draw it as a surface or, from above, as contour lines like a hiking map, cut slices through it by freezing one input, and read a loss with two parameters the same way.

2. Partial derivatives: the slope when one input moves M04.10.2 25 min

   Freeze one input of a function of two variables and an ordinary curve is left, with an ordinary slope. That slope is a partial derivative. Find it from the definition and by the rules of differentiation, and check it numerically.

3. The gradient: the direction of steepest climb M04.10.3 25 min

   Put the two partial derivatives side by side and they make an arrow on the map: the gradient. It points the way a function climbs fastest, its length is that fastest slope, and at right angles to it, along the contour line, the function stays level.

4. The lowest point of a bowl: the least-squares line M04.10.4 25 min

   At the lowest point of a smooth surface both partial derivatives are zero. Set them to zero for the mean squared error of a line through four points, solve the two equations, and prove the answer is the lowest point: the least-squares line y-hat = 1.4x + 0.5.

The interactive

M04.10-X1 · Slope compass. Drag a point over the contour map of the running bowl, the two-parameter loss or a saddle. Two side panels show the slices through the point with their tangent lines, the gradient arrow points the way of steepest climb, and a direction dial reads off the slope in any direction you choose: largest along the arrow, zero along the contour line, zero every way at a flat point. It lives in The gradient: the direction of steepest climb.