\( \newcommand{\vect}[1]{\boldsymbol{\mathbf{#1}}} \newcommand{\mat}[1]{\boldsymbol{\mathbf{#1}}} \newcommand{\R}{\mathbb{R}} \newcommand{\E}{\mathbb{E}} \newcommand{\Prob}{\mathbb{P}} \newcommand{\T}{^{\top}} \newcommand{\norm}[1]{\lVert #1 \rVert} \newcommand{\abs}[1]{\lvert #1 \rvert} \newcommand{\ihat}{\mkern1mu\hat{\boldsymbol{\imath}}\mkern2mu} \newcommand{\jhat}{\mkern1mu\hat{\boldsymbol{\jmath}}\mkern2mu} \newcommand{\pd}[2]{\frac{\partial #1}{\partial #2}} \newcommand{\argmax}{\operatorname*{arg\,max}} \newcommand{\argmin}{\operatorname*{arg\,min}} \newcommand{\cdata}[1]{\class{c-data}{#1}} \newcommand{\cmodel}[1]{\class{c-model}{#1}} \newcommand{\ctruth}[1]{\class{c-truth}{#1}} \newcommand{\cgrad}[1]{\class{c-grad}{#1}} \newcommand{\closs}[1]{\class{c-loss}{#1}} \newcommand{\cparam}[1]{\class{c-param}{#1}} \newcommand{\csecond}[1]{\class{c-second}{#1}} \renewcommand{\,}{\mkern3mu} \)

M05.4 · Jacobians and the multivariable chain rule

A free, illustrated module of 4 lessons and one interactive in M05 Multivariable and Matrix Calculus: The Jacobian: the best local linear map; The chain rule through intermediate variables; Composition multiplies Jacobians; The Jacobian determinant and local inverses.

M05.4 L2 · Intermediate 4 lessons · about 100 min Live · in review

Part of M05 Multivariable and Matrix Calculus in the Mathematics track.

A map with many inputs and many outputs bends space, but near any one point it acts like a linear map: the Jacobian, a matrix whose rows are the gradients of the outputs and whose columns say where a small step along each input goes. These four lessons build that matrix, use it to derive the chain rule for variables that depend on other variables, show that composing maps multiplies their Jacobians, which is why a network’s derivative is a product of layer derivatives, and read the Jacobian determinant as the factor by which a map scales small areas, the number that decides when a map can be undone near a point.

What you need first. Partial derivatives, gradients and the linear approximation of a function of several inputs (M05.2), matrices as linear maps (M02.4), matrix multiplication as composition (M02.5) and determinants as area factors (M02.8). The chain rule in one variable (M04.3.4) is used throughout. The code uses NumPy and SciPy, explaining each call.

Lessons

1. The Jacobian: the best local linear map M05.4.1 25 min

   A map with several inputs and several outputs bends the grid, but close to any point it acts like a matrix. Meet the Jacobian: its rows are the gradients of the outputs, its columns say where a small step along each input goes.

2. The chain rule through intermediate variables M05.4.2 25 min

   When the inputs of a function themselves depend on other variables, the rate of change adds one product per path. Derive the multivariable chain rule from the linear approximation, draw it as a tree, and use it on the running bowl and the running loss.

3. Composition multiplies Jacobians M05.4.3 25 min

   Do one map after another and the local linear maps compose too, so the Jacobian of the composite is the product of the Jacobians. Derive the general chain rule, check its shapes, and use it to differentiate a small neural network layer by layer.

4. The Jacobian determinant and local inverses M05.4.4 25 min

   The determinant of the Jacobian is the factor by which a smooth map scales small areas near a point. Measure it for polar coordinates and the running warp, and see why a non-zero determinant lets a map be undone near a point, with the inverse matrix as the inverse’s Jacobian.

The interactive

M05.4-X1 · Grid warp. Choose the linear map T, polar coordinates or the running warp and watch the grid bend. Drag a point to place a small square there: its exact image is drawn beside the parallelogram spanned by the two Jacobian columns, and as you shrink the square the two become one shape. The read-out gives the Jacobian, its determinant and the measured ratio of areas. It lives in The Jacobian: the best local linear map.