Skip to main content

Products and quotients

The slope of a product is not the product of the slopes. A rectangle whose two sides grow shows what it is, and the quotient rule follows from it. Worked on x e^x, x^2 sin x and tan x, and on the sigmoid, whose slope sigma(1 - sigma) is the one logistic regression trains with.
M04.3.3L1 Core25 minIn review

Before this lesson

M04.3.1 Powers, sums and constant multiples M04.3.2 Exponentials, sine and cosine

After this lesson you can

  • Derive the product rule from a rectangle whose two sides grow, and explain why the corner piece vanishes in the limit.
  • Derive the quotient rule from the product rule.
  • Differentiate products and quotients such as x e^x, x^2 sin x and tan x.
  • Show that the sigmoid has slope sigma(z)(1 - sigma(z)), and explain why that slope is tiny in its flat tails.

In review. The maths and code on this page are checked by automated tests on every change, but an independent reader has not signed it off yet. Spot a mistake or a confusing step? Tell us.

The question

Logistic regression turns a score \(z\) into a probability with the sigmoid, which can be written \({\sigma(z) = e^z / (1 + e^z)}\). Training needs the slope of \(\sigma\) at every step, and \(\sigma\) is a quotient of two functions whose slopes we know. Is its slope the quotient of those slopes? That would give \({e^z / e^z = 1}\) everywhere, which cannot be right for a curve that levels off at 0 and 1. So what is it?

From the last lesson

1. Why is the slope of \(a^x\) at any \(x\) equal to \(a^x\) times its slope at 0?

Show answer

The difference quotient factors as \({a^x \cdot (a^h - 1)/h}\), and the second factor is the difference quotient at 0, which does not depend on \(x\).

2. State the derivatives of \(\sin x\) and \(\cos x\), with the condition they need.

Show answer

\({(\sin x)' = \cos x}\) and \({(\cos x)' = -\sin x}\), with \(x\) in radians.

Intuition

Read a product \({f(x)\,g(x)}\) as the area of a rectangle with sides \(f(x)\) and \(g(x)\). When \(x\) moves on to \({x + h}\), both sides change: \(f\) by an amount \({\Delta f = f(x+h) - f(x)}\) and \(g\) by \({\Delta g = g(x+h) - g(x)}\). The new rectangle is the old one plus three pieces.

A rectangle and three lines of text. The blue rectangle, labelled f g, has width f along the bottom and height g up the left side. When x moves by h, the width grows by delta f and the height by delta g. A green strip along the top is labelled f times delta g, a green strip down the right side is labelled g times delta f, and a small purple corner is labelled delta f delta g. Beside it: the area grows by f delta g plus g delta f, in green, plus delta f delta g, in purple; divide by h, let h shrink: the corner fades; and in green, f g prime equals f g prime plus g f prime.
M04.3.3-F1

The product \({f\,g}\) as an area. The two green strips are each one side times the other side’s change. The purple corner is a change times a change, and it fades away.

The strip along the top is \({f \cdot \Delta g}\): the old width times the growth in height. The strip down the side is \({g \cdot \Delta f}\): the old height times the growth in width. The corner is \({\Delta f \cdot \Delta g}\), thin in both directions. When \(h\) is small, each strip is about one \(h\) thick, but the corner is thin twice over, so it shrinks much faster than the strips. Per unit of \(h\), only the strips survive:

the slope of a product is each function’s slope times the other function, added.

That is the product rule. It is not the product of the two slopes: the rectangle grows by two strips, never by a single piece made of the two changes.

Build it up

Step 1. The exact change in area. Multiply out the new rectangle and take away the old one:

\[ \begin{aligned} &(f + \Delta f)(g + \Delta g) - f g \\ &\quad= f\,\Delta g + g\,\Delta f + \Delta f\,\Delta g . \end{aligned} \]

Here \(f\) and \(g\) are short for \(f(x)\) and \(g(x)\). Nothing is approximated yet.

Step 2. Divide by \(h\). The change in area per unit of \(h\) is

\[ \begin{aligned} &f \cdot \frac{\Delta g}{h} + g \cdot \frac{\Delta f}{h} \\ &\quad + \Delta f \cdot \frac{\Delta g}{h} . \end{aligned} \]

Step 3. Let \(h\) shrink. The quotients \({\Delta g / h}\) and \({\Delta f / h}\) are the difference quotients of \(g\) and \(f\), which tend to \(g'(x)\) and \(f'(x)\). In the corner term, \({\Delta g / h}\) tends to \(g'(x)\) while \(\Delta f\) itself tends to 0, because a function with a derivative is continuous (M04.2.3). So the corner term tends to \({0 \cdot g'(x) = 0}\).

Two rectangles of area 6, blue, each grown by two green strips and a purple corner. Left, titled Nudge h: the strips have areas 1.8 along the top and 1.6 down the side, and the corner 0.48. Right, titled Nudge h over 2: the strips are half as thick, with areas 0.9 and 0.8, and the corner is so small that its area, 0.12, is written beside it with a pointer. Under each: strips 1.8 plus 1.6, corner 0.48; and strips 0.9 plus 0.8, corner 0.12.
M04.3.3-F2

Halve the nudge and each strip halves, but the corner shrinks to a quarter. Divided by \(h\), the strips stay the same size and the corner’s share halves again and again, down to nothing.

Step 4. Check it on the square. With \({f = g = x}\), the rectangle is the square of lesson M04.3.1, and the rule gives \({(x \cdot x)' = 1 \cdot x + x \cdot 1 = 2x}\): the two strips of that picture.

Step 5. A quotient from a product. Write \({Q = f/g}\) where \({g(x) \neq 0}\). Then \({f = Q\,g}\), a product. Differentiate both sides with the product rule, \({f' = Q'\,g + Q\,g'}\), and solve for \(Q'\). That gives the quotient rule, derived in full below.

The formal statement

NoteThe product and quotient rules

If \(f\) and \(g\) have derivatives at \(x\), then

\[ \cgrad{(f\,g)' = f'\,g + f\,g'} , \]

and, where \({g(x) \neq 0}\),

\[ \cgrad{\Big(\frac{f}{g}\Big)' = \frac{f'\,g - f\,g'}{g^2}} . \]

The product rule. Steps 1 to 3 are the proof. Written with the limit laws of M04.1.2 in one place:

\[ \begin{aligned} &\frac{f(x+h)\,g(x+h) - f(x)\,g(x)}{h} \\ &\quad= f(x) \cdot \frac{\Delta g}{h} + g(x) \cdot \frac{\Delta f}{h} \\ &\qquad + \Delta f \cdot \frac{\Delta g}{h} \\ &\quad\to f(x)\,g'(x) + g(x)\,f'(x) \\ &\qquad + 0 \cdot g'(x) . \end{aligned} \]

The picture used positive sides that grow, but the algebra of Step 1 holds for any signs, so the rule does too.

The quotient rule. From \({f = Q\,g}\) and the product rule, \({f' = Q'\,g + Q\,g'}\). Take \({Q\,g'}\) to the other side, divide by \(g\), and put \({Q = f/g}\) back:

\[ \begin{aligned} Q' &= \frac{f' - Q\,g'}{g} \\ &= \frac{f' - (f/g)\,g'}{g} \\ &= \frac{f'\,g - f\,g'}{g^2} . \end{aligned} \]

The last step multiplies the top and the bottom by \(g\). Mind the order on top: the slope of the top comes first. Swap the two terms and the sign of the answer flips.

Go deeper: why f/g has a derivative at all

The derivation above differentiates \({f = Q\,g}\), which assumes that \(Q\) has a derivative. Here is why it does. First the reciprocal \({1/g}\), straight from the definition: put the two fractions over a common denominator.

\[ \begin{aligned} &\frac{1}{h}\Big(\frac{1}{g(x+h)} - \frac{1}{g(x)}\Big) \\ &\quad= -\frac{g(x+h) - g(x)}{h} \\ &\qquad \cdot \frac{1}{g(x+h)\,g(x)} \\ &\quad\to -g'(x) \cdot \frac{1}{g(x)^2} . \end{aligned} \]

The limit holds because \({g(x+h) \to g(x) \neq 0}\), again by continuity. So \({1/g}\) has the derivative \({-g'/g^2}\), and then \({f/g = f \cdot (1/g)}\) has one by the product rule. Working it out gives the same rule:

\[ \begin{aligned} &f' \cdot \frac{1}{g} + f \cdot \Big(-\frac{g'}{g^2}\Big) \\ &\quad= \frac{f'\,g - f\,g'}{g^2} . \end{aligned} \]

Worked example

Two products. With \({f = x}\) and \({g = e^x}\), and then with \({f = x^2}\) and \({g = \sin x}\):

\[ \begin{aligned} (x\,e^x)' &= 1 \cdot e^x + x \cdot e^x \\ &= (x + 1)\,e^x , \\ (x^2 \sin x)' &= 2x \sin x + x^2 \cos x . \end{aligned} \]

At \({x = 1}\) the first slope is \({2e \approx 5.437}\). At \({x = \pi/2}\) the second is \({\pi \cdot 1 + \tfrac{\pi^2}{4} \cdot 0}\), which is \({\pi \approx 3.14}\).

A quotient. For \({\tan x = \sin x / \cos x}\), the top \(\sin x\) has slope \(\cos x\) and the bottom \(\cos x\) has slope \({-\sin x}\) (lesson M04.3.2). The top of the quotient rule is then

\[ \begin{aligned} &\cos x \cdot \cos x - \sin x \cdot (-\sin x) \\ &\quad= \cos^2 x + \sin^2 x = 1 , \end{aligned} \]

so \({(\tan x)' = 1/\cos^2 x}\). At \({x = 0}\) the slope of \(\tan x\) is 1, and at \({x = \pi/4}\), where \({\cos^2 x = \tfrac12}\), it is 2.

The rectangle with numbers. For \({x\,e^x}\) at \({x = 1}\) with \({h = 0.1}\), the sides are 1 and \({e \approx 2.718}\). The height grows by \({e^{1.1} - e \approx 0.286}\) and the width by 0.1:

Left, titled To scale: a tall blue rectangle of width x equals 1 and height e to the x equals 2.718, labelled x e to the x equals 2.718. A green strip of height 0.286 sits on top, a thin green strip of width h equals 0.1 runs down the right side, pointed to as 0.272, and a tiny purple corner is pointed to as 0.029. Right, titled Nudge x equals 1 by h equals 0.1, six lines: top strip 1 times 0.286 equals 0.286; side strip 2.718 times 0.1 equals 0.272; corner 0.1 times 0.286 equals 0.029; growth 0.586 in all; per unit of h, 0.586 divided by 0.1 equals 5.86, in vermillion; and in bold green, in the limit 2e is about 5.44.
M04.3.3-F3

The worked example drawn to scale. Per unit of \(h\) the growth is 5.86, already close to the exact slope \({2e \approx 5.44}\); with \({h = 0.01}\) it is 5.48.

The strips add up to \({0.286 + 0.272 = 0.558}\) and the corner to 0.029. With \({h = 0.01}\) the strips are 0.0273 and 0.0272 and the corner only 0.00027: making \(h\) ten times smaller made the strips ten times smaller and the corner a hundred times smaller.

The sigmoid. Multiply the top and bottom of \({1/(1 + e^{-z})}\) by \(e^z\) to get \({\sigma(z) = e^z / (1 + e^z)}\). The top \(e^z\) and the bottom \({1 + e^z}\) both have slope \(e^z\), so the quotient rule gives

\[ \begin{aligned} \sigma'(z) &= \frac{e^z\,(1 + e^z) - e^z \cdot e^z}{(1 + e^z)^2} \\ &= \frac{e^z}{(1 + e^z)^2} \\ &= \frac{e^z}{1 + e^z} \cdot \frac{1}{1 + e^z} . \end{aligned} \]

The first factor is \(\sigma(z)\). The second is \({1 - \sigma(z)}\), because \({1 - \tfrac{e^z}{1 + e^z} = \tfrac{1}{1 + e^z}}\). So

\[ \cgrad{\sigma'(z) = \sigma(z)\,\big(1 - \sigma(z)\big)} . \]

At \({z = 0}\), \({\sigma = \tfrac12}\) and the slope is \({\tfrac12 \cdot \tfrac12 = \tfrac14}\), its largest value. At \({z = 2}\), \({\sigma \approx 0.881}\) and the slope is about 0.105; at \({z = 5}\) it is about 0.0066. This is the identity that logistic regression trains with (lesson ML01.6.3), where it is derived a second way, with the chain rule of the next lesson.

A graph with z from minus 7 to 7. The blue S-shaped sigmoid rises from 0 to 1 through the blue dot at z equals 0, height 0.5, where a green tangent line touches it. Below it, the green bell-shaped slope curve, sigma prime of z equals sigma of z times 1 minus sigma of z, peaks at the green dot at height 0.25 above z equals 0, labelled largest slope 1 over 4 at z equals 0, and falls almost to 0 on both sides, where pointers mark a flat tail on the left and a flat tail with slope 0.0066 at z equals 5 on the right.
M04.3.3-F4

The sigmoid (blue) and its slope (green). The slope is largest, \(\tfrac14\), in the middle and almost 0 in the flat tails.

Code

From scratch, the three pieces of the growing rectangle for \({x\,e^x}\) at \({x = 1}\), for shrinking \(h\):

import math

def pieces(f, g, x, h):
    """The two strips and the corner by which the rectangle f(x) by g(x) grows when x moves to x + h."""
    df, dg = f(x + h) - f(x), g(x + h) - g(x)
    return f(x) * dg, g(x) * df, df * dg

for h in (0.1, 0.01, 0.001):
    top, side, corner = pieces(lambda x: x, math.exp, 1.0, h)
    print(f"h = {h:<6} strips {top:.5f} + {side:.5f}   corner {corner:.6f}   "
          f"per unit of h {(top + side + corner) / h:.4f}")
print("the product rule at x = 1: (x + 1) e^x =", round(2 * math.e, 4))
h = 0.1    strips 0.28588 + 0.27183   corner 0.028588   per unit of h 5.8630
h = 0.01   strips 0.02732 + 0.02718   corner 0.000273   per unit of h 5.4775
h = 0.001  strips 0.00272 + 0.00272   corner 0.000003   per unit of h 5.4406
the product rule at x = 1: (x + 1) e^x = 5.4366

Every rule of the worked example against a central difference (M04.2.4):

import numpy as np

def central(f, x, h=1e-5):
    return (f(x + h) - f(x - h)) / (2 * h)

def sigmoid(z):
    return 1 / (1 + np.exp(-z))

checks = {
    "x e^x": (lambda x: x * np.exp(x), lambda x: (x + 1) * np.exp(x)),
    "x^2 sin x": (lambda x: x ** 2 * np.sin(x), lambda x: 2 * x * np.sin(x) + x ** 2 * np.cos(x)),
    "tan x": (np.tan, lambda x: 1 / np.cos(x) ** 2),
    "sigmoid": (sigmoid, lambda z: sigmoid(z) * (1 - sigmoid(z))),
}
x = np.linspace(-1.2, 1.2, 9)
for name, (f, rule) in checks.items():
    print(f"{name:10} largest gap to the rule {np.max(np.abs(central(f, x) - rule(x))):.1e}")

z = np.array([0.0, 2.0, 5.0])
print("slope of the sigmoid at z = 0, 2, 5:", np.round(sigmoid(z) * (1 - sigmoid(z)), 4))
x e^x      largest gap to the rule 2.8e-10
x^2 sin x  largest gap to the rule 1.0e-10
tan x      largest gap to the rule 5.4e-09
sigmoid    largest gap to the rule 6.7e-12
slope of the sigmoid at z = 0, 2, 5: [0.25   0.105  0.0066]

SciPy names the sigmoid scipy.special.expit. Its slope from scipy.differentiate.derivative matches \({\sigma(1 - \sigma)}\):

from scipy.special import expit
from scipy.differentiate import derivative

z = np.linspace(-6, 6, 13)
result = derivative(expit, z)
print(f"largest gap to sigma (1 - sigma): {np.max(np.abs(result.df - expit(z) * (1 - expit(z)))):.1e}")
print("slope at 0:", round(float(result.df[6]), 6))
largest gap to sigma (1 - sigma): 8.0e-14
slope at 0: 0.25

Explore

The module’s interactive, the chain-rule gears, is in lesson M04.3.4. Choose the outer function \({y = \sigma(u)}\) there and turn the input: its gear ratio \({dy/du}\) is the slope found in this lesson, \({\sigma(1 - \sigma)}\), and it never passes 0.25. A sigmoid gear always slows the turn down.

Pitfalls

Two panels showing the blue curve tan x for x from 0 to about 1.33, rising ever more steeply, with a dot at x equals pi over 4, height 1. Left, marked with a cross and titled Slopes divided: a vermillion line through the dot falls from left to right, labelled slope of top divided by slope of bottom equals minus 1. Right, marked with a tick and titled Quotient rule: a green line of slope 2 touches the curve at the dot, labelled slope 1 over cosine squared x equals 2.
M04.3.3-F5

Wrong beside right. Dividing the slope of \(\sin x\) by the slope of \(\cos x\) gives a falling line, though \(\tan x\) rises everywhere. The quotient rule gives slope 2 at \({\pi/4}\).

“The slope of a product is the product of the slopes.”

Test it on \({x \cdot x}\): the slopes are 1 and 1, and their product is 1, but \(x^2\) has slope \(2x\). For \({x\,e^x}\) at \({x = 1}\) the false rule gives \({1 \cdot e \approx 2.72}\) instead of \({2e \approx 5.44}\). The rectangle shows why: it grows by two strips, each a change times the other side.

“The slope of a quotient is the quotient of the slopes.”

For the sigmoid that would be \({e^z/e^z = 1}\) at every \(z\), a straight line. For \(\tan x\) it would be \({\cos x / (-\sin x)}\), which is \(-1\) at \({\pi/4}\), a negative slope for a rising curve. Use the quotient rule, and start its top with the slope of the top.

“If the sigmoid is flat, the model must be right.”

A flat sigmoid only means that the score is far from 0. A positive example scored \({z = -5}\) gets \({\sigma \approx 0.0067}\), a confident and wrong prediction, and the slope there is about 0.0066, so a gradient step through the sigmoid alone barely moves. This is saturation, a first look at a problem that deep learning returns to (DL01.3). Lesson ML01.6.3 shows how the log loss cancels the small factor.

Check yourself

Recall

1. State the product rule.

Show answer

\({(f\,g)' = f'\,g + f\,g'}\): each function’s slope times the other function, added.

2. In the growing rectangle, which piece vanishes in the limit, and why?

Show answer

The corner \({\Delta f\,\Delta g}\). Per unit of \(h\) it is \({\Delta f \cdot (\Delta g/h)}\), which tends to \({0 \cdot g'(x) = 0}\), because \(\Delta f\) tends to 0.

3. State the quotient rule, with its condition.

Show answer

\({(f/g)' = (f'\,g - f\,g')/g^2}\), wherever \({g(x) \neq 0}\).

4. What is the derivative of \(\tan x\), and its value at \({\pi/4}\)?

Show answer

\({1/\cos^2 x}\), which is 2 at \({\pi/4}\).

5. Write the slope of the sigmoid in terms of \(\sigma\). Where is it largest, and how large?

Show answer

\({\sigma'(z) = \sigma(z)\,(1 - \sigma(z))}\). It is largest at \({z = 0}\), where it is \(\tfrac14\).

Exercises

1. ★ Differentiate \({x^3 e^x}\) and find its slope at \({x = 1}\).

Show solution

Product rule with \({f = x^3}\) and \({g = e^x}\): \({3x^2 e^x + x^3 e^x = (3x^2 + x^3)\,e^x}\). At \({x = 1}\): \({4e \approx 10.87}\).

2. ★★ Use the quotient rule to differentiate \({1/x}\) and \({x/(1 + x^2)}\). Where does the second have slope 0?

Show solution

For \({1/x}\) the top is 1, with slope 0: \({(0 \cdot x - 1 \cdot 1)/x^2 = -1/x^2}\), as M04.2.3 found from the definition. For \({x/(1 + x^2)}\):

\[ \begin{aligned} &\Big(\frac{x}{1 + x^2}\Big)' \\ &\quad= \frac{1 \cdot (1 + x^2) - x \cdot 2x}{(1 + x^2)^2} \\ &\quad= \frac{1 - x^2}{(1 + x^2)^2} . \end{aligned} \]

The bottom is never 0, so the slope is 0 exactly where \({1 - x^2 = 0}\): at \({x = 1}\) and \({x = -1}\).

3. ★★★ The activation function \(\tanh\) is

\[ \tanh z = \frac{e^z - e^{-z}}{e^z + e^{-z}} . \]

(a) Use the quotient rule on \({1/e^z}\) to show that the slope of \(e^{-z}\) is \({-e^{-z}}\). (b) Show that \({\tanh'(z) = 1 - \tanh^2 z}\). (c) Where is the slope largest, and how large is it?

Show solution

(a) Since \({e^{-z} = 1/e^z}\), the quotient rule gives

\[ \begin{aligned} \Big(\frac{1}{e^z}\Big)' &= \frac{0 \cdot e^z - 1 \cdot e^z}{(e^z)^2} \\ &= -\frac{1}{e^z} = -e^{-z} . \end{aligned} \]

(b) Write \({p = e^z - e^{-z}}\) and \({q = e^z + e^{-z}}\). By part (a), \({p' = e^z + e^{-z} = q}\) and \({q' = e^z - e^{-z} = p}\). So

\[ \begin{aligned} \tanh'(z) &= \frac{p'\,q - p\,q'}{q^2} \\ &= \frac{q^2 - p^2}{q^2} \\ &= 1 - \Big(\frac{p}{q}\Big)^2 \\ &= 1 - \tanh^2 z . \end{aligned} \]

(c) A square is never negative, so the slope is at most 1, and it equals 1 only where \({\tanh z = 0}\), at \({z = 0}\): four times the sigmoid’s largest slope.

Summary card

Each side’s change times the other side, added.

The anchor figure again: a blue rectangle f by g grown by a green strip f delta g along the top, a green strip g delta f down the side and a small purple corner delta f delta g, with the product rule beside it.
M04.3.3-F1

Two strips survive; the corner fades.

  • Product: \({(f\,g)' = f'\,g + f\,g'}\), so \({(x\,e^x)' = (x + 1)\,e^x}\).
  • Quotient: \({(f/g)' = (f'\,g - f\,g')/g^2}\), so \({(\tan x)' = 1/\cos^2 x}\).
  • Sigmoid: \({\sigma' = \sigma\,(1 - \sigma)}\), at most \(\tfrac14\), at \({z = 0}\).

Sources and next steps

Sources. Each was opened and checked against this lesson on 12 November 2026.

  • Gilbert Strang and Edwin Herman, Calculus Volume 1, OpenStax: Section 3.3 proves Theorem 3.5, the product rule, from the definition by adding and subtracting \({f(x)\,g(x+h)}\) and using the continuity of \(g\), and states Theorem 3.6, the quotient rule, with its proof omitted; Section 3.5 differentiates \(\tan x\) with the quotient rule and the identity \({\cos^2 x + \sin^2 x = 1}\) (Theorem 3.9).
  • Matthew Boelkins, Active Calculus, Section 2.3, The product and quotient rules. Warns that the derivative of a product is not in general the product of the derivatives, and derives the quotient rule by writing \({f = Q\,g}\) and applying the product rule, as here.
  • Ian Goodfellow, Yoshua Bengio and Aaron Courville, Deep Learning, Chapter 3, Section 3.10. Writes the sigmoid as \({\exp(x)/(\exp(x) + \exp(0))}\) (equation 3.33), gives its derivative \({\sigma(x)(1 - \sigma(x))}\) (equation 3.34), and notes that it saturates, becoming flat and insensitive to its input, when the input is very positive or very negative.
  • Marc Peter Deisenroth, A. Aldo Faisal and Cheng Soon Ong, Mathematics for Machine Learning, Section 5.1.2: the product rule (equation 5.29) and the quotient rule (equation 5.30).

Next. M04.3.4 The chain rule handles the last way of building functions, one inside another, and finds the slope of \(\sigma(wx)\) as a weight \(w\) moves.

This lesson unlocks

M04.3.4 The chain rule ML01.6.3 Training by gradient descent

Your progress is saved in this browser only. No account, no tracking.
Back to top