Skip to main content

Exponential growth and decay

The equation y’ = k y says that a quantity changes at a rate proportional to its size. Prove that y(0) e^(kt) is its only solution, find doubling times and half-lives, shift it to model cooling, and watch gradient flow on the running loss close in on its minimum like e^(-15t).
M04.9.2L1 Core25 minIn review

Before this lesson

M04.9.1 Equations about rates, and slope fields M04.4.4 Gradient descent and Newton's method in one dimension

After this lesson you can

  • Solve y’ = k y with a given starting value, and prove that y(0) e^(kt) is the only solution by differentiating y e^(-kt).
  • Compute a doubling time or a half-life as ln 2 divided by the size of k, and find k from two measurements.
  • Solve y’ = -k (y - c) by shifting, and use it to model cooling towards the temperature of a room.
  • Write gradient flow on the running loss as w’ = -15 (w - 47/30), solve it, and explain why it is gradient descent with infinitely small steps.

In review. The maths and code on this page are checked by automated tests on every change, but an independent reader has not signed it off yet. Spot a mistake or a confusing step? Tell us.

The question

Train the one-weight model of the running loss by gradient descent, and keep shrinking the learning rate while taking proportionally more steps: 10 steps of 0.01, then 100 steps of 0.001, then 1,000 steps of 0.0001. The weight at the end settles on one value, and its whole path settles on one smooth curve. Which curve? It turns out to be the curve of a cooling cup of tea and of a decaying radioactive sample, and a two-line argument shows that no other curve is possible.

From the last lesson

1. How do you check that \({y = 1 - e^{-t}}\) solves \({y' = 1 - y}\)?

Show answer

Work out both sides: \({y' = e^{-t}}\) and \({1 - y = e^{-t}}\). They agree for every \(t\).

2. What does an initial condition do?

Show answer

It picks one solution out of the family, by fixing the constant.

Intuition

The equation \({y' = ky}\), with a constant \(k\), says that a quantity changes at a rate proportional to its own size. Money earning interest does this, and so does a population with room to spare. A sample of radioactive atoms does it with \({k < 0}\): the more atoms there are, the more decay each second.

Two panels. Left, titled Growth, k greater than 0: a black curve rises from 1 at t equals 0 through black dots labelled 2 at t equals 2, 4 at t equals 4 and 8 at t equals 6, with dashed lines down to the time axis. Green tangent segments touch the curve at heights 2 and 4, labelled slope 0.69 and slope 1.39. Notes read every 2 units of time: times 2, and twice as high, twice as steep. Right, titled Decay, k less than 0: a black curve falls from 8 at t equals 0 through dots labelled 4, 2 and 1 at t equals 2, 4 and 6, with green tangent segments labelled slope minus 1.39 at height 4 and slope minus 0.69 at height 2, and the note every 2 units: times one half.
M04.9.2-F1

\({y' = ky}\) with \({k = \pm\ln 2/2}\). Equal times bring equal factors: the growing curve doubles every 2 units of time and the decaying one halves. The slope is \(k\) times the height, so twice as high means twice as steep.

Two features of the picture are the whole story. Over any 2 units of time the quantity is multiplied by the same factor, wherever you start. And the slope is proportional to the height: at height 4 the curve is twice as steep as at height 2. That is what makes exponential growth run away, and exponential decay level off.

Build it up

Step 1. Guess a solution. We need a function whose derivative is \(k\) times itself. The exponential \({e^t}\) is its own derivative (M04.3.2), and by the chain rule (M04.3.4)

\[ \big(e^{kt}\big)' = k\,e^{kt} . \]

A constant multiple keeps the property: \({(Ce^{kt})' = k\,(Ce^{kt})}\). At \({t = 0}\) the function \({Ce^{kt}}\) equals \(C\), so the constant is the starting value, and \({y = y(0)\,e^{kt}}\).

Step 2. Equal times, equal factors. Over a stretch of time \(T\) the solution changes to

\[ \begin{aligned} y(t + T) &= y(0)\,e^{kt}\,e^{kT} \\ &= y(t)\,e^{kT} , \end{aligned} \]

so it is multiplied by \({e^{kT}}\), whatever \(t\) is. The factor is 2 when \({kT = \ln 2}\). So the doubling time is \({\ln 2/k}\), and for \({k < 0}\) the half-life is \({\ln 2/\abs{k}}\).

Step 3. Approach to a level. Tea does not cool to 0 °C; it cools towards the room. Newton’s law of cooling says the rate is proportional to the difference from the room temperature \(c\):

\[ y' = -k\,(y - c), \qquad k > 0 . \]

Shift the unknown: let \({u = y - c}\). Then \({u' = y'}\), because \(c\) is constant, so \({u' = -ku}\). By Step 1, \({u = u(0)\,e^{-kt}}\), and adding \(c\) back gives

\[ y = c + \big(y(0) - c\big)\,e^{-kt} . \]

The gap \({y - c}\) decays exponentially and halves every \({\ln 2/k}\).

Step 4. Gradient flow. Gradient descent moves a weight by \({-\eta\,L'(w)}\) in one step (M04.4.4). Count each step as taking \(\eta\) units of time. Then the weight moves at speed \({-L'(w)}\), and as the steps shrink its path tends to the solution of the gradient flow

\[ w' = -L'(w) . \]

For the running loss \({L'(w) = 15w - 23.5}\), which is \({15\,(w - 47/30)}\), so the flow is \({w' = -15\,(w - 47/30)}\). That is Step 3 with \({k = 15}\) and \({c = 47/30}\):

\[ \begin{aligned} w(t) &= \frac{47}{30} \\ &\quad + \Big(w(0) - \frac{47}{30}\Big)\,e^{-15t} . \end{aligned} \]

Two panels. Left, titled Downhill at speed minus L prime of w: the purple parabola of the running loss L against w from 0 to 2.2, with an ink star at its lowest point near w equals 1.57. Orange dots on the parabola mark where the weight is every 0.02 units of time, starting from a large dot at w equals 0, labelled start w equals 0, high on the left: far apart where the curve is steep, then bunched at the bottom next to the star. Right, titled Closing in like e to the minus 15 t: a black curve w of t rises from 0 at t equals 0 towards a dashed level line labelled w equals 47 over 30, with the same orange dots on it. A bracket at t equals 0.06 measures the distance to the level, labelled distance to 47 over 30: 47 over 30 times e to the minus 15 t, and a dashed vertical line at t equals ln 2 over 15, about 0.046, is marked half way.
M04.9.2-F2

Gradient flow on the running loss from \({w = 0}\). Left: the weight slides down the loss, fast where it is steep and slowly near the bottom. Right: the same run against time, closing in on \({47/30}\).

The formal statement

NoteTheorem: the solutions of y′ = k y

Let \(k\) be a constant. A differentiable function \(y\) on an interval containing 0 satisfies \({y' = ky}\) if and only if

\[ y(t) = y(0)\,e^{kt} . \]

Proof. If \({y = y(0)\,e^{kt}}\), then \({y' = k\,y(0)\,e^{kt} = ky}\) by the chain rule. For the other direction, let \(y\) be any solution and set \({u(t) = y(t)\,e^{-kt}}\). By the product rule (M04.3.3) and the chain rule,

\[ \begin{aligned} u' &= y'\,e^{-kt} - k\,y\,e^{-kt} \\ &= (y' - ky)\,e^{-kt} \\ &= 0 , \end{aligned} \]

because \({y' = ky}\). A function with zero derivative on an interval is constant (M04.6.3, from the mean value theorem), so \({u(t) = u(0) = y(0)}\) for every \(t\). Multiplying by \({e^{kt}}\) gives \({y(t) = y(0)\,e^{kt}}\). ∎

Two results follow at once, by Steps 2 and 3. The doubling time of \({y' = ky}\) with \({k > 0}\), and the half-life with \({k < 0}\), are both \({\ln 2/\abs{k}}\), whatever the starting value. The equation \({y' = -k\,(y - c)}\) has exactly the solutions \({c + (y(0) - c)\,e^{-kt}}\), because the shifted quantity \({y - c}\) solves \({u' = -ku}\) and the theorem applies to it.

Along a gradient flow the loss never rises. By the chain rule,

\[ \begin{aligned} \frac{d}{dt}\,L\big(w(t)\big) &= L'(w)\,w' \\ &= -L'(w)^2 \le 0 . \end{aligned} \]

This is the continuous form of the fact, proved in M04.4.4, that a small step against the slope goes downhill. For the running loss, \({L(w) = 7.5\,(w - 47/30)^2 + L^*}\) with \({L^* = 11/120}\), the lowest value. So along the flow the gap above the lowest value is

\[ \begin{aligned} L - L^* &= 7.5\,\Big(w(0) - \frac{47}{30}\Big)^2 e^{-30t} , \end{aligned} \]

which shrinks twice as fast as the distance \({w - 47/30}\).

Go deeper: why gradient descent with tiny steps follows the flow

One step of gradient descent with learning rate \(\eta\) takes \(w\) to \({w - \eta\,(15w - 23.5)}\). Subtract \({47/30}\) from both, using \({15w - 23.5 = 15\,(w - 47/30)}\):

\[ \begin{aligned} &w - 15\eta\,(w - \tfrac{47}{30}) - \tfrac{47}{30} \\ &\quad = (1 - 15\eta)\,(w - \tfrac{47}{30}) , \end{aligned} \]

so each step multiplies the distance by \({1 - 15\eta}\), as in ML01.5.3. Reaching time \(t\) takes \({t/\eta}\) steps, so the distance is multiplied by \({(1 - 15\eta)^{t/\eta}}\). Take its logarithm and rearrange:

\[ \begin{aligned} &\frac{t}{\eta}\,\ln(1 - 15\eta) \\ &\quad = -15t \cdot \frac{\ln(1 - 15\eta)}{-15\eta} . \end{aligned} \]

The last fraction has the form \({\ln(1 + x)/x}\) with \({x = -15\eta}\). That is the difference quotient of \(\ln\) at 1 with step \(x\), so as \({\eta \to 0}\) it tends to the slope of \(\ln\) at 1, which is \({1/1 = 1}\) (M04.3.4). The logarithm therefore tends to \({-15t}\), and the factor tends to \({e^{-15t}}\). At \({t = 0.1}\) the factors for \({\eta = 0.01}\), 0.001 and 0.0001 are 0.1969, 0.2206 and 0.2229, closing in on \({e^{-1.5} \approx 0.2231}\).

Frontier note: momentum as a differential equation

Gradient descent becomes the first-order flow \({w' = -L'(w)}\) in the limit of small steps. Su, Boyd and Candès (2016) took the same limit for Nesterov’s accelerated method, a gradient method with momentum, and found a second-order equation, \({X'' + \frac{3}{t}X' + \nabla f(X) = 0}\) (their equation 3), in which a unit of time stands for about \({1/\sqrt{s}}\) steps of size \(s\). Studying the equation, rather than the steps, explains why momentum methods overshoot and oscillate on their way to a minimum.

Worked example

A cup of tea is poured at 90 °C in a room at 20 °C, and reads 60 °C ten minutes later (an invented example). Find \(k\), the temperature after 30 minutes, and the time at which the tea reaches 40 °C.

The model. By Step 3 with \({c = 20}\) and \({y(0) = 90}\), the temperature after \(t\) minutes is

\[ y(t) = 20 + 70\,e^{-kt} . \]

Find k from the second reading. Putting in \({y(10) = 60}\) gives \({60 = 20 + 70\,e^{-10k}}\), so

\[ \begin{aligned} e^{-10k} &= \frac{40}{70} = \frac{4}{7} , \\ k &= \frac{\ln(7/4)}{10} \approx 0.0560 . \end{aligned} \]

That is per minute. The gap above the room therefore halves every \({\ln 2/k \approx 12.4}\) minutes.

After 30 minutes. No logarithm is needed: \({e^{-30k} = (e^{-10k})^3 = (4/7)^3 = 64/343}\), so

\[ \begin{aligned} y(30) &= 20 + 70 \cdot \frac{64}{343} \\ &\approx 33.06 \text{ °C} . \end{aligned} \]

When it reaches 40 °C. Solve \({20 + 70\,e^{-kt} = 40}\):

\[ \begin{aligned} e^{-kt} &= \frac{20}{70} = \frac{2}{7} , \\ t &= \frac{\ln(7/2)}{k} \approx 22.4 \text{ min} . \end{aligned} \]

A graph of temperature in degrees C, from 0 to 100, against time in minutes, from 0 to 50. A dashed line at 20 is labelled room, 20 degrees C. A black curve falls from 90 towards the room line. Two blue dots mark the readings, labelled 90 degrees C at the start and 60 degrees C after 10 min. A vermillion dot on the curve at 22.4 minutes and 40 degrees has dashed guide lines to both axes and the label 40 degrees C after 22.4 min. Two grey brackets from the room line up to the curve, at 12.4 and 24.8 minutes, are labelled gap 35 and gap 17.5, and a note reads the gap above the room halves every 12.4 min.
M04.9.2-F3

The worked example. Two readings fix \(k\); the model then answers every other question. The gap above the room, 70 degrees at the start, halves every 12.4 minutes.

The figure gives a check by eye: the gap halves to 35 at 12.4 minutes (55 °C) and to 17.5 at 24.8 minutes (37.5 °C), and 40 °C falls between them, at 22.4 minutes.

Code

The formulas from scratch, with the tea:

import math

def approach(t, y0, c, k):
    """The solution of y' = -k (y - c) with y(0) = y0."""
    return c + (y0 - c) * math.exp(-k * t)

k = math.log(70 / 40) / 10                       # from 90 to 60 degrees in 10 minutes
print(f"k = {k:.4f} per minute; the gap halves every {math.log(2) / k:.1f} min")
print(f"after 30 min: {approach(30, 90, 20, k):.2f} degrees")
print(f"40 degrees after {math.log(70 / 20) / k:.1f} min")
k = 0.0560 per minute; the gap halves every 12.4 min
after 30 min: 33.06 degrees
40 degrees after 22.4 min

The gradient flow is the same formula with \({k = 15}\) and \({c = 47/30}\). A central difference checks that its slope is \({-L'(w)}\) at every time tried, and the loss falls all the way:

def flow(t, w0=0.0):
    """Gradient flow on the running loss, w' = -15 (w - 47/30), from w(0) = w0."""
    return approach(t, w0, 47 / 30, 15)

loss = lambda w: 7.5 * w**2 - 23.5 * w + 18.5
slope = lambda w: 15 * w - 23.5
h = 1e-6
for t in (0.0, 0.05, 0.1, 0.2):
    w = flow(t)
    speed = (flow(t + h) - flow(t - h)) / (2 * h)
    print(f"t = {t:4}: w = {w:.4f}, w' = {speed:8.4f}, -L'(w) = {-slope(w):8.4f}, L = {loss(w):.4f}")
t =  0.0: w = 0.0000, w' =  23.5000, -L'(w) =  23.5000, L = 18.5000
t = 0.05: w = 0.8266, w' =  11.1006, -L'(w) =  11.1006, L = 4.1991
t =  0.1: w = 1.2171, w' =   5.2436, -L'(w) =   5.2436, L = 1.0082
t =  0.2: w = 1.4887, w' =   1.1700, -L'(w) =   1.1700, L = 0.1373

Now the question of the hook. Gradient descent with ever smaller learning rates, run to the same time \({t = 0.1}\), lands ever closer to the flow:

def descend(eta, t_end, w=0.0):
    """Gradient descent on the running loss, one step per eta units of time."""
    for _ in range(round(t_end / eta)):
        w = w - eta * slope(w)
    return w

for eta in (0.01, 0.001, 0.0001):
    print(f"eta = {eta:<6} {round(0.1 / eta):5} steps: w = {descend(eta, 0.1):.5f}")
print(f"gradient flow at t = 0.1:     w = {flow(0.1):.5f}")
eta = 0.01      10 steps: w = 1.25823
eta = 0.001    100 steps: w = 1.22105
eta = 0.0001  1000 steps: w = 1.21749
gradient flow at t = 0.1:     w = 1.21710

The library version fits \(k\) to measurements. Taking logarithms of the gap turns the curve into a straight line, \({\ln(y - c) = \ln(y(0) - c) - kt}\), so numpy.polyfit with degree 1 recovers \(k\) as minus the slope:

import numpy as np

minutes = np.array([0.0, 10.0, 20.0, 30.0])
temps = 20 + 70 * (4 / 7) ** (minutes / 10)       # readings that follow the model exactly
line_slope, line_intercept = np.polyfit(minutes, np.log(temps - 20), 1)
print(f"fitted k = {-line_slope:.4f}, starting gap = {np.exp(line_intercept):.1f}")
fitted k = 0.0560, starting gap = 70.0

Explore

The module’s interactive lives in lesson M04.9.4. Choose y′ = y or Cooling, switch Euler’s steps off, and click at different heights: every solution of \({y' = y}\) is a multiple of \({e^t}\), and every cooling curve heads for 20 °C, from above or from below.

Pitfalls

Two panels of temperature against minutes from 0 to 90, each with a dashed line at 20 labelled room and an orange starting dot at 90. Left, marked with a cross and titled Heads for 0 degrees C: a vermillion curve labelled y prime equals minus k y falls through the room line and keeps falling towards 0, with the note colder than the room. Right, marked with a tick and titled Settles at the room: a black curve labelled y prime equals minus k times (y minus 20) falls and levels off along the room line, with the note levels off at 20 degrees C.
M04.9.2-F4

Wrong beside right, with the same \(k\). A rate proportional to the temperature itself cools the tea below the room; a rate proportional to the difference stops at the room.

“Cooling is y′ = −ky.”

That equation sends every temperature to 0 °C, whatever the room. Newton’s law of cooling makes the rate proportional to the difference from the room, \({y' = -k\,(y - c)}\), and its solutions level off at \(c\).

“The constant goes on the end: y = e^(kt) + C.”

In M04.6.3 a constant was added to every antiderivative, but here it multiplies. Substitute and see: \({(e^{kt} + C)' = k\,e^{kt}}\), while \({k\,(e^{kt} + C) = k\,e^{kt} + kC}\). The two agree only when \({C = 0}\). The family of solutions is \({Ce^{kt}}\), with \({C = y(0)}\).

“Gradient flow reaches the minimum.”

Not in any finite time. The distance \({\frac{47}{30}\,e^{-15t}}\) shrinks but is never zero; it falls below 1% of its starting size after \({t = \ln 100/15 \approx 0.31}\). Newton’s method, by contrast, lands on the minimum of the running loss in one step (M04.4.4). A training run stops when it is close enough, never because it has arrived.

Check yourself

Recall

1. What are all the solutions of \({y' = ky}\)?

Show answer

\({y(t) = y(0)\,e^{kt}}\), and nothing else: \({y\,e^{-kt}}\) has derivative zero, so it is constant.

2. What is the doubling time of \({y' = 0.2\,y}\)?

Show answer

\({\ln 2/0.2 \approx 3.47}\) units of time, from any starting value.

3. How do you solve \({y' = -k\,(y - c)}\)?

Show answer

Shift: \({u = y - c}\) satisfies \({u' = -ku}\), so \({u = u(0)\,e^{-kt}}\) and \({y = c + (y(0) - c)\,e^{-kt}}\).

4. Write the gradient flow of the running loss and its solution from \({w(0) = 0}\).

Show answer

\({w' = -L'(w) = -15\,(w - 47/30)}\), solved by \({w(t) = \frac{47}{30}\,(1 - e^{-15t})}\).

5. Why can the loss never rise along a gradient flow?

Show answer

By the chain rule its rate of change is \({L'(w)\,w' = -L'(w)^2}\), which is never positive.

Exercises

1. ★ A culture of bacteria doubles every 3 hours. Find \(k\), and the factor by which it grows in 10 hours.

Show solution

The doubling time is \({\ln 2/k = 3}\), so \({k = \ln 2/3 \approx 0.2310}\) per hour. In 10 hours the culture is multiplied by \({e^{10k} = 2^{10/3} \approx 10.08}\).

2. ★★ A medicine leaves the blood with a half-life of 6 hours (invented numbers). What fraction of a dose remains after 24 hours, and when does it fall to 1%?

Show solution

24 hours are four half-lives, so \({(1/2)^4 = 1/16 = 6.25\%}\) remains. The decay constant is \({k = \ln 2/6 \approx 0.1155}\) per hour, and the fraction left is \({e^{-kt}}\). It equals 0.01 when

\[ \begin{aligned} t &= \frac{\ln 100}{k} = \frac{6 \ln 100}{\ln 2} \\ &\approx 39.9 \text{ hours} . \end{aligned} \]

3. ★★★ Take any parabola \({L(w) = \frac{a}{2}(w - w^*)^2 + c}\) with \({a > 0}\). Show that gradient flow from \({w(0) = w_0}\) gives \({w(t) = w^* + (w_0 - w^*)\,e^{-at}}\), that the gap above the lowest value is \({\frac{a}{2}(w_0 - w^*)^2 e^{-2at}}\), and that \(w\) never passes \(w^*\). How long does the gap of the running loss take to halve?

Show solution

\({L'(w) = a\,(w - w^*)}\), so the flow is \({w' = -a\,(w - w^*)}\): approach to the level \(w^*\) with rate \(a\). By Step 3, \({w(t) = w^* + (w_0 - w^*)\,e^{-at}}\). Then

\[ \begin{aligned} L - c &= \tfrac{a}{2}\,(w - w^*)^2 \\ &= \tfrac{a}{2}\,(w_0 - w^*)^2\,e^{-2at} . \end{aligned} \]

Since \({e^{-at} > 0}\), the distance \({w - w^*}\) keeps the sign of \({w_0 - w^*}\) for ever: the weight never passes the minimum. For the running loss \({a = 15}\), so the gap halves when \({e^{-30t} = \tfrac12}\), after \({t = \ln 2/30 \approx 0.0231}\).

Summary card

A rate proportional to the amount gives \({y(0)\,e^{kt}}\), and nothing else.

The anchor figure again: a growing curve that doubles every 2 units of time and a decaying curve that halves every 2 units, each with green tangents whose slopes are k times the height.
M04.9.2-F1

Equal times, equal factors.

  • Growth and decay. \({y' = ky}\) has exactly the solutions \({y(0)\,e^{kt}}\).
  • Doubling time and half-life. \({\ln 2/\abs{k}}\), from any start.
  • Approach to a level. \({y' = -k\,(y - c)}\) gives \({y = c + (y(0) - c)\,e^{-kt}}\).
  • Gradient flow. \({w' = -L'(w)}\). On the running loss the distance to \({47/30}\) shrinks like \({e^{-15t}}\), and the loss never rises.

Sources and next steps

Sources. Each was opened and checked against this lesson on 12 November 2026.

  • Gilbert Strang and Edwin Herman, Calculus Volume 1 (OpenStax), Section 6.8: the growth model \({y = y_0 e^{kt}}\) and its differential equation (Equation 6.27), the doubling time and the half-life, both \({\ln 2/k}\), and Newton’s law of cooling, \({T' = -k(T - T_a)}\), solved by the change of variables \({y = T - T_a}\).
  • Gilbert Strang and Edwin Herman, Calculus Volume 2 (OpenStax), Section 4.3: Newton’s law of cooling as a differential equation (Equation 4.6), with a worked example that finds the constant from a second reading (Example 4.13).
  • Weijie Su, Stephen Boyd and Emmanuel J. Candès, A Differential Equation for Modeling Nesterov’s Accelerated Gradient Method: Theory and Insights, Journal of Machine Learning Research 17 (2016). Section 1 explains that optimisation methods are tied to differential equations by letting the step size become very small, and presents the second-order equation (3) for Nesterov’s method, which Section 2 derives.

Next. M04.9.3 Separable equations and the logistic curve solves a wider class of equations by separating the variables, and finds that the sigmoid is the solution of logistic growth.

This lesson unlocks

M04.9.3 Separable equations and the logistic curve

Your progress is saved in this browser only. No account, no tracking.
Back to top