Skip to main content

Does the sum settle? Convergence tests

Terms that shrink to 0 do not guarantee a finite sum: 1 + 1/2 + 1/3 + … grows for ever, like ln n. Compare sums with areas to settle every sum of 1/n^p, justify the ratio test by a geometric series, and see why a learning-rate schedule of 1/t works.
M04.8.2L1 Core25 minIn review

Before this lesson

M04.8.1 Sequences and the geometric series M04.7.3 Improper integrals: areas that run to infinity

After this lesson you can

  • Explain why the terms of a convergent series must tend to 0, and use that to show that a series diverges.
  • Prove that the harmonic series diverges by grouping its terms, and trap its partial sums between ln(n + 1) and 1 + ln n.
  • Decide whether a sum of 1/k^p converges by comparing it with an integral.
  • Apply the ratio test, justify it by comparison with a geometric series, and recognise when it is silent.
  • Compute partial sums with a loop and with numpy.cumsum, and explain why a running total in single precision can stop growing.

In review. The maths and code on this page are checked by automated tests on every change, but an independent reader has not signed it off yet. Spot a mistake or a confusing step? Tell us.

The question

Stochastic gradient descent is often run with a learning rate that shrinks as training goes on, such as \({\eta_t = 1/t}\) at step \(t\). The classic theory, going back to Robbins and Monro, asks two things of such a schedule: the learning rates must add up to infinity, so that the weights can travel as far as they need, and their squares must add up to a finite number, so that the noise dies away. Can numbers that shrink to 0 add up to infinity? After a billion terms, \({1 + \tfrac12 + \tfrac13 + \cdots}\) has reached only about 21.3. And why should \({1 + \tfrac14 + \tfrac19 + \cdots}\) stay finite?

From the last lesson

1. For which ratios \(r\) does the geometric series \({a + ar + ar^2 + \cdots}\) converge, and to what?

Show answer

For \({|r| < 1}\), to \({a/(1 - r)}\).

2. What does it mean for a sequence to converge to \(\ell\)?

Show answer

For every tolerance \({\varepsilon > 0}\) there is an \(N\) from which on every term lies within \(\varepsilon\) of \(\ell\).

Intuition

The terms of both sums shrink to 0. Only one of the sums settles.

Two graphs with n from 1 to 60 and a shared height from 0 to 5. Left, titled sum of 1 over k climbs for ever: thin blue bars for the terms 1 over k shrink towards 0, labelled terms 1 over k shrink to 0, while a vermillion curve of partial sums S n climbs to 4.68 at n equals 60 and is still rising. It runs inside a sky-blue band labelled trapped between ln of n plus 1 and 1 plus ln n. Right, titled sum of 1 over k squared levels off: the blue bars of 1 over k squared vanish almost at once, and the vermillion partial sums flatten just under a black dashed line labelled sum pi squared over 6, about 1.645.
M04.8.2-F1

Terms that shrink to 0 are not enough. Left: the partial sums of \({1/k}\) climb for ever, trapped in a band that grows like \({\ln n}\). Right: the partial sums of \({1/k^2}\) level off at \({\pi^2/6}\).

Here a series runs from \({k = 1}\), and its partial sum \({S_n = a_1 + \dots + a_n}\) is the total of its first \(n\) terms. Shrinking terms are necessary: if the totals settle, the terms must die out. They are not enough. What decides is how fast the terms shrink, and the sharpest way to measure that is to compare the sum with something we can already add up: an area, or a geometric series.

Build it up

Step 1. A convergent series has vanishing terms. Each term is the difference of two partial sums, \({a_n = S_n - S_{n-1}}\). If \({S_n \to S}\), then \({S_{n-1} \to S}\) as well, so \({a_n \to S - S = 0}\).

Step 2. The harmonic series diverges. The harmonic series is \({1 + \tfrac12 + \tfrac13 + \cdots}\). After the first term, group the terms into blocks of 1, 2, 4, 8, … terms. The block that ends at \({1/2^j}\) holds \({2^{j-1}}\) terms, and none is smaller than \({1/2^j}\), so the block adds up to at least \({2^{j-1}/2^j = \tfrac12}\).

Bars of height 1 over k for k from 1 to 16, in blue. A label above the first bar reads first term 1. The rest are grouped by alternating yellow shading into blocks: k equals 2; k equals 3 and 4; k equals 5 to 8; and k equals 9 to 16. Dashed black lines across each of the last three blocks mark the height of its smallest term, 1 over 4, 1 over 8 and 1 over 16. Above the blocks: one half; at least 2 times one quarter, equals one half; at least 4 times one eighth, equals one half; at least 8 times one sixteenth, equals one half.
M04.8.2-F2

Each block is at least as big as its smallest term times the number of terms, and that product is always \({\tfrac12}\).

So the first \({2^j}\) terms add up to at least \({1 + j/2}\): past 3 after 16 terms, past 6 after 1024. The partial sums pass every bound, and a convergent sequence is bounded (M04.8.1), so the harmonic series diverges.

Step 3. How fast it grows. Stand a bar of height \({1/k}\) on each stretch \({[k, k + 1]}\). The curve \({1/x}\) falls, so each bar reaches above it, and the bars together cover the area under \({1/x}\) from 1 to \({n + 1}\), which is \({\ln(n + 1)}\) (M04.6.3). Moving every bar after the first one step left, onto \({[k - 1, k]}\), tucks it under the curve, inside the area from 1 to \(n\), which is \({\ln n}\). Together:

\[ \ln(n + 1) \le S_n \le 1 + \ln n . \]

The partial sums track \({\ln n}\). A billion terms give about 21.3; reaching 100 takes about \({1.5 \times 10^{43}}\) terms.

Step 4. Faster-shrinking terms. For \({1/k^2}\), the bars moved left sit under the curve \({1/x^2}\), whose area from 1 to \(n\) is \({1 - 1/n}\). So \({S_n \le 1 + (1 - 1/n) < 2}\) for every \(n\). The partial sums only rise, since every term is positive, and they never pass 2, so they settle.

The formal statement

Two graphs for x from 1 to 7. Left, titled bars above 1 over x: the blue curve 1 over x with the area under it shaded blue, and vermillion bars of heights 1, one half and so on to one sixth standing on the stretches from 1 to 2, 2 to 3 and so on, each reaching above the curve. Labels: bars: 1 plus one half plus and so on plus one sixth; area under 1 over x: ln 7. Right, titled bars below 1 over x squared: the blue curve 1 over x squared with the area under it shaded, and vermillion bars of heights one quarter, one ninth and so on to one forty-ninth on the stretches from 1 to 2 up to 6 to 7, each tucked under the curve. Labels: bars: one quarter plus one ninth plus and so on plus one forty-ninth; area under 1 over x squared: 1 minus one seventh.
M04.8.2-F3

A sum compared with an area. The bars of the harmonic series stand above \({1/x}\), whose area grows without bound; the bars of \({\sum 1/k^2}\) after the first sit below \({1/x^2}\), whose area stays below 1.

NoteResults: four tests
  • Terms must vanish. If \({\sum a_k}\) converges, then \({a_k \to 0}\). So a series whose terms do not tend to 0 diverges.
  • Comparison. If \({0 \le a_k \le b_k}\) for every \(k\) and \({\sum b_k}\) converges, then \({\sum a_k}\) converges.
  • Integral comparison. If \(f\) is positive and falling on \({[1, \infty)}\), with \({a_k = f(k)}\), then \({\sum a_k}\) converges exactly when \({\int_1^\infty f(x)\,dx}\) does. So \({\sum 1/k^p}\) converges exactly when \({p > 1}\).
  • Ratio test. If every \({a_k \neq 0}\) and \({|a_{k+1}/a_k| \to \rho}\), the series converges when \({\rho < 1}\) and diverges when \({\rho > 1}\). When \({\rho = 1}\) the test is silent.

The first result is Step 1. For comparison, the partial sums of \({\sum a_k}\) rise, because no term is negative, and each stays below the sum of \({\sum b_k}\). A rising sequence that never passes a ceiling settles on a limit: the completeness of the real numbers, met in M04.1.3.

For integral comparison, the bars of Step 3 work for any positive falling \(f\):

\[ \begin{aligned} \int_1^{n+1} f(x)\,dx &\le S_n , \\ S_n &\le a_1 + \int_1^{n} f(x)\,dx . \end{aligned} \]

If the integral to infinity converges, the second line gives the partial sums a ceiling. If it diverges, the first line pushes them past every bound. For \({f(x) = 1/x^p}\) with \({p > 0}\), M04.7.3 showed that \({\int_1^\infty x^{-p}\,dx}\) converges exactly when \({p > 1}\). For \({p \le 0}\) the terms \({1/k^p}\) are at least 1 and do not vanish. So the schedule \({\eta_t = 1/t}\) of the opening question meets both demands: \({\sum 1/t}\) diverges, while \({\sum 1/t^2}\) converges.

The ratio test, by comparison with a geometric series. Suppose \({\rho < 1}\) and pick a number \(r\) between them, such as \({r = (\rho + 1)/2}\). From some \(K\) on, the ratios are within \({r - \rho}\) of \(\rho\), so \({|a_{k+1}| \le r\,|a_k|}\). Applying this \(j\) times gives

\[ |a_{K+j}| \le |a_K|\, r^j \quad (j = 0, 1, 2, \dots). \]

The right-hand sides form a geometric series with ratio \({r < 1}\), so by comparison the sizes \({|a_K| + |a_{K+1}| + \cdots}\) add up to at most \({|a_K|/(1 - r)}\); the terms before \(K\) only add a fixed amount. A series whose sizes add up converges itself (proved below). If \({\rho > 1}\), the sizes grow from some \(K\) on, so the terms do not vanish. If \({\rho = 1}\), anything can happen: the ratios \({k/(k+1)}\) of \({\sum 1/k}\) and \({k^2/(k+1)^2}\) of \({\sum 1/k^2}\) both tend to 1, and one series diverges while the other converges.

The sum of \({\sum 1/k^2}\) is \({\pi^2/6 \approx 1.6449}\), a result whose proof needs more than this course. The partial sums \({S_n}\) of the harmonic series minus \({\ln n}\) settle on \({0.5772\ldots}\), called Euler’s constant.

Go deeper: why sizes that add up are enough

Suppose \({\sum |a_k|}\) converges, with terms of either sign. Split each term into a positive part \({p_k = (|a_k| + a_k)/2}\) and a negative part \({q_k = (|a_k| - a_k)/2}\). Both lie between 0 and \({|a_k|}\), so \({\sum p_k}\) and \({\sum q_k}\) converge by comparison. Since \({a_k = p_k - q_k}\), the partial sums of \({\sum a_k}\) are differences of two convergent sequences, so they converge by the limit laws.

Worked example

Does \({\sum_{k=1}^\infty k/2^k = \tfrac12 + \tfrac24 + \tfrac38 + \cdots}\) converge, and how big can its sum be?

The ratio. Divide one term by the one before:

\[ \begin{aligned} \frac{a_{k+1}}{a_k} &= \frac{k + 1}{2^{k+1}} \cdot \frac{2^k}{k} \\ &= \frac{k + 1}{2k} = \frac{1}{2} + \frac{1}{2k} . \end{aligned} \]

It tends to \({\rho = \tfrac12 < 1}\), so the series converges.

A ceiling. From \({k = 2}\) on, \({1/(2k) \le \tfrac14}\), so every ratio is at most \({\tfrac34}\). Each term is then at most \({\tfrac34}\) of the one before, and with \({a_2 = \tfrac12}\):

\[ a_k \le \tfrac12 \left(\tfrac34\right)^{k-2} \quad (k \ge 2). \]

The geometric lid adds up to \({\tfrac12 / (1 - \tfrac34) = 2}\), and the first term adds \({\tfrac12}\), so the sum is at most 2.5.

Two graphs. Left, titled terms under a geometric lid: blue bars of k over 2 to the k for k from 1 to 12, of heights 0.5, 0.5, 0.375, 0.25 and shrinking, labelled terms k over 2 to the k. Vermillion dots joined by a line start on top of the second bar and stay above every later bar, labelled lid one half times three quarters to the k minus 2. Right, titled partial sums settle: vermillion dots of the partial sums climb from S 1 equals 0.5 through 1, 1.375 and 1.625 and flatten under a black dashed line labelled sum 2, which lies below a dotted vermillion line labelled ceiling from the lid: 2.5.
M04.8.2-F4

The worked example as a picture. From \({k = 2}\) on, every term sits under the geometric lid, so the sum is at most 2.5; the partial sums in fact close in on 2.

The partial sums are 0.5, 1, 1.375, 1.625, 1.78125, 1.875, …, and they close in on 2 (exercise 3 proves it). The ceiling 2.5 was not the sum, but it was enough to prove that a sum exists.

Code

From scratch, a running total, checked against \({\ln n}\):

import math

def partial_sums(term, n):
    """S_1, ..., S_n of the series term(1) + term(2) + ..., by a running total."""
    sums, total = [], 0.0
    for k in range(1, n + 1):
        total += term(k)
        sums.append(total)
    return sums

H = partial_sums(lambda k: 1 / k, 10_000)
for n in (10, 100, 1000, 10_000):
    print(f"S_{n} = {H[n - 1]:.4f}   ln n = {math.log(n):.4f}   gap = {H[n - 1] - math.log(n):.4f}")
S_10 = 2.9290   ln n = 2.3026   gap = 0.6264
S_100 = 5.1874   ln n = 4.6052   gap = 0.5822
S_1000 = 7.4855   ln n = 6.9078   gap = 0.5777
S_10000 = 9.7876   ln n = 9.2103   gap = 0.5773

With NumPy, numpy.cumsum gives a million partial sums at once:

import numpy as np

k = np.arange(1, 1_000_001)
H = np.cumsum(1 / k)                              # partial sums of 1/k
S2 = np.cumsum(1 / k**2)                          # partial sums of 1/k^2
print("first n with S_n > 10:", int(np.argmax(H > 10)) + 1)
print("sum of 1/k^2 after 10, 1000, 10^6 terms:", S2[[9, 999, 999_999]].round(6),
      " pi^2/6 =", round(math.pi**2 / 6, 6))

terms = np.arange(1, 41) / 2.0 ** np.arange(1, 41)
print("ratios of k/2^k:", (terms[1:6] / terms[:5]).round(4))
print("partial sums of k/2^k:", np.cumsum(terms)[[0, 1, 2, 3, 9, 39]].round(5))
first n with S_n > 10: 12367
sum of 1/k^2 after 10, 1000, 10^6 terms: [1.549768 1.643935 1.644933]  pi^2/6 = 1.644934
ratios of k/2^k: [1.     0.75   0.6667 0.625  0.6   ]
partial sums of k/2^k: [0.5     1.      1.375   1.625   1.98828 2.     ]

The same running total in single precision, the format most neural networks train in, against numpy.sum, which adds in pairs:

k32 = np.arange(1, 3_000_001, dtype=np.float32)
running = np.cumsum(1 / k32, dtype=np.float32)    # one term at a time, rounded to float32
print("float32 running total after 2.2 and 3 million terms:", running[2_199_999], running[-1])
print("float32 numpy.sum of the same terms:", np.sum(1 / k32, dtype=np.float32))
print("true value of S_3000000:", round(float(np.sum(1 / np.arange(1, 3_000_001))), 4))
float32 running total after 2.2 and 3 million terms: 15.403683 15.403683
float32 numpy.sum of the same terms: 15.491338
true value of S_3000000: 15.4913

The running total stops changing after about 2.1 million terms, near 15.40: there float32 numbers lie about \({10^{-6}}\) apart, and a term less than half that gap rounds away. Adding in pairs keeps the totals small for longer and gets 15.4913. Neither result says anything about the true series.

Explore

This module’s interactive, the Taylor builder, lives in the next lesson. Its partial sums are polynomials, and the ratio test of this lesson predicts where they settle.

Pitfalls

Two graphs of the ratio a k plus 1 over a k against k from 1 to 20, each with a black dashed line at 1. Left, marked with a cross and titled ratios creep up to 1: blue dots for the harmonic series climb from 0.5 towards 1, labelled sum of 1 over k: ratio k over k plus 1 tends to 1, with the note every ratio is below 1, yet the sum diverges. Right, marked with a tick and titled ratios stay below three quarters: blue dots for k over 2 to the k start at 1, drop to 0.75 at k equals 2 and fall towards one half, under an orange dashed line labelled three quarters, from k equals 2 on; notes say ratio k plus 1 over 2 k tends to one half, and the sum converges.
M04.8.2-F5

Wrong beside right. Every ratio of the harmonic series is below 1, but the ratios creep up to 1, and the series diverges. The ratios of \({\sum k/2^k}\) stay below \({\tfrac34}\), and it converges.

“The terms shrink to 0, so the series converges.”

Vanishing terms are needed, but they are not enough. The terms \({1/k}\) vanish and the harmonic series still diverges, because they vanish too slowly.

“Every ratio is below 1, so the ratio test says it converges.”

The test needs the limit \(\rho\) of the ratios to be below 1, so that the ratios stay below some fixed number \({r < 1}\). The ratios \({k/(k+1)}\) of the harmonic series are all below 1 but tend to 1.

“The ratios of \({\sum 1/k^2}\) tend to 1, so it diverges.”

At \({\rho = 1}\) the ratio test says nothing at all. Integral comparison settles this series: it converges, to \({\pi^2/6}\).

“The running total has stopped growing, so the sum is finite.”

In float32 the running total of \({1/k}\) stops near 15.40, though the true partial sums grow for ever. A computer can only show what happens to finitely many terms in finite precision. Convergence is a question about all of them, settled by a proof.

Check yourself

Recall

1. If \({\sum a_k}\) converges, what must the terms do, and why?

Show answer

Tend to 0, because \({a_n = S_n - S_{n-1}}\) and both partial sums tend to the same limit.

2. Why does the harmonic series diverge?

Show answer

Grouped into blocks of 1, 2, 4, 8, … terms, every block adds up to at least \({\tfrac12}\), so the first \({2^j}\) terms add up to at least \({1 + j/2}\), which passes every bound.

3. Between which two numbers does the partial sum \({1 + \tfrac12 + \cdots + \tfrac1n}\) lie?

Show answer

Between \({\ln(n + 1)}\) and \({1 + \ln n}\), from comparing the bars with the area under \({1/x}\).

4. For which \(p\) does \({\sum 1/k^p}\) converge?

Show answer

Exactly for \({p > 1}\).

5. What does the ratio test conclude when \({\rho < 1}\), \({\rho > 1}\) and \({\rho = 1}\)?

Show answer

The series converges, it diverges, and nothing at all.

Exercises

1. ★ Converge or diverge? (a) \({\sum k/(k + 1)}\) (b) \({\sum 1/k^{1.5}}\) (c) \({\sum 1/\sqrt{k}}\)

Show solution

(a) The terms tend to 1, not 0, so it diverges.

(b) \({p = 1.5 > 1}\), so it converges.

(c) \({1/\sqrt{k} = 1/k^{0.5}}\) and \({p = 0.5 \le 1}\), so it diverges.

2. ★★ Use the ratio test, and when it is silent, a comparison. (a) \({\sum_{k \ge 0} 3^k/k!}\) (b) \({\sum_{k \ge 1} 2^k/k^2}\) (c) \({\sum_{k \ge 1} 1/(k^2 + 1)}\)

Show solution

(a) The ratio is \({\frac{3^{k+1}}{(k+1)!} \cdot \frac{k!}{3^k} = \frac{3}{k + 1} \to 0}\), so it converges. (Its sum is \({e^3 \approx 20.09}\), as M04.8.4 shows.)

(b) The ratio is \({2k^2/(k + 1)^2 \to 2 > 1}\), so it diverges.

(c) The ratio tends to 1, so the test is silent. But \({0 < 1/(k^2 + 1) \le 1/k^2}\), and \({\sum 1/k^2}\) converges, so this series converges by comparison.

3. ★★★ Let \({S_n = \sum_{k=1}^n k/2^k}\). Show that \({S_n - \tfrac12 S_n = 1 - 1/2^n - n/2^{n+1}}\), and deduce that the sum is 2.

Show solution

Halving each term and renaming \({k + 1}\) as \(k\),

\[ \begin{aligned} \tfrac12 S_n &= \sum_{k=1}^{n} \frac{k}{2^{k+1}} \\ &= \sum_{k=2}^{n+1} \frac{k - 1}{2^k} . \end{aligned} \]

Subtract term by term. The \({k = 1}\) term of \(S_n\), \({\tfrac12}\), has no partner; for \({2 \le k \le n}\) the difference is \({(k - (k - 1))/2^k = 1/2^k}\); and the last term of \({\tfrac12 S_n}\), \({n/2^{n+1}}\), is subtracted. So

\[ \begin{aligned} S_n - \tfrac12 S_n &= \sum_{k=1}^{n} \frac{1}{2^k} - \frac{n}{2^{n+1}} \\ &= 1 - \frac{1}{2^n} - \frac{n}{2^{n+1}} , \end{aligned} \]

using the finite geometric sum. Doubling, \({S_n = 2 - 2/2^n - n/2^n}\). As \({n \to \infty}\), \({1/2^n \to 0}\), and \({n/2^n \to 0}\) because it is a term of the convergent series itself. So \({S_n \to 2}\).

Summary card

Shrinking terms are necessary, not sufficient: compare the sum with an area or a geometric series.

The anchor figure again: the partial sums of 1 over k climb for ever inside a band that grows like ln n, while those of 1 over k squared level off at pi squared over 6.
M04.8.2-F1

One sum climbs, one settles.

  • Vanishing terms. If \({\sum a_k}\) converges, then \({a_k \to 0}\); the reverse fails.
  • Harmonic series. Its partial sums lie between \({\ln(n + 1)}\) and \({1 + \ln n}\): it diverges.
  • p-series. \({\sum 1/k^p}\) converges exactly when \({p > 1}\).
  • Ratio test. \({|a_{k+1}/a_k| \to \rho}\): converges if \({\rho < 1}\), diverges if \({\rho > 1}\), silent if \({\rho = 1}\).

Sources and next steps

Sources. Each was opened and checked against this lesson on 12 November 2026.

  • Gilbert Strang and Edwin Herman, Calculus Volume 2 (OpenStax). Section 5.2 groups the harmonic series (5.5) to show that its partial sums satisfy \({S_{2^j} > 1 + j/2}\). Section 5.3 gives the divergence test (Theorem 5.8), the integral test (Theorem 5.9) with the bars above \({1/x}\) of Figure 5.12, and the p-series (5.9). Section 5.6 gives the ratio test (Theorem 5.16) and shows with p-series that it is inconclusive when \({\rho = 1}\).
  • Léon Bottou, Frank E. Curtis and Jorge Nocedal, Optimization Methods for Large-Scale Machine Learning, SIAM Review 60 (2018). Section 4.2 states the step-size requirement of Robbins and Monro, a sum of step sizes that diverges and a sum of their squares that converges (4.19), and Theorem 4.7 uses step sizes \({\beta/(\gamma + k)}\).
  • NIST, Digital Library of Mathematical Functions: equation 25.6.1 gives \({\zeta(2) = \pi^2/6}\), where \({\zeta(s) = \sum_{n \ge 1} n^{-s}}\) by equation 25.2.1; equation 5.2.3 defines Euler’s constant as the limit of \({1 + \tfrac12 + \cdots + \tfrac1n - \ln n}\), which is \({0.57721\ldots}\).
  • NumPy developers, numpy.cumsum (NumPy 2.5 reference), whose notes warn that its last value may differ from numpy.sum, which may add in pairs to reduce rounding error.

Next. M04.8.3 Taylor polynomials builds the polynomials whose partial sums stand in for \({e^x}\), \({\sin x}\) and the loss itself.

This lesson unlocks

M04.8.3 Taylor polynomials

Your progress is saved in this browser only. No account, no tracking.
Back to top