\( \newcommand{\vect}[1]{\boldsymbol{\mathbf{#1}}} \newcommand{\mat}[1]{\boldsymbol{\mathbf{#1}}} \newcommand{\R}{\mathbb{R}} \newcommand{\E}{\mathbb{E}} \newcommand{\Prob}{\mathbb{P}} \newcommand{\T}{^{\top}} \newcommand{\norm}[1]{\lVert #1 \rVert} \newcommand{\abs}[1]{\lvert #1 \rvert} \newcommand{\ihat}{\mkern1mu\hat{\boldsymbol{\imath}}\mkern2mu} \newcommand{\jhat}{\mkern1mu\hat{\boldsymbol{\jmath}}\mkern2mu} \newcommand{\pd}[2]{\frac{\partial #1}{\partial #2}} \newcommand{\argmax}{\operatorname*{arg\,max}} \newcommand{\argmin}{\operatorname*{arg\,min}} \newcommand{\cdata}[1]{\class{c-data}{#1}} \newcommand{\cmodel}[1]{\class{c-model}{#1}} \newcommand{\ctruth}[1]{\class{c-truth}{#1}} \newcommand{\cgrad}[1]{\class{c-grad}{#1}} \newcommand{\closs}[1]{\class{c-loss}{#1}} \newcommand{\cparam}[1]{\class{c-param}{#1}} \newcommand{\csecond}[1]{\class{c-second}{#1}} \renewcommand{\,}{\mkern3mu} \)

Alignment: labels are matched, not positions

When two pandas objects meet in arithmetic, their labels are matched first. Build alignment by hand over the union of labels, see where NaN comes from, centre a table by columns and by rows with sub(axis=0), and use fill_value and reindex.

P02.4.4L1 Core25 min

In review (what this means)

The maths and code on this page are checked by automated tests on every change, but an independent reader has not signed it off yet. Spot a mistake or a confusing step? Tell us.

Before this lesson: P02.4.3 Selecting rows and columns: loc and iloc P02.2 Broadcasting and vectorization

What you will be able to do

• Predict the result of adding two Series whose labels differ, including its labels, its order and where NaN appears.
• Build label alignment by hand as a dictionary look-up over the union of the labels.
• Subtract a Series from a DataFrame along the columns, and along the rows with sub(…, axis=0).
• Supply a stand-in for a missing partner with fill_value, and conform a Series to a list of labels with reindex.

The question

Two loggers send their noon readings. The coast file lists Monday, Tuesday and Wednesday; the hill file lists Tuesday, Monday and Thursday. Add the two columns as NumPy arrays and Monday’s coast reading is added to Tuesday’s hill reading, with no error and no warning. What does pandas do instead, and what should a sum be for a day that only one file has?

From the last lesson

1. How many rows does wide.loc['tue':'wed'] have, and why?

Show answer

Two. loc finds the positions of 'tue' and 'wed', 1 and 2, and a label slice includes its end.

2. What is wrong with wide[mask]['hill'] = 0, and what is the fix?

Show answer

It writes into a temporary object that behaves as a copy, so wide is unchanged and pandas warns with ChainedAssignmentError. Write wide.loc[mask, 'hill'] = 0.

Intuition

Before pandas adds two Series, it lines up their labels. A value is added to the value with the same label, wherever each sits. This is alignment, and it has no counterpart in NumPy.

P02.4.4-F1

a + b matches the labels. Monday meets Monday and Tuesday meets Tuesday; Wednesday and Thursday have no partner and give NaN. The result has every label of either Series.

Three things in the picture are the whole idea:

• Matching. mon is added to mon and tue to tue, even though mon sits at position 0 in a and position 1 in b.
• The union. The result has a row for every label that appears in either Series: four rows from two Series of three.
• NaN for no partner. wed is only in a and thu only in b. A sum needs two numbers, so their result is NaN, “not a number”, pandas’ mark for a missing value.

Build it up

Step 1. What NumPy does. Arrays have no labels, so NumPy pairs position 0 with position 0.

P02.4.4-F2

The same two columns, added by position (left) and matched by label (right). Only the right-hand sums add a day’s coast reading to the same day’s hill reading.

import numpy as np
import pandas as pd

a = pd.Series([21, 27, 27], index=['mon', 'tue', 'wed'])     # coast
b = pd.Series([23, 19, 20], index=['tue', 'mon', 'thu'])     # hill
print(a.to_numpy() + b.to_numpy())

[44 46 47]

The numbers 44, 46 and 47 are each a coast reading plus a hill reading from another day. Nothing fails, which is what makes the mistake dangerous.

Step 2. What pandas does. The same + on the Series matches labels:

print(a + b)

mon    40.0
thu     NaN
tue    50.0
wed     NaN
dtype: float64

Monday is \({21 + 19 = 40}\) and Tuesday \({27 + 23 = 50}\). Two details of the print-out need a word. The values are written 40.0 and 50.0, because NaN exists only among floats (P02.1.4), so a result that holds one NaN becomes float64 throughout. And the labels are in the order mon, thu, tue, wed: when the two indexes are not identical, pandas sorts the union, and for text that means alphabetical order, not calendar order.

Step 3. A stand-in for the missing side. The method form of + is a.add(b), and it takes a fill_value that is used when exactly one side is missing:

print(a.add(b, fill_value=0))

mon    40.0
thu    20.0
tue    50.0
wed    27.0
dtype: float64

Thursday becomes \({0 + 20 = 20}\) and Wednesday \({27 + 0 = 27}\). Whether that is sensible depends on what the numbers mean. For counts, a missing count often really is 0. For temperatures it is nonsense: Thursday’s coast reading was not 0 degrees. Only use fill_value when the stand-in is true. sub, mul and div take fill_value in the same way.

Step 4. Choosing the labels. reindex conforms a Series to a list of labels you give: it keeps the values whose labels are in the list, in the list’s order, and puts NaN where a label has no value. It is how you get calendar order back, or check which days are missing:

days = ['mon', 'tue', 'wed', 'thu']
print(a.reindex(days))

mon    21.0
tue    27.0
wed    27.0
thu     NaN
dtype: float64

The formal statement

NoteAlignment

For Series a and b, the result of a + b has one row for each label in the union of their indexes. Its value at label \(\ell\) is \({a[\ell] + b[\ell]}\) if both have the label, and NaN otherwise. If the two indexes are identical, the result keeps their order; if not, the union is sorted (when the labels can be compared, as text with text or numbers with numbers). The same holds for -, * and /.

A DataFrame and a Series meet the same way, with one choice to make: which edge of the table do the Series’ labels belong to? By default pandas matches the Series index against the column labels, then repeats the Series down every row, as broadcasting stretched a row in P02.2.3.

P02.4.4-F3

wide - wide.mean(). The station means are labelled by station, so they line up with the columns and are repeated down the rows. Each reading becomes its distance from its station’s mean.

So wide - wide.mean() works: wide.mean() is indexed by coast, hill, valley, which are the column labels. The day means, wide.mean(axis=1), are indexed by days, which are not column labels, so they need the other edge. The methods sub, add, mul and div take axis, and axis=0 matches the Series against the index instead. Then each row has its own day’s mean taken away: for Monday, \({21 - 22}\), \({19 - 22}\) and \({26 - 22}\), which are \({-1}\), \({-3}\) and 4.

Worked example

Add a and b by hand, label by label, with and without fill_value=0.

P02.4.4-F4

Alignment as a loop over the sorted union of the labels. A dictionary look-up gives each side’s value or nothing.

The labels of a are mon, tue, wed and those of b are tue, mon, thu. The two indexes differ, so the result’s labels are their union, sorted: mon, thu, tue, wed.

For each label, look it up on both sides:

• mon: 21 in a, 19 in b, so the sum is 40.
• thu: nothing in a, 20 in b, so the sum is NaN; with fill_value=0 it is \({0 + 20 = 20}\).
• tue: 27 in a, 23 in b, so the sum is 50.
• wed: 27 in a, nothing in b, so the sum is NaN; with fill_value=0 it is \({27 + 0 = 27}\).

There are 4 labels in the union, 2 in both Series (so 2 sums), and 2 in exactly one (so 2 NaN).

Code

From scratch first. Each Series becomes a dictionary from label to value (to_dict), and the sum is a loop over the union of the keys.

def align_add(x, y, fill_value=None):
    """x + y for two dicts from label to value, the way pandas aligns two Series."""
    if list(x) == list(y):
        labels = list(x)                      # identical indexes keep their order
    else:
        labels = sorted(set(x) | set(y))      # otherwise: the sorted union
    out = {}
    for label in labels:
        left, right = x.get(label), y.get(label)
        if left is None and right is not None and fill_value is not None:
            left = fill_value
        if right is None and left is not None and fill_value is not None:
            right = fill_value
        if left is None or right is None:
            out[label] = float('nan')
        else:
            out[label] = float(left + right)
    return out

print(align_add(a.to_dict(), b.to_dict()))
print(align_add(a.to_dict(), b.to_dict(), fill_value=0))

{'mon': 40.0, 'thu': nan, 'tue': 50.0, 'wed': nan}
{'mon': 40.0, 'thu': 20.0, 'tue': 50.0, 'wed': 27.0}

The hand-made version gives the same labels, the same order and the same values as pandas, NaN included. Now tables. Rebuild wide, centre each station, then each day, and look at the mistake from the pitfalls below.

temps = np.array([[21, 19, 26], [27, 23, 28], [27, 18, 30], [29, 20, 32]])
wide = pd.DataFrame(temps, index=days, columns=['coast', 'hill', 'valley'])

by_station = wide - wide.mean()
day_means = wide.mean(axis=1)
by_day = wide.sub(day_means, axis=0)
wrong = wide - day_means

print(by_station)
print(by_day)
print(wrong.shape, list(wrong.columns), int(wrong.isna().sum().sum()))

     coast  hill  valley
mon   -5.0  -1.0    -3.0
tue    1.0   3.0    -1.0
wed    1.0  -2.0     1.0
thu    3.0   0.0     3.0
     coast  hill  valley
mon   -1.0  -3.0     4.0
tue    1.0  -3.0     2.0
wed    2.0  -7.0     5.0
thu    2.0  -7.0     5.0
(4, 7) ['coast', 'hill', 'mon', 'thu', 'tue', 'valley', 'wed'] 28

Every column of by_station and every row of by_day now has mean 0: the centring of feature scaling (ML01.3.3), done with labels so that each number meets the right mean. The last line is the trap: wide - day_means has 7 columns and all 28 entries are NaN.

Explore

P02.4-X1

Label aligner

Take labels out of either Series or put them back, and turn a Series’ rows round. Watch the top sum pair whatever happens to share a position, and the bottom sum follow the labels. Can you make the two sums agree? Then switch on fill_value=0 and see which NaN it replaces.

Pitfalls

P02.4.4-F5

Wrong beside right. Subtracting the day means with - matches the day labels against the column labels, finds nothing in common, and returns a 4 by 7 table of NaN. sub(..., axis=0) matches them against the index.

“wide - wide.mean(axis=1) subtracts each day’s mean from its row.”

A DataFrame minus a Series matches the Series’ labels against the columns. The day labels mon, tue, wed, thu are not columns, so the result’s columns are the union of both sets of labels, seven in all, and nothing is ever matched: every entry is NaN. NumPy would have refused, with shapes (4, 3) and (4,) (P02.2.4); pandas quietly returns a useless table. Write wide.sub(wide.mean(axis=1), axis=0).

“NaN in my result means a reading was missing.”

Not always. Here every reading was present in its own file; the NaN came from labels that had no partner. Before filling or dropping NaN (P02.5.2), check whether the labels were meant to match: 'mon' and 'Mon', or 'mon' and 'mon ' with a stray space, are different labels (P02.7.3).

“When I want position-by-position, I should drop the labels and hope.”

If you really mean “row 0 with row 0”, say so: a.to_numpy() + b.to_numpy(). Doing it by accident is the expensive version. A model trained on a feature table whose target column was sorted differently learns from every example paired with another example’s answer, and nothing crashes. Alignment is pandas’ protection against exactly that.

Check yourself

Recall

1. What are the labels of a + b for a labelled mon, tue, wed and b labelled tue, mon, thu, and in what order?

Show answer

The union, sorted because the indexes differ: mon, thu, tue, wed.

2. Why are the sums printed as 40.0 and 50.0 rather than 40 and 50?

Show answer

The result holds NaN, which is a float, so the whole Series becomes float64.

3. What does a.add(b, fill_value=0) give for a label that only b has?

Show answer

b’s value plus 0: for thu, 20.0. A label missing from both sides would still give NaN.

4. In wide - wide.mean(), which labels are matched with which?

Show answer

The index of the Series, the station names, is matched against the column labels of wide; the Series is then repeated down every row.

5. How do you subtract each day’s mean from its own row?

Show answer

wide.sub(wide.mean(axis=1), axis=0): axis=0 matches the Series against the index.

Exercises

1. ★ Let s = pd.Series([1, 2, 3], index=['x', 'y', 'z']) and t = pd.Series([10, 20], index=['z', 'x']). Give s + t (labels, order and values), then s.add(t, fill_value=0) and s.to_numpy()[:2] + t.to_numpy().

Show solution

The indexes differ, so the labels are the sorted union x, y, z. x: \({1 + 20 = 21}\); y has no partner, NaN; z: \({3 + 10 = 13}\). So s + t is x 21.0, y NaN, z 13.0. With fill_value=0, y becomes \({2 + 0 = 2}\): x 21.0, y 2.0, z 13.0. By position, \({[1, 2]} + {[10, 20]} = {[11, 22]}\), which pairs x with z and y with x.

2. ★★ Express every station relative to the coast on the same day, so that the coast column becomes all zeros. Write the pandas line and give the result for Wednesday and Thursday.

Show solution

The coast column is a Series indexed by days, so it must be matched against the index: wide.sub(wide['coast'], axis=0). Wednesday: \({27 - 27 = 0}\), \({18 - 27 = -9}\), \({30 - 27 = 3}\). Thursday: \({29 - 29 = 0}\), \({20 - 29 = -9}\), \({32 - 29 = 3}\). Without axis=0, wide - wide['coast'] would match the days against the columns and give a 4 by 7 table of NaN.

3. ★★★ Show that for two Series with no repeated labels, the number of NaN in a + b is the number of labels that are in exactly one of the two indexes, and that the length of a + b is the number in the first plus the number in the second minus the number in both. Check both counts on the a and b of this lesson.

Show solution

The result has one row per label in the union \({A \cup B}\) of the two label sets. A row is a number exactly when its label is in both, that is in \({A \cap B}\), and NaN otherwise, that is when the label is in \({A \cup B}\) but not in \({A \cap B}\): in exactly one of the two. Counting the union, each label of \(A\) and each label of \(B\) is counted once, and those in both are counted twice, so

\[ \abs{A \cup B} = \abs{A} + \abs{B} - \abs{A \cap B}. \]

Here \({A = \{\text{mon}, \text{tue}, \text{wed}\}}\) and \({B = \{\text{tue}, \text{mon}, \text{thu}\}}\), so \({\abs{A \cap B} = 2}\) and \({\abs{A \cup B}} = {3 + 3 - 2} = 4\), the length of a + b. The labels in exactly one are wed and thu: 2 NaN, as printed.

Summary card

pandas matches labels before it does arithmetic.

P02.4.4-F1

Matched by label; NaN where there is no partner.

• a + b has the union of the labels: sorted if the indexes differ, NaN where a label has no partner.
• By hand: a loop over the union with a dictionary look-up on each side.
• a.add(b, fill_value=0) stands in for one missing side; reindex(labels) sets the labels and their order.
• df - series matches the Series to the columns; df.sub(series, axis=0) matches it to the index.

Sources and next steps

Sources. Each was opened and checked against this lesson on 15 November 2026.

• pandas development team, Intro to data structures, pandas User Guide, version 3.1.1. Operations between Series align the data on label; the result of an operation between unaligned Series has the union of the indexes, and a label found in only one is marked missing, NaN. Between a DataFrame and a Series the default is to align the Series index on the DataFrame columns, broadcasting row-wise.
• pandas development team, Essential basic functionality, pandas User Guide, version 3.1.1. Flexible binary operations: add, sub, mul, div match on the index or the columns through the axis keyword, as in df.sub(column, axis=0). Missing data and operations with fill values: fill_value substitutes when at most one of the two values is missing. Reindexing inserts NA markers for labels where no data existed.
• pandas development team, pandas.Index.union, API reference, version 3.1.1: with the default sort=None the union is sorted, except when the two indexes are equal and in two edge cases that the page lists (an empty index, labels that cannot be compared).
• Wes McKinney, Python for Data Analysis, chapter 5, 3rd edition, open access, section 5.2: arithmetic and data alignment, arithmetic methods with fill values, and operations between DataFrame and Series.

Next. Module P02.5, Cleaning, reshaping and joining, reads real files with missing values and matches whole tables on a key, the same idea as alignment applied to rows. Module P02.6 groups rows by a label and summarises each group.

This lesson unlocks

P02.5 Cleaning, reshaping and joining P02.6 Group-by and aggregation

Your progress is saved in this browser only. No account, no tracking.