\( \newcommand{\vect}[1]{\mathbf{#1}} \newcommand{\mat}[1]{\mathbf{#1}} \newcommand{\R}{\mathbb{R}} \newcommand{\E}{\mathbb{E}} \newcommand{\Prob}{\mathbb{P}} \newcommand{\T}{^{\top}} \newcommand{\norm}[1]{\lVert #1 \rVert} \newcommand{\abs}[1]{\lvert #1 \rvert} \newcommand{\ihat}{\mkern1mu\hat{\boldsymbol{\imath}}\mkern2mu} \newcommand{\jhat}{\mkern1mu\hat{\boldsymbol{\jmath}}\mkern2mu} \newcommand{\pd}[2]{\frac{\partial #1}{\partial #2}} \newcommand{\argmax}{\operatorname*{arg\,max}} \newcommand{\argmin}{\operatorname*{arg\,min}} \newcommand{\cdata}[1]{\class{c-data}{#1}} \newcommand{\cmodel}[1]{\class{c-model}{#1}} \newcommand{\ctruth}[1]{\class{c-truth}{#1}} \newcommand{\cgrad}[1]{\class{c-grad}{#1}} \newcommand{\closs}[1]{\class{c-loss}{#1}} \newcommand{\cparam}[1]{\class{c-param}{#1}} \newcommand{\csecond}[1]{\class{c-second}{#1}} \renewcommand{\,}{\mkern3mu} \)

S01.2 · Sampling and bias

A free, illustrated module of 4 lessons and one interactive in S01 Exploratory Data Analysis: Population, sample and simple random sampling; Two ways to be wrong: sampling variability and bias; Stratified, cluster and systematic sampling; How samples go wrong: selection, non-response and survivorship.

S01.2 L1 · Core 4 lessons · about 100 min Live · in review

Part of S01 Exploratory Data Analysis in the Statistics and Data Analysis track.

A table is a sample from something larger, and what it can tell you depends on how its rows got in. These four lessons name the population, the frame and the sample, separate the scatter of repeated samples from the bias of a method, show how drawing inside each group and weighting by group shares cuts the scatter, and catalogue the rules for getting into a table (convenience, volunteering, non-response, survivorship, size-biased sampling) that quietly bend every answer.

What you need first. The flats table and the kinds of variable (S01.1), counting combinations (M01.8), and a little NumPy and pandas with seeded random generators (P02). No probability theory is needed.

Lessons

1. Population, sample and simple random sampling S01.2.1 25 min

   A table is a sample from something larger. Name the target population, the sampling frame and the sample, tell a parameter from a statistic, and see why a simple random sample has means that average out to the population mean.

2. Two ways to be wrong: sampling variability and bias S01.2.2 25 min

   Repeated samples scatter around their own centre, and a biased method puts that centre off the truth. Measure both with means alone, watch the scatter shrink as samples grow, and see why no sample size repairs a sample drawn from the wrong place.

3. Stratified, cluster and systematic sampling S01.2.3 25 min

   Draw inside each district and weight the district means by their shares of the city: the course’s first weighted mean. See why stratifying cuts the scatter, what whole buildings and every 25th listing save and risk, and how a stratified data split uses the same idea.

4. How samples go wrong: selection, non-response and survivorship S01.2.4 25 min

   Convenience and volunteer samples, frames that leave units out, non-response and survivorship, each worked in counts and means. Derive why asking members instead of groups weights each group by its size, and ask of any data set who is missing and why.

The interactive

S01.2-X1 · Sampling bench. Draw sample after sample from 48 invented flats, by lot, from the corner nearest to hand, or from the flats that choose to answer. Each sample’s mean drops onto a number line, and the read-out tracks the centre, the bias and the scatter as the sample size grows. It lives in Two ways to be wrong: sampling variability and bias.