The code behind it: build the t-statistic from scratch¶

Part 2 · Lesson 15 · the code behind it · data: ToothGrowth · Swim (paired) (same as Lesson 15)

↩ A companion to Lesson 15: t-tests: one-sample, two-sample & paired.

↩ Back: Lesson 15: t-tests: one-sample, two-sample & paired · ↑ Course map · Also: Another dataset →

In Lesson 15 you ran three t-tests with one-line library calls — stats.ttest_1samp, stats.ttest_ind(..., equal_var=False), stats.ttest_rel. They each printed a t and a p and you trusted them. This companion opens the hood: you'll build that t yourself, straight from the formula, so a t-statistic stops being a number SciPy hands you and becomes three lines of arithmetic you could write from memory.

Same guinea pigs, same swimmers, same final numbers — but the focus shifts from what the test means to how t is actually computed. We'll go in the order a programmer works:

1. The goal — what we're building.
2. The data in code — load it the same way the core lesson does, and pull out the groups.
3. Build it step by step — the one-sample, Welch two-sample, and paired t, each from its formula, each checked against scipy.stats.ttest_* (print match? True).
4. The idiom — vectorized differences for the paired test, and the Welch degrees-of-freedom formula.
5. Refactor into functions — wrap it all as reusable t_test_one_sample() and welch_t().
6. Now you code — write-it-yourself exercises.
7. What you learned — the coding and the statistics.

You don't need Lesson 15 fresh in mind, but it helps. Run each grey cell with Shift + Enter, top to bottom.

New to notebooks? The grey boxes below are code. Click one and press Shift + Enter to run it — the result (a number, a table, or a chart) appears right underneath. Run them top to bottom. You do not need to understand the first "Setup" cell — it just loads the tools and the data, so run it and move on. And you can't break anything: re-running a cell is always safe. (Brand new to all this? The five-minute Lesson 0 — How these notebooks work shows you the basics first.)

1) The goal¶

By the end of this notebook you'll have written the t-statistic three ways — for a single group against a target, for two independent groups (Welch's version), and for matched pairs — and confirmed each one reproduces scipy.stats.ttest_* to the last decimal. Nothing here needs new statistics; everything is the Python underneath the one-line calls you already ran.

The whole lesson rests on one formula, a signal-to-noise ratio:

$$t = \frac{\text{estimate} - \text{null value}}{\text{standard error}}.$$

The estimate is a sample mean (or a difference of means); the standard error is how much a mean of that size typically wobbles. Every test below is just what we plug into the numerator and denominator.

The tools we'll use, and what each is for:

Tool	What it is	What we use it for
pandas	tables (DataFrame) and columns (Series)	loading the CSVs, selecting groups
NumPy	fast math on whole arrays at once	the formulas (mean, std, sqrt, ...)
scipy.stats	a library of ready-made tests	the ttest_* calls we check against

One rule we'll repeat like a mantra: a sample standard deviation divides by $n-1$, written ddof=1 in NumPy/pandas. Get that wrong and your t won't match SciPy — so it appears in every formula below.

2) The data in code¶

Before any formula, a programmer interrogates the object: how big is it, what are the columns, what do the first rows look like? Then we pull out exactly the arrays we'll compute on.

• ToothGrowth — 60 guinea pigs, each given vitamin C as orange juice (OJ) or ascorbic acid (VC); len is the tooth-cell length (the response). We'll group by supp.
• Swim — 12 swimmers, each timed in a wetsuit and a swimsuit; the two velocity columns are wet.suit.velocity and swim.suit.velocity. The same swimmer appears in both columns — that's the pairing.

3) Build it step by step¶

Now the heart of it. Each test is a one-line scipy call and a short formula — we'll write the formula in NumPy and check the two agree to the last decimal. Once you've built it, the library call is no longer a mystery box.

One-sample t — a mean against a target¶

The question from the core lesson: is the OJ group's mean tooth growth different from a benchmark of 18 units? The formula is a z-score with the sample spread in the denominator:

$$t = \frac{\bar{x} - \mu_0}{s/\sqrt{n}}, \qquad df = n - 1.$$

The two-sided p-value is the area in both tails beyond $|t|$: 2 * stats.t.sf(abs(t), df), where sf is the "survival function" (the area to the right). Build it, then check against stats.ttest_1samp.

$t = 2.21$ on $df = 29$, giving $p = 0.0353$ — and our from-scratch numbers match SciPy exactly. The denominator s/sqrt(n) is the only subtle part: it uses the sample SD (ddof=1), because in real life we never know the true population spread.

Two-sample Welch t — two independent groups¶

Now compare OJ against VC (different guinea pigs, so independent groups). Welch's t-test does not assume the two groups share a variance, so each group keeps its own $s$ and $n$:

$$t = \frac{\bar{x}_1 - \bar{x}_2}{\sqrt{\dfrac{s_1^2}{n_1} + \dfrac{s_2^2}{n_2}}}.$$

The degrees of freedom are not simply $n_1+n_2-2$; Welch uses the Welch–Satterthwaite approximation, which we build below. Then p = 2 * stats.t.sf(|t|, df) as before.

$t = 1.92$ on a fractional $df = 55.31$ (that decimal df is the Welch–Satterthwaite approximation at work), giving $p = 0.0606$ — all three numbers match SciPy. Note the p-value is just above 0.05: a sizable sample gap that the test still calls non-significant. "Not significant" means "couldn't rule out zero," not "no effect."

Paired t — the key trick: reduce each pair to one difference¶

The swim data has the same swimmer in two columns. The paired t-test is not a new formula — it is the one-sample test applied to the differences $d_i = \text{wet}_i - \text{suit}_i$, tested against 0. Collapse the 12 pairs to 12 differences, then it's exactly the Section-3.1 recipe with $\mu_0 = 0$.

$t = 12.32$ on $df = 11$, $p \approx 8.9\times10^{-8}$ — a thunderingly significant wetsuit effect, and our from-scratch t matches both ttest_rel and ttest_1samp on the differences. That equality is the whole idea: a paired test is a one-sample test on the differences.

Why pairing helps — the headline contrast¶

Now the punchline from the core lesson. What if we ignore the pairing and (wrongly) run a two-sample test on the same two columns, pretending the 12 wetsuit times and 12 swimsuit times are unrelated groups?

Same data, opposite verdict: the correct paired test gives $p \approx 9\times10^{-8}$, the wrong unpaired one gives $p \approx 0.185$. Pairing subtracts away the large between-swimmer variation (the speeds are correlated $r = 0.99$), shrinking the standard error so the same gap looks decisive. If your two numbers come from the same unit, pair them.

4) The idiom — vectorized differences, and building the Welch df¶

Vectorized differences vs. a loop¶

The paired test hinges on d = wet - suit. NumPy subtracts the two arrays element by element in one expression — no loop over swimmers. You could write the loop; it gives the same numbers but is slower and noisier to read. Compare them.

Reading the Welch df formula¶

The scariest line in Section 3 was the Welch–Satterthwaite degrees of freedom. It is just a weighted combination of the two groups' variance contributions:

$$df = \frac{\left(\dfrac{s_1^2}{n_1} + \dfrac{s_2^2}{n_2}\right)^2} {\dfrac{(s_1^2/n_1)^2}{n_1-1} + \dfrac{(s_2^2/n_2)^2}{n_2-1}}.$$

Naming the two pieces a = s1**2/n1 and b = s2**2/n2 (which is exactly what we did) turns that wall of symbols into one readable line. Watch how it lands between the smallest group's $n-1$ and the pooled $n_1+n_2-2$.

5) Refactor into reusable functions¶

You've now written the same handful of lines a few times. The programmer's reflex is to wrap repeated work in a function: name it, give it inputs, return the result. Then "run a one-sample t-test" or "run a Welch test" is one call you can reuse on any data — and a paired test is just t_test_one_sample on the differences, with no new code at all.

A function is a contract: give me data, I'll give you (t, df, p). Once it's correct, you stop thinking about the formulas and start thinking in groups and comparisons — the leap from "running statistics" to "doing statistics in code." (And yes, scipy.stats.ttest_* exists; the point was to build it so you know what it's doing.)

6) Now you code¶

Write the code yourself. Add a new cell (the + button in the toolbar) and try each — predict the answer before you run it.

1. A one-sided p-value. Our t_test_one_sample returns a two-sided p (it doubled the tail). For the OJ group vs 18, the alternative "the mean is greater than 18" needs just the right tail: stats.t.sf(t, df) with no factor of 2. Compute it for t_test_one_sample(oj, 18.0) and confirm it's exactly half the two-sided p.

2. Pooled (equal-variance) two-sample t. Write pooled_t(a, b) using the equal-variance formula: $s_p = \sqrt{\dfrac{(n_a-1)s_a^2 + (n_b-1)s_b^2}{n_a+n_b-2}}$, then $t = (\bar a - \bar b)/(s_p\sqrt{1/n_a + 1/n_b})$, with $df = n_a+n_b-2$. Test pooled_t(oj, vc) against stats.ttest_ind(oj, vc, equal_var=True) (note equal_var=True, the pooled version). It should match.

3. Paired without the helper. Build the paired t for the swim data in three lines without calling t_test_one_sample: compute d = wet - suit, then t = d.mean() / (d.std(ddof=1)/np.sqrt(len(d))), then the p-value. Check it equals stats.ttest_rel(wet, suit).

4. A different group and target. The strongest tooth-growth effect is at the highest dose. Pull oj2 = tooth[(tooth['supp']=='OJ') & (tooth['dose']==2.0)]['len'].to_numpy() and vc2 = tooth[(tooth['supp']=='VC') & (tooth['dose']==2.0)]['len'].to_numpy(), then run welch_t(oj2, vc2). Predict first: at dose 2.0 the two delivery methods are famously almost identical — does your from-scratch t come out near zero?

What you learned¶

Coding

• The t-statistic is one formula — (estimate - null) / standard_error — and the only moving parts are what you put in the numerator and denominator. You wrote all three tests from it.
• A two-sided p-value is 2 * stats.t.sf(abs(t), df); sf is the right-tail area, and you double it for both tails. Halve it for a one-sided test.
• Vectorization — d = wet - suit subtracts two whole arrays in one expression — replaces the loop, reads as the math, and runs far faster.
• Boolean masks (tooth.loc[tooth['supp']=='OJ', 'len']) pull one group's response out of a tidy table.
• Refactor repeated work into functions (t_test_one_sample, welch_t) with clear inputs and a return, then reuse them — the paired test needed no new function, just t_test_one_sample(wet-suit).

Statistics (now demystified)

• The one-sample t = (\bar{x}-\mu_0)/(s/\sqrt{n}) on $df=n-1$: you got $t=2.21$, $p=0.0353$, matching ttest_1samp exactly.
• The Welch two-sample t keeps each group's own $s^2/n$ and uses the Welch–Satterthwaite $df$ (a fractional $55.31$, not an integer): $t=1.92$, $p=0.0606$, matching ttest_ind(equal_var=False).
• The paired test is literally a one-sample t on the differences $d=\text{wet}-\text{suit}$ against 0 ($t=12.32$, $p\approx9\times10^{-8}$) — ttest_rel and ttest_1samp(d) give the identical number.
• Pairing beats ignoring it: the correct paired test ($p\approx9\times10^{-8}$) detected an effect the wrong unpaired test ($p\approx0.185$) missed, because pairing subtracts away large between-subject variation (speeds correlated $r=0.99$) and shrinks the standard error.
• Everywhere, a sample SD uses ddof=1 (divide by $n-1$); NumPy's default ddof=0 would make every t come out a hair too big and stop matching SciPy.

Back to the core lesson (Lesson 15: t-tests: one-sample, two-sample & paired) for the meaning of these numbers, or read on through the course.

---

↩ Back to the lesson: Lesson 15: t-tests: one-sample, two-sample & paired · ↑ Course map · Also: Another dataset