p-values, errors & power

A hypothesis test draws a line in the sand. The null world (no effect) is the blue curve; the true world (a real effect of size \(\delta\), in standard-error units) is the orange curve. We reject the null when the test statistic lands past the critical value set by \(\alpha\). Drag the pieces and watch the four outcomes trade off.

Try it: with \(\delta = 0\) (no real effect), the reject rate is exactly \(\alpha\) — that's the Type I error rate. Slide \(\delta\) up, or lower \(\alpha\): power (catching a real effect) rises with effect size and sample size, and falls as you demand stricter evidence.