A wind-tunnel test of airfoil drag, smooth against rough. Each run scatters with σ = 0.25 N of noise. Decide whether roughness really changes the drag, and watch what replication does.
Each round the tunnel secretly sets the rough airfoil's true effect to δ = 0 or δ = 0.30 N, 50:50. Choose how many runs to make, look at the data, then call it.
Costs 2.0 points (0.25 per run, 2n runs). A correct call earns 10, a wrong one loses 10.
| Source | SS | df | MS | F |
|---|---|---|---|---|
| Finish | – | – | – | – |
| Error | – | – | – | |
| Total | – | – |
| Estimated effect | – |
| 95% CI on the effect | – |
| F crit (α = 0.05) | – |
| p | – |
| Round | n | Effect (N) | p | Your call | Truth | Points |
|---|
Each experiment draws n smooth and n rough runs with noise σ = 0.25 N and tests at α = 0.05. Set δ = 0 to see false positives; set δ above 0 to see power.
Try: δ = 0 and n = 3, then switch on peeking. Then δ = 0.30 with n = 3 and n = 12.
With no real effect, p is spread evenly from 0 to 1 (dashed line), so 5% land below 0.05 by chance.