Statistical versus practical significance

Learning objectives

• Distinguish statistical significance from practical significance.
• Predefine a minimum important difference for garment work.
• Report effect, CI, p-value and decision together.

Core idea

A small p-value only says the observed data are unlikely under H0. Whether the effect matters depends on tolerance, cost, capacity and customer risk.

Decision method

• Before analysis, state the minimum important difference (MID).
• After analysis, report point estimate, CI and p-value with units.
• Compare CI limits with MID, not p with alpha alone.
• Nonsignificance is not proof of equality; check the CI width.

Garment-factory example

A 0.05 cm mean chest-width shift is statistically significant on n = 2,000 but sits well inside a ±1.0 cm tolerance — practically irrelevant. A 3-point FPY gain (72 → 75%) with CI (1.4, 4.6) saves real repair cost and is practically material.

Method

1. Define MID up front.
2. Compute effect + CI + p.
3. State statistical and practical conclusions separately.

Common mistakes

• Reporting only p without effect size.
• Treating p > 0.05 as proof of no difference.