How to use the statistical power calculator
- Choose the test: a two-sample t test (two separate groups), a one-sample or paired t test (one group, or before and after on the same people), or a test of two proportions.
- Choose whether to find the sample size or the power. For the power, also enter the sample size you have.
- Enter the effect size d for a t test (the difference in means divided by the standard deviation), or the two proportions you expect. Then set the significance level, the target power and whether the test is one-sided or two-sided.
- Read the answer and the table of other sample sizes. The line "smallest effect at the target power" shows the smallest effect that your sample could detect.
Formula
t tests: power = P(noncentral t beyond the critical value)
noncentrality = d x sqrt(n) for one sample or pairs, d x sqrt(n / 2) for two groups of n
degrees of freedom = n - 1 or 2n - 2
proportions: power = Phi((|p2 - p1| - z x SE0) / SE1), with SE0 from the pooled rate and SE1 from each rate
Phi is the standard normal cumulative distribution and z is the normal value for the significance level. The sample size is the smallest whole number whose power reaches the target.
Two groups and a medium effect
Take two groups, an effect size of d = 0.5, a two-sided significance level of 0.05 and a target power of 0.80. The smallest group size that reaches 80 percent power is 64 per group, 128 people in all, and its power is 80.15 percent. With the same settings and 30 per group, the power is only 47.8 percent. A one-sample or paired test with d = 0.5 needs 34 pairs.
For two proportions of 10 percent and 15 percent with the same settings, the answer is 686 per group, with a power of 80.02 percent. Doubling the effect size cuts the sample needed to about a quarter, which is why a realistic effect size matters more than any other input.
What power means
Power is the chance that a study finds an effect when the effect is real. A power of 80 percent means that, if the true effect is exactly the size you entered, four out of five studies like it would reach the significance level. It says nothing about how likely the effect is to be real in the first place.
Low power wastes effort in both directions. A small study often misses real effects, and the effects it does find tend to be overstated. The table of other sample sizes shows how quickly power grows at first and how slowly it grows near the top, so you can see the price of each extra step.
Choosing an effect size
The effect size should come from what would matter in practice or from earlier work, not from what makes the sample small. Values of d near 0.2, 0.5 and 0.8 are often called small, medium and large, but those labels are only conventions and fit some fields badly. If you are unsure, run the calculation for a few values. The tool also reports the smallest effect your sample can detect, which is a useful sanity check.
For related ideas, see the sample size calculator for surveys, the Cohen's d calculator to turn two groups into an effect size, and the p value calculator. When the data may be far from a bell curve, the Wilcoxon signed-rank test calculator tests paired data with ranks.
Assumptions and limits
- The t test power uses the exact noncentral t distribution with equal group sizes, equal variances and data that are close to normal.
- The proportion test uses the usual normal approximation with a pooled standard error under the null, which is rough for very small groups or rates near 0 or 1.
- The effect size, significance level and target power are example values you can change. Nothing about your own study is assumed.
- Dropouts, missing data, unequal groups and repeated looks at the data all reduce power and are not modeled.
Frequently asked questions
Why does a one-sided test need fewer people?
A one-sided test puts the whole significance level in one tail, so the cutoff is lower and an effect in that direction is found sooner. It only makes sense when an effect in the other direction would be treated the same as no effect, and you must decide that before you see the data.
What is the difference between power and the significance level?
The significance level is the chance of a false alarm when there is no effect. Power is the chance of a true detection when there is an effect of the size you entered. Lowering the significance level makes false alarms rarer but lowers the power unless the sample grows.
Can I use this after a study has finished?
Using the observed effect to compute power afterward adds little, because it just restates the p value. Planning calculations before the study are the useful ones. After the study, report the effect and its confidence interval instead.
Why does the proportion answer differ slightly from other tools?
Several formulas exist for comparing two proportions. This page uses a pooled standard error under the null and a separate one under the alternative. Others use a simpler form or a continuity correction, and answers can differ by a few people per group.