How to use the tukey hsd post hoc calculator
- Choose a significance level such as 0.05.
- Add one row per group with its name, sample size, mean and standard deviation. Three or more groups are needed.
- Run the calculation. The page pools the group variances into a mean square error and then compares every pair of means.
- Read the table of differences with the q statistic, the adjusted p value and the interval for each pair, and the smallest difference that counts when the groups are the same size.
Formula
Mean square error = sum of (n - 1) x sd^2 over the groups / (N - k), with N - k error degrees of freedom
q = |mean i - mean j| / sqrt(MSE / 2 x (1/n_i + 1/n_j))
Adjusted p = the upper tail of the studentized range distribution at q, with k groups and N - k degrees of freedom
Interval = difference plus or minus q critical x sqrt(MSE / 2 x (1/n_i + 1/n_j))
With unequal sample sizes this is the Tukey-Kramer version. The studentized range distribution is evaluated by numerical integration that agrees with reference software to about eight digits.
Three groups of five
Group A has mean 10 and standard deviation 2, Group B has mean 12.5 and 2.2, and Group C has mean 15 and 1.8, each with five values. The mean square error is 4.02667 with 12 error degrees of freedom, and the critical q at the 0.05 level is 3.77293.
The smallest difference that counts for equal groups of five is 3.3858. Groups A and B differ by 2.5 (adjusted p 0.1622, interval -0.88584 to 5.8858), so that pair is not flagged. A and C differ by 5, which is above the cutoff, so one of the three pairs differs.
Why not many t tests
Comparing every pair with separate t tests raises the chance of at least one false positive well above the level you chose. Tukey's method controls the error rate across all the pairs at once by using the studentized range, so the adjusted p value already accounts for the number of groups.
Run the overall test first with the ANOVA calculator. For other ways to adjust many p values, see the multiple comparison p adjuster.
Reading the intervals
An interval that does not contain zero means that pair differs at your level. The intervals are simultaneous, so all of them together hold with the stated confidence. To describe how big a difference is in practical terms, pair the result with an effect size from the Cohen's d calculator.
Assumptions and limits
- Observations are independent, and each group is roughly normal.
- The groups share a common variance. The pooled mean square error is used for every pair.
- The page works from summaries. It cannot check these assumptions for you.
- Choose the significance level before you look at the results.
Frequently asked questions
When do I use Tukey HSD?
Use it after an analysis of variance shows that group means are not all equal, to find which pairs differ. It compares every pair while controlling the overall error rate.
What does the adjusted p value mean?
It is the p value for a pair after allowing for the number of comparisons. Compare it directly with your significance level.
Does it work with unequal group sizes?
Yes. The page uses the Tukey-Kramer form, which adjusts the standard error for each pair.
What is critical q?
It is the upper percentile of the studentized range for the number of groups and the error degrees of freedom. A q statistic above it marks a pair that differs.