How to use the p-value and t-test calculator
- Choose What do you have. Pick a z, t, chi-square or F statistic if your software already gave you one. Pick a t-test option if you have summary numbers (mean, standard deviation and size) or the raw measurements.
- Enter the numbers. For a statistic, add its degrees of freedom. For raw data, paste each sample separated by commas, spaces or new lines. For paired data, list the two measurements of each pair in the same order.
- Choose the alternative hypothesis: two-tailed if you are testing for any difference, right-tailed if you expect a larger value, left-tailed if you expect a smaller one. Set your significance level, commonly 0.05, before looking at the result.
- Read the p-value and the decision. For t-tests you also get the standard error, a confidence interval for the difference and Cohen's d. Use Copy or Print to keep the result.
Formula
One sample: t = (mean - mu0) / (s / sqrt(n)), df = n - 1
Two samples, Welch: t = (mean1 - mean2) / sqrt(s1^2 / n1 + s2^2 / n2)
Welch df = (a + b)^2 / (a^2 / (n1 - 1) + b^2 / (n2 - 1)), with a = s1^2 / n1 and b = s2^2 / n2
Two samples, pooled: sp^2 = ((n1 - 1) s1^2 + (n2 - 1) s2^2) / (n1 + n2 - 2), SE = sp sqrt(1 / n1 + 1 / n2)
Two-tailed p = 2 x the smaller tail area; one-tailed p = the tail area in the chosen direction
The tail areas are computed from the regularized incomplete beta function (t and F) and incomplete gamma function (chi-square and normal). The confidence interval is the estimate plus or minus the critical t value times the standard error.
Two groups, Welch test
Sample 1 has mean 5.2, standard deviation 1.1 and 12 values. Sample 2 has mean 4.5, standard deviation 1.4 and 10 values. The standard error is the square root of 1.1^2 / 12 + 1.4^2 / 10, which is 0.544824, so t = 0.7 / 0.544824 = 1.284818. The Welch degrees of freedom come to 16.9679.
The two-tailed p-value for that t with those degrees of freedom is 0.21612. That is above 0.05, so the data do not give enough evidence to say the two means differ at that level. Pooling the variances instead gives t = 1.314202 with 20 degrees of freedom and p = 0.20365, the same conclusion.
What a p-value tells you
A p-value is the probability of seeing a statistic at least as extreme as yours if the null hypothesis were true. A small value means the data would be surprising under the null hypothesis, which is evidence against it. It does not give the probability that the null hypothesis is true, and it does not measure the size or importance of an effect.
Choose the significance level (alpha) before looking at the data. If p is at most alpha you reject the null hypothesis, otherwise you do not reject it. Failing to reject is not proof that nothing is going on, especially with a small sample.
How accurate the numbers are
The distribution functions were checked against an independent method: a numeric integration of each probability density, run in a separate Python script on 220 combinations of statistic and degrees of freedom, including very small tail areas. Upper tail areas agreed to better than 1e-10 relative error in every case, and critical values match the standard published table entries for the t, chi-square and F distributions.
Degrees of freedom up to 1,000,000 are accepted. P-values smaller than 1e-300 are shown as such because a computer cannot tell them apart from zero.
Choosing between the tests
Use the Welch version of the two-sample test unless you have a strong reason to believe both groups have the same variance, since it stays reliable when the spreads differ. Use the paired test when each value in one list has a natural partner in the other, such as before and after measurements on the same person.
For a test of counts in a table, the chi-square calculator builds the statistic for you. To turn a value into a z-score first, use the z-score calculator, and for an interval estimate of a mean see the confidence interval calculator.
Assumptions and limits
- The t-tests assume the data are roughly normally distributed or the samples are large enough for the mean to be close to normal, and that observations are independent within and between groups (except for the pairing in a paired test).
- Standard deviations are the sample version that divides by n - 1, the one most software reports. Entering the population version gives a slightly wrong t statistic.
- The tool computes the test you choose and nothing else. It does not correct for running many tests, check the assumptions, or work out sample size or power.
- Statistics are read as typed. A chi-square or F statistic cannot be negative, and the standard deviation of a sample must be above zero.
Frequently asked questions
What is the difference between a one-tailed and a two-tailed p-value?
A two-tailed p-value counts extreme results in both directions and is twice the smaller tail area. A one-tailed p-value counts only the direction you predicted before collecting data. Pick the direction in advance, not after seeing which way the data went.
Why is my p-value slightly different from my statistics program?
Small differences usually come from rounding the statistic or the degrees of freedom before typing them in, or from using pooled variances where the program used the Welch test. Enter the unrounded values, or paste the raw data here, to compare like with like.
Can I get a p-value from a z-score without a table?
Yes. Choose a z statistic, enter the score, and the tool returns the tail area directly. For z = 1.96 the two-tailed p-value is 0.049996, which is why 1.96 is the usual cutoff at the 0.05 level.
What does Cohen's d show in the t-test results?
It is the estimated difference divided by the standard deviation, a way to describe the size of an effect that does not depend on the sample size. Rough labels such as small, medium and large are conventions, not rules, so judge the value in the context of your own field.