How to use the runs test for randomness
- Choose Numbers to compare each value with the median, or Two symbols for a sequence like heads and tails.
- Paste the values in the order they happened, separated by spaces, commas or new lines.
- Read the number of runs against the number expected for a random order.
- Use the exact p value for small samples and the normal approximation for larger ones.
Formula
Expected runs = 2 n1 n2 / (n1 + n2) + 1
Variance = 2 n1 n2 (2 n1 n2 - n1 - n2) / ((n1 + n2)^2 (n1 + n2 - 1))
z = (runs - expected) / sqrt(variance)
Exact: P(R = 2k) = 2 C(n1-1, k-1) C(n2-1, k-1) / C(n1+n2, n1)
A run is an unbroken stretch of the same symbol. n1 and n2 count the two kinds. The exact odd-run probabilities add two products of binomials, and the two-sided p value doubles the smaller tail.
Two sequences
For HHTTHTHH there are 5 H and 3 T. The runs are HH, TT, H, T and HH, so R = 5. Under random order the expected number is 2 x 5 x 3 / 8 + 1 = 4.75, the variance is 1.4732 and z = 0.2060, with a normal p value of 0.8368. The exact p value is 1, so nothing looks unusual.
The 20 values 11 14 19 3 17 10 4 8 6 14 18 10 15 20 6 10 3 9 16 11 have a median of 10.5, ten above and ten below, in 9 runs against 11 expected, with z = -0.9189, a normal p value of 0.3581 and an exact p value of 0.4844. By contrast 20 coin results that alternate HTHT... have 20 runs against 11 expected, z = 4.1352 and a p value below 0.0001.
What a runs test can and cannot show
Too few runs means like items cluster together, which suggests a trend, a drift or a streaky process. Too many runs means the values flip back and forth more than chance would, which suggests alternation or a correcting process. A small p value says the order is unlikely to be random but does not say which of these is the cause.
The test looks only at order and ignores size. To check dependence at several lags use the autocorrelation calculator, or to test whether values look normally distributed use the Shapiro-Wilk calculator. Convert any result to a probability with the p value calculator.
How numbers are turned into two symbols
With numbers, each value is marked as above or below the median. Values exactly equal to the median are left out, because they belong to neither side, and the page tells you how many were dropped. The runs are then counted on what remains. This version is sometimes called the runs test above and below the median.
Assumptions and limits
- The values are listed in the order they were collected and each is a single observation.
- Both kinds must appear. With numbers, at least some values must be above the median and some below.
- The exact p value is shown for up to 200 usable values. The normal approximation is rough when either kind has fewer than 10.
- The two-sided exact p value doubles the smaller tail and is capped at 1.
Frequently asked questions
How many data points do I need?
At least 4 usable values are needed to run the test, but it has little power with so few. Try for 20 or more, with at least 10 of each kind, so that the normal approximation is reasonable.
What does a p value of 0.05 mean here?
It means that if the order were random, a number of runs this far from the expected one would happen about 5 times in 100. It is evidence against randomness, not proof, and it does not explain why the order is unusual.
Why are values equal to the median dropped?
A value equal to the median is neither above nor below it, so it cannot be given a symbol. Dropping ties is the standard approach, and the page reports how many were removed.
Can I test a yes or no series?
Yes. Choose two symbols and type them in order, such as Y and N, or write them together like YYNYN. The test counts how often the answer switches.