How to use the confusion matrix calculator
- Enter the number of true positives, false positives, false negatives and true negatives. The examples show the layout.
- Open the options to change beta for the F-beta score. Beta above 1 favors recall and below 1 favors precision.
- Read the headline accuracy and the key measures above the table.
- Use the table of all measures, with the formula for each, to find the one you need.
Formula
accuracy = (TP + TN) / all, precision = TP / (TP + FP), recall = TP / (TP + FN), specificity = TN / (TN + FP)
F1 = 2 TP / (2 TP + FP + FN), F-beta = (1 + b^2) TP / ((1 + b^2) TP + b^2 FN + FP)
MCC = (TP TN - FP FN) / sqrt((TP + FP)(TP + FN)(TN + FP)(TN + FN))
kappa = (observed agreement - chance agreement) / (1 - chance agreement), Youden's J = recall + specificity - 1
A measure is shown as not defined when its denominator is zero, for example precision when nothing was predicted positive.
A classifier tested on 300 cases
Suppose TP = 86, FP = 14, FN = 9 and TN = 191. Accuracy is 277 / 300 = 92.33%. Precision is 86 / 100 = 86.00%, recall is 86 / 95 = 90.53% and specificity is 191 / 205 = 93.17%. The F1 score is 0.8821 and the F2 score is 0.8958.
The Matthews correlation coefficient is 0.8259 and Cohen's kappa is 0.8253. The negative predictive value is 95.5%, the prevalence in this data is 31.67%, the positive likelihood ratio is 13.26 and the negative likelihood ratio is 0.1017.
Why more than one measure
Accuracy can look good when one class is rare: a model that always says negative is 95% accurate when only 5% of cases are positive. Precision and recall show how well the positive class is handled, F1 combines them, and the Matthews correlation and kappa use all four cells, so they stay informative when the classes are unbalanced.
Precision and the predictive values depend on how common positives are in the group tested, so they do not carry over to a group with a different mix. Sensitivity and specificity are properties of the test itself. To go from them to the chance that a result is right in a given group, use the Bayes Theorem Calculator.
Limits
These measures describe how a classifier or test performed on the counts you enter. They do not carry intervals, and small counts make every measure uncertain. This page is for evaluating models and tests, and it is not medical advice.
More statistics tools
For related statistical tests and measures, try the Test Item Analysis Calculator, Mann-Whitney U Test Calculator and Fisher Exact Test Calculator.
Assumptions and limits
- The four counts come from one set of cases, each counted in exactly one cell.
- Positive means the class of interest, and the labels are known.
- Counts are whole numbers; the example counts are shown until you replace them.
- Percentages are shown to two decimals and ratios to four.
Frequently asked questions
What is the difference between precision and recall?
Precision is the share of predicted positives that are really positive. Recall is the share of real positives that were found. A strict model can have high precision and low recall, and the reverse.
What is the F1 score?
It is the harmonic mean of precision and recall, 2 TP divided by 2 TP plus FP plus FN. It is high only when both are high.
When should I use the Matthews correlation coefficient?
Use it when the classes are unbalanced, because it uses all four cells and gives a low value to a model that only predicts the common class.
Why is a measure shown as not defined?
Its formula would divide by zero. For example, precision cannot be worked out when the model never predicts positive.