How to use the auc roc calculator
- Paste one line per case with a score and a label, such as "0.82, 1". Use 1 for a positive case and 0 for a negative case.
- A higher score should mean more likely positive. The example data show the layout.
- Open the options to change the confidence level for the interval.
- Read the area under the curve, the interval, the ROC points and the cutoff with the largest true positive rate minus false positive rate.
Formula
for each score cutoff c: TPR = positives with score >= c / all positives, FPR = negatives with score >= c / all negatives
AUC = area under the points (FPR, TPR) by the trapezoid rule
check: AUC = (rank sum of positives - n+ (n+ + 1) / 2) / (n+ x n-), with ties ranked by average
SE (Hanley and McNeil) = sqrt((A (1 - A) + (n+ - 1)(Q1 - A^2) + (n- - 1)(Q2 - A^2)) / (n+ n-)), Q1 = A / (2 - A), Q2 = 2 A^2 / (1 + A)
The two ways of finding the area are both calculated and agree. The area is the chance that a randomly chosen positive case has a higher score than a randomly chosen negative one, with ties counted as half.
Twelve scored cases
Six positive and six negative cases with scores 0.95 (positive), 0.88 (positive), 0.80 (negative), 0.74, 0.66, 0.61, 0.55, 0.47, 0.38, 0.30, 0.22 and 0.10 give an area of 0.7778. Of the 36 pairs of one positive and one negative, 28 have the positive higher, and 28 / 36 = 0.7778.
The standard error is 0.1402, so the 95% interval runs from 0.503 up to 1 (the top is capped at 1). The cutoff with the largest true minus false rate is a score of 0.66 or more, which catches 4 of 6 positives (0.667) and 1 of 6 negatives (0.167).
Reading the curve
An AUC of 0.5 means the score ranks cases no better than chance and 1 means it ranks every positive above every negative. Below 0.5 means the scores run the wrong way and can be flipped. The curve itself shows the trade between catching positives and raising false alarms at every cutoff.
The cutoff with the largest true minus false rate (Youden's J) is one convention. It treats both kinds of error as equally costly, which is often not true. See the Confusion Matrix Calculator for the measures at a single cutoff.
Cautions
The interval is a normal approximation that works best with a fair number of cases of each kind. With very few positives or negatives it can be misleading, and the true uncertainty is larger. Use this page to evaluate models and tests on data you already have, and not to make decisions about a person.
More statistics tools
For related statistical tests and measures, try the Confusion Matrix Calculator, Test Item Analysis Calculator and Mann-Whitney U Test Calculator.
Assumptions and limits
- Each line has one score and one label, and the labels are only 0 and 1.
- A higher score means more likely to be positive.
- Tied scores are treated as a single cutoff, and ties count as half in the rank method.
- The interval uses the Hanley and McNeil standard error with a normal approximation.
Frequently asked questions
What is AUC in a ROC curve?
It is the area under the curve of true positive rate against false positive rate. It equals the chance that a random positive case scores higher than a random negative case.
What is a good AUC value?
There is no universal threshold. 0.5 is chance and 1 is perfect, and what counts as useful depends on the task, the alternatives and the cost of errors.
How is the best cutoff chosen?
The page shows the cutoff with the largest true positive rate minus false positive rate. That treats both errors as equally costly, so choose a different cutoff if one error is worse.
Why do the two AUC values agree?
Both describe the same quantity. The trapezoid area under the curve equals the share of positive and negative pairs ranked correctly, with ties as half.