How this calculation works
A scored classifier can trade missed positives against false alarms as its threshold changes. ROC coordinates show that tradeoff across the observed sample, while AUC summarizes score ordering rather than one selected cutoff. This page groups exact score ties, calculates empirical AUC and compares user-entered false-positive and false-negative costs at the observed thresholds. It does not turn an empirical minimum into a validated future operating rule.
ROC area = trapezoid sum over tied score groups; sample cost = FP × costFP + FN × costFNA worked example
The example has two positives and two negatives. A positive score 0.9 outranks both negatives, while a positive score 0.8 outranks the negative 0.1 and ties the negative 0.8. The pairwise credit is three and a half out of four, giving AUC 0.875. With equal unit error costs, the smallest observed total error cost is one, achieved by more than one threshold.
Good to know
This is an empirical descriptive calculation. No cross-validation, confidence interval, prevalence adjustment or probability calibration is supplied. A medical diagnostic or other consequential threshold needs separate validation and domain acceptance criteria. The page does not infer those criteria from the sample or its lowest arithmetic cost.
What this calculation answers
A scored classifier can trade missed positives against false alarms as its threshold changes. ROC coordinates show that tradeoff across the observed sample, while AUC summarizes score ordering rather than one selected cutoff.
This page groups exact score ties, calculates empirical AUC and compares user-entered false-positive and false-negative costs at the observed thresholds. It does not turn an empirical minimum into a validated future operating rule.
The Binary confusion matrix precision recall and F1 calculator summarizes one classification threshold. A ROC curve follows several thresholds and groups tied scores before computing area.
Use the right input conventions
Each row supplies a name, score and observed binary label. At least one positive and one negative label are required. Choose whether higher or lower scores indicate the positive class before interpreting the curve.
Exact tied scores enter the threshold together. Splitting one tie by arbitrary row order would manufacture a different ROC path and potentially an incorrect AUC.
Interpret the result
AUC assigns half credit to a positive-negative pair with tied scores. It is not classification accuracy or a calibrated probability. Two models can have similar AUC while differing greatly near the threshold relevant to a particular decision.
The minimum-cost threshold uses this sample’s class prevalence and the entered costs. A different deployment population, validation sample or cost model can change that threshold even when the score ranking is unchanged.
Work through the example
The example has two positives and two negatives. A positive score 0.9 outranks both negatives, while a positive score 0.8 outranks the negative 0.1 and ties the negative 0.8.
The pairwise credit is three and a half out of four, giving AUC 0.875. With equal unit error costs, the smallest observed total error cost is one, achieved by more than one threshold.
Where the model stops
This is an empirical descriptive calculation. No cross-validation, confidence interval, prevalence adjustment or probability calibration is supplied.
A medical diagnostic or other consequential threshold needs separate validation and domain acceptance criteria. The page does not infer those criteria from the sample or its lowest arithmetic cost.
A few questions, answered
Why group equal scores before drawing ROC points?
The same score cannot be separated by one deterministic threshold; tied observations must cross together.
What changes in the comparison?
Make missed positives five times as costly as false alarms and compare the sample cost minimum.