ROC Cutoff Selection

There is no universally best ROC cutoff. The appropriate threshold depends on the cost of missed positive cases, the cost of false alarms and whether the threshold was chosen for exploration, screening, confirmation or validation of an existing rule.

Choose the cutoff purpose before interpreting the result

Balanced cutoff using the Youden index

J = sensitivity + specificity − 1

Maximizing J gives sensitivity and specificity equal importance. It is a useful exploratory summary when false negatives and false positives have comparable consequences. It does not account for prevalence, downstream costs or the consequences of a decision, so “optimal” should be understood as optimal for this mathematical rule.

Screening: prioritize sensitivity

Screening aims to miss as few positive cases as practical. Enter the minimum acceptable sensitivity. numiqo then chooses the observed cutoff with the highest specificity among those that reach the requested sensitivity. If no observed cutoff meets the request, no operating point is reported.

The requested minimum applies to the estimate in the current sample. Its confidence interval can extend below the requested value, so the result is not a guarantee of population performance.

Confirmatory testing: prioritize specificity

A confirmatory or rule-in threshold is intended to limit false positive classifications. Enter the minimum acceptable specificity. numiqo selects the eligible observed cutoff with the highest sensitivity. The same uncertainty warning applies: a sample estimate meeting the target does not prove that the population value meets it.

Evaluate a prespecified cutoff

Enter a threshold defined before examining the current outcome data—for example, a published threshold, a manufacturer’s rule or a cutoff from a development sample. numiqo evaluates the value exactly as entered and does not search for a better-looking threshold. This separates validation of an existing rule from data-driven optimization.

Measures reported at the selected cutoff

PPV and NPV depend on the prevalence represented in the analyzed data. They can change substantially when the same test and cutoff are applied in a population with a different prevalence. LR+ and LR− do not use prevalence directly.

Bootstrap confidence intervals

numiqo uses 2,000 stratified bootstrap resamples, drawing positive and negative cases separately. For an optimized cutoff, the cutoff-selection step is repeated within each resample. For an existing cutoff, the same entered threshold is evaluated in every resample. Percentile 95% intervals are reported for the cutoff when it is estimated, sensitivity, specificity, Youden J, LR+, LR−, PPV and NPV when defined.

Avoid optimistic cutoff claims

Selecting and evaluating a cutoff on the same observations generally gives an optimistic impression of performance. Report that the cutoff was data-derived, include its uncertainty and, where the decision matters, validate it in independent data. Also state whether high values (value ≥ cutoff) or low values (value ≤ cutoff) indicate the positive outcome.

Related ROC guides

• ROC curve calculator overview
• Comparing paired and independent ROC curves
• Methods and pROC reference validation
• ROC curve and AUC tutorial

References

• Youden WJ. Index for rating diagnostic tests. Cancer. 1950.
• Hajian-Tilaki K. Receiver operating characteristic curve analysis for medical diagnostic tests.