What is a confidence score?
A confidence score is a number a model outputs alongside each prediction to indicate how sure it is, typically ranging from 0 to 1. Setting a threshold on that score converts the model's raw outputs into concrete decisions: keep predictions above the cutoff, discard those below it. Where you set the threshold trades precision against recall.
A high confidence score is not the same as a high probability of being correct unless the model is well calibrated, which is a common and important pitfall.
Key takeaways
- A confidence score expresses a model's certainty in a single prediction.
- A threshold on the score decides which predictions are kept.
- Confidence is only trustworthy as a probability if the model is calibrated.
How it works
Most classifiers and detectors produce a score per class, often from a softmax or sigmoid layer. Applications then apply a threshold, and in detection an additional step like non-maximum suppression removes overlapping duplicates. Sweeping the threshold is exactly what generates precision-recall and ROC curves.
Why it matters
The confidence threshold is one of the most consequential knobs in a deployed model, because it sets the balance between missing real cases and raising false alarms. Treating raw confidence as a true probability without checking calibration can lead to overconfident, poorly-tuned systems, which is why calibration and threshold selection go hand in hand.
Frequently asked questions
Is a confidence score the same as a probability?
Only if the model is calibrated. Many models are overconfident, so a 0.9 score does not always mean a 90 percent chance of being correct.
How do you choose a confidence threshold?
By looking at the precision-recall trade-off and picking the operating point that matches the tolerance for false positives versus missed detections.
Related terms