What is uncertainty sampling?
Uncertainty sampling is a way to decide what to label next. Instead of picking data at random, it runs the current model over unlabeled samples and selects the ones where the model is least sure, since those are the examples most likely to teach it something new. The newly labeled samples are added to training, the model improves, and the cycle repeats.
It is the most common query strategy within active learning.
Key takeaways
- It labels the samples the model is least confident about.
- Those uncertain samples are the most informative to learn from.
- It reaches strong performance with fewer labels than random sampling.
How it works
The model produces a confidence measure for each unlabeled sample, such as a low top score, a small margin between the top two classes, or high entropy across classes. The most uncertain samples are sent for labeling, then folded back into training. Because it relies on confidence, it works best when the model is reasonably calibrated.
Why it matters
Labeling budget is finite, and uncertainty sampling spends it where the model is weakest, which usually improves performance faster than labeling everything or sampling randomly. It is a cornerstone of active learning and a practical lever for cutting annotation cost.
Frequently asked questions
How does the model measure uncertainty?
Commonly through low confidence, a small margin between the top predictions, or high entropy across the class probabilities.
Why does calibration matter for uncertainty sampling?
If confidence scores do not reflect true accuracy, the model may flag the wrong samples as uncertain, so good calibration makes the strategy more reliable.
Related terms