Zero-shot labeling is auto-labeling where a model annotates classes it was never explicitly trained on, by being told the target categories at inference time, usually through a text prompt. It labels new concepts with no task-specific training examples.
Zero-shot labeling lets you label categories the model has never seen labeled examples of. Instead of training a detector on thousands of "forklift" images, you give a foundation or vision-language model the class names, often as a text prompt, and it applies them based on the general associations it learned in pretraining.
"Zero-shot" means zero task-specific training examples. The model generalizes from language and broad knowledge to your label set. Few-shot labeling is the close cousin, where you supply a handful of examples to sharpen results.
Key takeaways
The model labels classes it was never trained on, specified at inference, usually through text.
Zero task-specific examples are needed, because the knowledge comes from pretraining.
It is fast and flexible but less reliable than a model trained on your exact classes.
What zero-shot labeling provides
How zero-shot compares to few-shot and fully trained approaches.
How zero-shot compares to few-shot and fully trained approaches.
Approach
Training examples for your classes
Trade-off
Zero-shot
None
Fastest and most flexible, least precise
Few-shot
A handful
A little setup, better fit
Trained or fine-tuned
Many
Most accurate, most labeling upfront
How it works
You give a model your class names, run it over the data, and its predictions become candidate labels. In FiftyOne, you apply zero-shot models, store their predictions, and review which classes the model handles confidently versus which need examples or human correction.
Why it matters
Zero-shot labeling is how you bootstrap a brand-new label set with no labeled data to start from, which is invaluable for exploration and cold starts. Results hinge on prompt wording far more than people expect. The same class called "person," "pedestrian," or "human" can yield noticeably different recall, because the model is matching your text against its learned associations, not a fixed definition. So zero-shot quality is partly a prompt-engineering problem, and ambiguous or jargon class names are where it silently underperforms.
Frequently asked questions
What does "zero-shot" mean?
The model labels classes with zero task-specific training examples, generalizing from pretraining.
What is the difference between zero-shot and few-shot labeling?
Zero-shot uses no examples of your classes. Few-shot uses a small number to improve accuracy.
Is zero-shot labeling accurate enough to ship?
It is great for a first pass and exploration, but specialized or ambiguous classes usually need human review or a trained model.