What is self-supervised learning?
Self-supervised learning is a way to train models without manual labels by inventing a task whose answer is already contained in the data. Common examples include hiding part of an input and asking the model to reconstruct it, or treating two augmentations of the same image as a matching pair. The model learns rich, general representations in the process, which can then be adapted to specific tasks with far less labeled data.
It sits between supervised learning, which needs labels, and unsupervised learning, which seeks structure without an explicit target.
Key takeaways
- The training signal is generated from the unlabeled data itself.
- It produces general representations that transfer to many tasks.
- It is the dominant way to pretrain foundation models.
How it works
A pretext task is defined, such as masked reconstruction or contrastive matching, and the model is trained on large unlabeled datasets to solve it. Solving the pretext task forces the model to learn useful structure. The pretrained model is then adapted to a target task through fine-tuning or by using its embeddings directly.
Why it matters
Labeled data is expensive and scarce, while unlabeled data is abundant, and self-supervised learning is what converts that abundance into capability. It is the engine behind foundation models and a practical way to cut the labeling burden of building strong visual and multimodal systems.
Frequently asked questions
How is it different from unsupervised learning?
Self-supervised learning defines an explicit predictive task using the data as its own label, while unsupervised learning generally looks for structure without a specific target to predict.
Why is self-supervised learning important for foundation models?
It lets models learn from massive unlabeled datasets, producing transferable representations that only need a small amount of labeled data to specialize.
Related terms