1ConceptBeginner 1
What is the difference between supervised and unsupervised learning?
Short answer
Supervised learning trains on examples with known answers, such as emails labelled spam or not, and learns to predict the label. Unsupervised learning has no labels and finds structure on its own, such as grouping customers into segments. Language model pre-training is often called self-supervised: the next word is the label.
The explanation
In supervised learning you have input and output pairs and measure success by comparing predictions with the true outputs, for example classification and regression. In unsupervised learning you only have inputs and look for patterns: clustering, dimensionality reduction, anomaly detection.
Self-supervised learning sits between them: the labels come from the data itself. Predicting the next token in text needs no human labelling, which is why models can train on enormous corpora. Fine-tuning a model on labelled examples is then supervised again.
If a telecom company has past customers labelled 'left' or 'stayed', a churn model trained on them is supervised learning. If it only has usage data with no labels and wants to discover natural customer groups, such as night-time data users or mostly-calls users, clustering them is unsupervised learning. Language models learn by predicting the next word in text, which is called self-supervised.
They may follow up with
- Where does reinforcement learning fit?
- Why is next-token prediction called self-supervised?
Avoid
- Saying unsupervised means no data is needed
- Mixing up clustering and classification