Label smoothing exists to fix one quiet problem:
Neural networks become overconfident.
In standard classification, targets are one-hot:
correct class → 1
others → 0
This tells the model:
Be absolutely certain.
🔍 What Label Smoothing Does
Instead of hard targets, we soften them.
Example (3 classes, smoothing = 0.1):
correct class → 0.9
others → 0.05
The model is no longer pushed toward extreme certainty.
🎯 Why It Works
One-hot targets force logits to grow very large to minimize cross-entropy.
This leads to:
📈 Overconfidence
⚠️ Poor calibration
🧠 Brittle generalization
Label smoothing acts as regularization in probability space.
It tells the model:
Be confident, but not blindly certain.
🏗 Where It’s Used
🤖 Image classification (ResNets, EfficientNet)
📝 Transformers and language models
🏆 Large-scale training pipelines
⚠️ Key Things to Know
🚫 Too much smoothing hurts accuracy
⚖️ Typical values: 0.05 to 0.1
🧪 Helps generalization more than training loss
📉 Often improves calibration
✅ In short:
Label smoothing prevents the model from collapsing into extreme certainty.
It trades a tiny bit of training confidence for better real-world behavior.