Lesson 8 · 9 min
Overfitting and regularization
The model that fits the training data perfectly is rarely the best model. Six tools to keep it honest.
What overfitting looks like
Train loss keeps dropping. Validation loss bottoms out, then starts climbing again. The model has begun memorizing noise in the training set instead of learning generalizable patterns.
The goal is not zero training loss. The goal is lowest validation loss.