Overfitting is when a model memorizes its training data instead of learning generalizable patterns: scoring brilliantly on examples it saw and failing on new on
When a model memorizes its training data instead of learning generalizable patterns, scoring brilliantly on examples it saw and failing on new ones. It is detected by the gap between training and held-out validation performance.
The concept generalizes: benchmark contamination is overfitting at industry scale, and prompt-tuning against a tiny test set overfits evaluations. Held-out data discipline is the universal antidote.