Thermodynamics Reveals the Generalization in the Interpolation Regime
1/n

🔗 Link of the paper: https://arxiv.org/abs/2510.06028

In the realm of overparameterized NNs, one can achieve almost zero training error on any data, even random labels, that yield massive test errors.
So, how can we tell when such a model truly generalizes? 🤔

In the figure above from the famous paper, the same model achieves nearly zero training error on both random and true labels. Therefore, the key to generalization must lie within the structure of the data itself.
https://arxiv.org/abs/1611.03530

#neural_networks
#learning_theory