راستشو بخواید دوست داشتم خیلی بهتر معرفی کنم ولی الان خیلی دل و دماغ نوشتن ندارم ولی خواستم بگم که من فردا، دوشنبه، ساعت ۱۷ به وقت اروپای مرکزی و ۱۸:۳۰ ایران قراره که در ELLIS reading group مربوط به ریاضیات یادگیری عمیق Math&Efficiency of DL مقاله اخیرمو ارائه بدم. اگه دوست داشتید قطعا خوشحال میشم که ببینمتون.
Reading group: https://sites.google.com/view/efficientml
اینم عنوان و چکیده ی سخنرانی هست که قراره بکنم.
Temperature Paths Reveal Generalization in the Interpolation Regime
Abstract: Overparameterized neural networks can fit both meaningful data and random labels, often achieving essentially zero training error in both cases. This makes the final training error a poor predictor of generalization in the interpolation regime. In this talk, I will present a data-dependent view of this problem based on Gibbs posteriors and Langevin Monte Carlo algorithms using the PAC-Bayes framework. The key idea is to look not only at the final interpolating model but also at the entire temperature path: how quickly the expected training loss decreases as the inverse temperature increases. We show that, for the Gibbs posterior, the PAC-Bayes complexity term admits an exact integral representation as an area under this mean training-loss curve. This area separates true-label learning from random-label memorization and leads to quantitative bounds that track the observed test error in overparameterized neural networks.
ArXiv: https://arxiv.org/abs/2510.06028
Zoom: https://us02web.zoom.us/j/86351793196?pwd=inqIyzuhb6k1P1VqgnMKaVWEr5pbtW.1
@SingularThinker
Post #1216
784
Singular Thinker Thermodynamics Reveals the Generalization in the Interpolation Regime 1/n 🔗 Link of the paper: https://arxiv.org/abs/2510.06028 In the realm of overparameterized NNs, one can achieve almost zero training error on any data, even random labels, that yield…Google DLMath&Efficiency A public ELLIS reading group exploring the interplay between the mathematical foundations of deep learning and the practical challenge of making ML efficient — from optimization theory to hardware-aware training. Learn more about our topics…
- 🔥 6
- ❤ 2