TGViewer
Singular Thinker Singular Thinker @singularthinker · 1.29K subscribers
Post #1216 784
Singular Thinker Thermodynamics Reveals the Generalization in the Interpolation Regime 1/n 🔗 Link of the paper: https://arxiv.org/abs/2510.06028 In the realm of overparameterized NNs, one can achieve almost zero training error on any data, even random labels, that yield…
راستشو بخواید دوست داشتم خیلی بهتر معرفی کنم ولی الان خیلی دل و دماغ نوشتن ندارم ولی خواستم بگم که من فردا، دوشنبه، ساعت ۱۷ به وقت اروپای مرکزی و ۱۸:۳۰ ایران قراره که در ELLIS reading group مربوط به ریاضیات یادگیری عمیق Math&Efficiency of DL مقاله اخیرمو ارائه بدم. اگه دوست داشتید قطعا خوشحال میشم که ببینمتون.

Reading group: https://sites.google.com/view/efficientml

اینم عنوان و چکیده ی سخنرانی هست که قراره بکنم.

Temperature Paths Reveal Generalization in the Interpolation Regime

Abstract: Overparameterized neural networks can fit both meaningful data and random labels, often achieving essentially zero training error in both cases. This makes the final training error a poor predictor of generalization in the interpolation regime. In this talk, I will present a data-dependent view of this problem based on Gibbs posteriors and Langevin Monte Carlo algorithms using the PAC-Bayes framework. The key idea is to look not only at the final interpolating model but also at the entire temperature path: how quickly the expected training loss decreases as the inverse temperature increases. We show that, for the Gibbs posterior, the PAC-Bayes complexity term admits an exact integral representation as an area under this mean training-loss curve. This area separates true-label learning from random-label memorization and leads to quantitative bounds that track the observed test error in overparameterized neural networks.

ArXiv: https://arxiv.org/abs/2510.06028

Zoom: https://us02web.zoom.us/j/86351793196?pwd=inqIyzuhb6k1P1VqgnMKaVWEr5pbtW.1

@SingularThinker
Google DLMath&Efficiency A public ELLIS reading group exploring the interplay between the mathematical foundations of deep learning and the practical challenge of making ML efficient — from optimization theory to hardware-aware training. Learn more about our topics…
  • 🔥 6
  • ❤ 2
More from @singularthinker
  1. Oct 15, 2026من این پست رو که گذاشتم خیلی ابهام ایجاد کرد که خب مشکلش چیه اینا که گفتن ما از ai استفاده…
  2. Oct 15, 2026Proof abundance, verification scarcity
  3. Oct 15, 2026حالا بهونه این کار این بود که یکی از این خبرهای جالبی که من نرسیدم راجع بهش صحبت کنم و با…
  4. Oct 15, 2026داشتم فک میکردم خب این حس FOMO همگانی که همگی داریم چقدر داره آزاردهنده میشه برامون. خیلی…
  5. Oct 13, 2026Success as a scientist is minimally about raw problem-solving ability. Instead, it's more…
  6. Oct 12, 2026البته اشتباه نشه این افتخار نیست عزیزان من ریدم به این اصالت به قول دوست عزیزمون. من به شخ…
Threads Profile ViewerView any public Threads profile without an account.Open ThreadLook →Writing with AI? Make it sound human.Metric37 rewrites AI drafts so they read naturally. Free AI detector, 1,500 words free.Try Metric37 →