Very nice article by M. Aminian in SIAM News entitled "The Connection Between Applied Mathematics and Deep Learning".
The article dissects and explains in very clear terms the gist of my plenary talk at the recent SIAM Mathematics of Data Science conference entitled "The Deep Learning - Applied Mathematics Connection".
It discusses the specifics of optimization as it pertains to DL (stochastic optimization in high-dimensional spaces), the surprising fact that widely overparamterized DL models work well, the use of DL in traditional applications of scientific computing: PDE solving, lattice quantum computation, graph convolutional nets and the connection with Fourier transform, and the rising topic of self-supervised learning.
The methods of Applied Mathematics could help explain some of the mysteries of deep learning
SIAM News paper:
https://sinews.siam.org/Details-Page/the-connection-between-applied-mathematics-and-deep-learning Video of my talk:
https://youtu.be/9sqoe_krQ1E