"A Mathematical Explanation of Transformers" is a recent paper in which the authors construct a rigorous mathematical model of the Transformer architecture and large language models.

The Transformer is presented as a discretization of a continuous integro-differential equation. Self-attention is described as a non-local integral operator, layer normalization as a projection onto a constrained set, and fully connected layers and activation functions are incorporated into the same mathematical framework.

The authors then use operator splitting and numerical discretization to derive the standard Transformer architecture and extend this approach to multi-head attention, Vision Transformers, and convolutional Transformers.

I have previously shared several materials on the mathematics of neural networks, Transformers, and large language models, but new and interesting developments are constantly emerging in this field.

https://arxiv.org/pdf/2510.03989