generalization, continued:
> We argue that Transformers will generalize to harder instances on algorithmic tasks iff the algorithm can be written in the RASP-L programming language (Weiss et al). By design, each line of RASP-L code can be compiled into weights of 1 Transformer layer.
https://arxiv.org/abs/2310.16028