Approximating How Single Head Attention Learns
Snell et al., [Berkeley]
arxiv.org/abs/2103.07601

A look inside LSTM seq2seq with attention training dynamics. The main idea of the paper is KTIW – Knowledge to Translate Individual Words. To explain the dynamics, the authors divide training into two stages: uniform attention (KTIW) and non-uniform attention.

In the first stage of model training, attention does not change significantly from the uniform, and the model mainly learns to translate individual words (KTIW, a.k.a. dictionary translation). After KTIW is learned, attention starts forming its patterns, and this process is driven by the KTIW. As correct word translations are already more probable, now attention mainly needs to align the words from the source and target language.

To quantitatively test the hypothesis, they develop a new lexical prob that is essentially hard attention. Yet, the most impressive result is that attention cannot learn a simple copy operation if KTIW is not learned.