Schmidhuber’s Fast-Weight work of 1991-92 introduced an explicit mechanism for token-to-token interaction inside recurrent neural networks (RNNs).

Each time step wrote an outer-product fast weight that linked the current hidden state (a “key”) with a “value.”

📚Статья

@datascienceiot