You could have invented transformers from memory

🔸The author discusses how large language models can recall obscure facts from every corner of the world, and how this ability is similar to a memory store that holds key-value pairs. The author then develops an intuitive way to look at linear maps, MLPs, and self-attention as different kinds of memory stores.

2026-10-16 19:41

💬 comments
🌍 @hackernews_summary