[Google Research]
Constructing Transformers For Longer Sequences with Sparse Attention Methods
Posted by Avinava Dubey, Research Scientist, Google Research Natural language processing NLP models based on Transformers, such as BERT, RoBERTa, T5, or GPT3, are successful for a wide variety of tasks and a mainstay of modern NLP research. The versatility and robustness of Transformers are the