Megatron-LM
Megatron (1, 2, and 3) is a large, powerful transformer developed by the Applied Deep Learning Research team at NVIDIA. This repository is for ongoing research related to training large transformer language models at scale. We developed efficient, model-parallel (tensor, sequence, and pipeline), and multi-node pre-training of transformer based models such as GPT, BERT, and T5 using mixed precision.

Creator: makeplane
Stars ⭐️: 7.1k
Forked By: 1.6k
GithubRepo:
https://github.com/NVIDIA/Megatron-LM
➖➖➖➖➖➖➖➖➖➖➖➖➖➖
Join @github_repositories_bds for more cool repositories.
*This channel belongs to @bigdataspecialist group