Build a Large Language Model from Scratch! 🚀

This repository provides code examples for developing, pretraining, and fine-tuning a Large Language Model (LLM) from the ground up. It serves as the official codebase for the book "Build a Large Language Model (From Scratch)." 📘

Notebook examples are included for each chapter:

Chapter 1: Understanding Large Language Models 🧠
Chapter 2: Working with Text Data 📝
Chapter 3: Coding Attention Mechanisms ⚙️
Chapter 4: Implementing a GPT Model from Scratch 🏗
Chapter 5: Pretraining on Unlabeled Data 📊
Chapter 6: Fine-tuning for Text Classification 🏷
Chapter 7: Fine-tuning to Follow Instructions 🗣

Repository: https://github.com/rasbt/LLMs-from-scratch 🔗