Foundations of Large Language Models - 277 Pages

A foundational guide to how LLMs work, focusing on core principles; even though it doesn’t dive directly into cybersecurity, it covers architectures, training strategies, prompting, alignment, and inference methods

Chapter 1️⃣ - introduces the basics of pre-training. This is the foundation of large language models, and common pre-training methods and model architectures will be discussed here.

Chapter 2️⃣ - introduces generative models, which are the large language models we commonly refer to today. After presenting the basic process of building these models, we will also explore how to scale up model training and handle long texts.

Chapter 3️⃣ - introduces prompting methods for large language models. We will discuss various prompting strategies, along with more advanced methods such as chain-of-thought reasoning and automatic prompt design.

Chapter 4️⃣ - introduces alignment methods for large language models. We will focus on instruction fine-tuning and alignment based on human feedback.

Chapter 5️⃣ - introduces inference methods for large language models. We will discuss various decoding algorithms, acceleration methods, and the inference-time scaling issue.

Source: https://arxiv.org/pdf/2501.09223 By Tong Xiao and Jingbo Zhu - Thanks, Alex Axworthy for sharing.

#JingboZhu #TongXiao #LLMBasics #NLPModels #AIResearch #ModelTraining #DeepLearning #SelfSupervised #PromptDesign #ModelScaling #TransformerAI #TokenPrediction #TextGeneration #LanguageModel #EncoderDecoder #BERTModel #GPTModel #T5Model #BARTModel #FineTuning #ZeroShot #FewShot #RLHFTraining #AIAlignment #AIInference #MaskedLearning #CausalModeling #SequenceModel #LanguageTasks #AIOptimization #ModelLoss #CrossEntropy #TextPrompt