Pre-Chunking vs. Post-Chunking (On-Demand Chunking)

This visual breaks down two common ways to chunk documents in Retrieval-Augmented Generation (RAG) systems,and when each makes sense.

Pre-Chunking
Documents are cleaned, split into chunks, embedded, and stored ahead of time.
•  Pros: Fast retrieval at query time, simpler runtime pipeline.
•  Cons: Rigid,changing chunk size or strategy means reprocessing the entire dataset.
•  Best for: Stable datasets, high-throughput apps, predictable queries.

Post-Chunking / On-Demand Chunking
Documents are stored whole; chunking happens after retrieval based on the user’s query.
•  Pros: More flexible and query-aware, often more relevant context.
•  Cons: Higher latency and infrastructure complexity.
•  Best for: Evolving content, exploratory queries, precision-focused use cases.

🔑 Takeaway:
There’s no one-size-fits-all. If speed and scale matter most, pre-chunk. If adaptability and relevance are key, post-chunk. Many production systems even combine both.