Definition
Chunking is the preprocessing step in a retrieval-augmented generation pipeline that divides source documents into the units that get embedded, indexed, and retrieved. Chunks that are too large dilute the embedding with unrelated content and waste context; chunks that are too small lose the surrounding context needed to make the passage meaningful on its own. Overlapping windows are the standard mitigation for facts that straddle a boundary.
Better systems chunk along the document's own structure — headings, sections, function definitions — rather than at a fixed character count, so each chunk is a coherent idea. Common refinements include attaching the document title and section path to each chunk so retrieved text carries its context, and storing a small chunk for matching while returning a larger surrounding window to the model. Chunking is usually the highest-leverage knob in a disappointing RAG system, ahead of swapping the embedding model.
How it comes up in interviews
"The retrieved context is irrelevant — what do you check first?" is a standard RAG debugging question, and chunking should be near the front of your answer alongside the embedding model and the query itself. Being specific about sizes and overlap, and about structure-aware splitting for code or tables, reads as hands-on rather than theoretical.