Definition
Retrieval-Augmented Generation (RAG) is an architecture that answers questions from knowledge missing from an LLM's weights — private, niche, or newer than its training data — by retrieving relevant documents at query time and injecting them into the prompt. The model changes no weights; the distinction from fine-tuning is that RAG supplies knowledge per-request while fine-tuning bakes in behavior.
A production RAG pipeline has two halves. Ingestion: documents are split into chunks, embedded, and indexed in a vector database. Serving: the query is embedded, similar chunks are retrieved (often hybrid — vector similarity plus keyword search, merged), optionally reranked by a cross-encoder, and the top passages go into the prompt with instructions to answer only from them. Grounding the answer in retrieved text is also the most direct hallucination mitigation, since the model has something concrete to cite — and something to admit is absent.
How it comes up in interviews
"Design a RAG system" is widely reported as the most common opening question in 2026 AI-engineer interviews. The differentiating follow-ups are retrieval quality (chunking strategy, hybrid search, reranking), permission enforcement inside retrieval (per-user ACLs), and evaluation of retrieval and answers separately.