Definition
Top-p (nucleus) sampling is an LLM decoding strategy that sorts candidate tokens by probability and keeps the smallest prefix whose probabilities sum past a threshold p (commonly 0.9), then renormalizes and samples from that nucleus. Unlike top-k, which keeps a fixed count of candidates, top-p's set adapts to the shape of the distribution: when the model is confident the nucleus is tiny, and when it is uncertain the nucleus widens.
That adaptivity is why top-p is the more common production default — a fixed k is too wide for confident distributions (admitting junk tokens) and too narrow for flat ones (cutting off legitimate diversity). Top-p composes with temperature, which reshapes the distribution before the nucleus is selected.
How it comes up in interviews
A staple of AI coding rounds: implement top-p from raw logits, which tests softmax, sorting, cumulative sums, and the off-by-one at the threshold boundary. Conceptual follow-up: explain when top-p and top-k behave differently and why top-p won.