Definition
Temperature is a decoding parameter that scales an LLM's logits before they are converted to probabilities: values above 1 flatten the distribution (more randomness and diversity), values below 1 sharpen it toward the likeliest tokens, and temperature 0 collapses sampling to greedy decoding — always the single highest-probability token.
Greedy decoding is mostly deterministic but not exactly: batching effects and floating-point accumulation order can still cause tiny run-to-run variations in practice, a nuance worth knowing when someone claims temperature 0 guarantees reproducibility. Typical settings: near 0 for extraction and structured output, where creativity is a bug; 0.7–1.0 for open-ended generation.
How it comes up in interviews
Usually probed as applied judgment: "your JSON-extraction endpoint occasionally invents fields — what do you check first?" (temperature, then schema-constrained decoding). The temperature-0-is-not-perfectly-deterministic nuance is a common senior-level follow-up.