A hyperparameter used in LLMs to control the randomness or creativity of the generated text. Lower temperatures produce more deterministic outputs, while higher
Temperature scales the randomness of token sampling: at 0 the model picks the most probable token nearly every time (deterministic-ish, focused); higher values flatten the probability distribution so less-likely tokens get chosen more often (varied, creative). Related controls, top-p, top-k, clip which candidates are eligible before sampling.
Temperature is the simplest reliability dial in AI engineering: extraction, classification, and code want low temperature; brainstorming and copywriting tolerate or benefit from higher values. Mis-set temperature masquerades as model failure: a routine first check in output-quality debugging.
Nearly but not perfectly: serving infrastructure (batching, hardware nondeterminism) can still produce minor variation. For strict reproducibility, pin model versions and use provider determinism features where offered.
Default low (0–0.3) for anything needing accuracy or consistent structure; raise only when variety is the goal. Tune empirically against your eval set rather than folklore.
They interact: top-p limits candidate tokens, temperature reshapes their probabilities. Common practice adjusts one while leaving the other near default; moving both aggressively makes behavior hard to reason about.