Temperature: The Randomness Dial

Temperature

Part of: Temperature & Sampling

If sampling is drawing weighted tickets from a hat, temperature is the knob that decides how lopsided the hat is. Crank it down and the favorite gets almost every ticket, the model becomes focused and repeatable. Crank it up and the long shots get a real chance, the model gets wild and inventive. What it is Temperature is a single number (commonly 0 to 2) that reshapes the probability distribution before sampling. It doesn't change what the model thinks is likely; it changes how strongly it favors the likely options. - Low temperature (→ 0): sharpen the distribution. The top token's lead grows; everything else shrinks toward zero. At 0 it's effectively greedy, deterministic and focused. - High temperature ( 1): flatten the distribution. Gaps between tokens narrow, so unlikely tokens become genuinely reachable. Output gets creative, and at extremes, incoherent. How it works Under the hood, each token has a raw score (a logit ). Temperature divides every logit by T before they're turned into probabilities by softmax : - Dividing by a small T (like 0.5) makes the scores spread further apart - the leader pulls away - sharper . - Dividing by a large T (like 2.0) squashes the scores toge

Challenge: The Temperature Dial