Temperature: Tuning Randomness

Temperature

Part of: Prompt Engineering

Ask a model to "name a fruit" ten times at temperature 0 and you'll likely get "apple" ten times. Crank the temperature up and you'll start seeing "dragonfruit," "kumquat," "persimmon." Temperature is the dial that controls how adventurous the model gets when picking its next token, and misunderstanding it is one of the most common prompt-engineering mistakes. What it is Temperature is a single number (typically 0 to 1 , some APIs allow higher) that controls randomness in token selection . It is NOT a "creativity score" you turn up for better writing. It's a risk knob: low means play it safe, high means take chances. More randomness is not the same as more quality. How it works At each step the model produces a ranked list of likely next tokens with probabilities attached. Temperature reshapes that list before a token is sampled: - Low temperature (near 0) : sharpen toward the top choice. The model almost always grabs the most likely token. Output is focused and repeatable. - High temperature (toward 1) : flatten the odds. Lower-ranked tokens get a real shot. Output is varied and sometimes off the rails. Picture the probabilities as a hill. Low temperature makes the hill into a sha

Challenge: Greedy Pick and the Sampling Pool