Probability Distributions: The Next-Token Lottery

Probability

Part of: How AI Actually Works

When the model "picks the next token," it does not pick. It rolls a weighted die. Before any word comes out, the model holds a full probability distribution over every token in its vocabulary at once, maybe 50,000 numbers, each saying "this is how likely I am to come next." The word you see is one draw from that lottery. What it is A probability distribution is a list of likelihoods that covers every option and sums to exactly 1.0 (100%). For next-token prediction, there is one entry per vocabulary token. " blue" might get 0.62, " green" 0.10, " falling" 0.03, and tens of thousands of others split the rest. The model never outputs just one guess, it always produces the whole spread, and sampling reaches in to pull one out. But the model does not compute these probabilities directly. It first produces raw, unbounded scores called logits : one per token, any real number, positive or negative. A function named softmax then squashes those logits into a clean probability distribution. How it works Softmax does two jobs: it makes every number positive (by exponentiating), then it normalizes them to sum to 1. Bigger logit in, bigger probability out, but the gaps get exaggerated, so the to

Challenge: Softmax Sampler Odds