How does an AI chatbot pick its next word?

A chatbot writes one small piece at a time. It reads the context, scores possible next tokens, turns those scores into probabilities, chooses one, adds it and repeats.

▶ Try it yourself: move temperature and the deterministic sample point.

What does temperature do?

For illustrative logit z and temperature T, softmax gives probability exp(z/T) divided by the sum across candidates. Lower temperature concentrates probability on the top score. Higher temperature spreads probability among lower-ranked choices.

The sample point selects the probability interval it lands in. The tool keeps that point deterministic so the choice is inspectable and repeatable.

What is illustrative?

The prompt, five token candidates and every score are invented for this explainer. They are not from a named chatbot and are not a benchmark. Real vocabularies are much larger and systems may also use top-k, top-p and repetition penalties.

Temperature does not add facts or make an answer true. It changes only how sharply the system favors the scores it already produced.

Sources: Google for Developers, Softmax; Hinton, Vinyals and Dean, softmax temperature; Holtzman et al., text degeneration.