How Chatbots Think

Forward: the network reads text and outputs probabilities for the next token (blue wave).
Compare: the loss measures how much probability it gave to the token that actually came next.
Backward: backpropagation computes, for every weight, which way to turn it to reduce the loss (orange wave).
Update: nudge every weight a tiny step that way. Watch “get” become more likely in the scene.

How large language models such as ChatGPT, Claude and Gemini actually work: tokens, embeddings, next-word prediction, training by gradient descent, attention and the Transformer, scale, human feedback, hallucinations, tools and reasoning, with live demos you can play with.

Read the full story as text · อ่านภาษาไทย

This interactive 3D film needs JavaScript. PlanetPast home