How Large Language Models Work: A Visual Guide

TL;DR

LLM stands for large language model. It is a type of artificial intelligence (AI) that learns patterns in text and uses them to produce answers, summaries, and explanations.

• The model learns during training. An ordinary chat provides context without automatically retraining it.
• To answer, it uses your message and the available information, then builds a response one small piece of text at a time.
• A convincing answer can still be wrong. Verify important details with reliable sources.

We will build one internal AI support desk across this series. In our fictional example, an employee named Emily asks: “Can I claim a $45 dinner on a work trip?” This first episode explains the part that reads the question and writes a response: the language model.

It can produce a helpful draft, but general training cannot establish the company’s current private travel policy. That missing information will give us a reason to add the next layer.

1. Training: learning patterns from examples

Before our support desk receives Emily’s question, its language model has already been trained. During training, it processes many examples of text, predicts the next token, compares that prediction with the actual continuation, and adjusts internal numbers called weights. Repeating this process develops patterns involving language, information, and the relationships between ideas. The GPT-2 paper describes this prediction task.

The sky sentence is a close-up of that training task. Across many examples, the model learns how a question is phrased, how an explanation develops, and which terms tend to appear together. That helps it interpret Emily’s request. Its weights capture learned patterns rather than acting as a reliable catalog of every source. Training can also preserve errors and memorized passages.

Additional training can improve instruction following. For example, people can provide preferred responses or compare alternative answers. This helps shape the model’s behavior, but it does not make every answer correct. See the research on learning from human feedback.

2. Tokens: turning text into usable input

Emily’s question enters the application as text. A tokenizer separates that text into tokens: whole words, parts of words, punctuation, or other text units. The model processes these units rather than treating the question as one indivisible block. The exact boundaries depend on the tokenizer; the short word below makes this easier to see.

For an illustrative example, Unbelievable! could be split into Un + believ + able + !. A real tokenizer might split it differently. Token boundaries are not necessarily syllables. SentencePiece describes one tokenization approach.

Each token receives a numerical ID and a learned numerical representation, called an embedding. Order matters too: “review, then approve” describes a different sequence from “approve, then review.” The network needs both the text units and their relationships.

3. Attention: connecting relevant information

Now supply a few short excerpts: “Breakfast limit: $20. Dinner limit: $50. Taxi receipt: required.” Ask, “What is the dinner limit?” The relevant detail is “Dinner limit: $50,” rather than the breakfast amount or taxi condition. These simplified excerpts let us examine how the model uses information already provided; they are not the full travel policy we will retrieve in the next episode.

Attention combines information from different positions in the sequence, giving those positions different amounts of influence. It helps the network relate a question to useful context. This is a numerical operation; it does not guarantee that the model will identify every important detail.

Many text LLMs use a transformer architecture. Its layers combine attention with other calculations to process the sequence. When generating the next token, this kind of language model uses the available earlier text. The original transformer paper introduced the architecture.

4. Generation: choosing the next token

After processing the context, the model generates its answer one token at a time. The same mechanism used to draft Emily’s reply is easier to inspect with a shorter input: “The sky is…” The model estimates probabilities for possible next tokens. “Blue,” “clear,” and “dark” could each fit a different context. A selection rule chooses a continuation from those probabilities.

The following values are invented to illustrate the process. They describe possible text continuations, not the probability that a claim is true.

After selecting “ blue”, the model processes the extended sequence, “The sky is blue”, and estimates what follows. It continues until a stopping condition ends the response. These tokens happen to look like words; others may be smaller text fragments.

Repeating this operation can produce a paragraph or a long explanation. The full answer does not need to exist somewhere ready to copy. Changing the input to “At night, the sky is…” also changes which continuations fit. GPT-3 explored the capabilities of large models using this approach.

5. Context and tools: supplying useful information

Our support desk already has Emily’s question. If the application also supplies a relevant travel-policy excerpt, that text becomes part of the context: the instructions, messages, and documents available for the answer. The model can use that policy without permanently learning it through training.

Context has a size limit, and the model can still overlook details. Saved product memories are a separate feature. Neither a long conversation nor a memory feature guarantees that every relevant fact will be available in each response.

Access to outside information depends on the application. A connected search tool can retrieve an authorized policy; a calculator can supply a calculation. These capabilities require actual connections. Writing “read the company’s files” in a prompt does not grant document access or permission to approve a reimbursement.

6. Verification: separating fluent text from facts

If Emily asks about a company rule that has not been supplied, the model lacks the evidence needed to establish it. It may acknowledge that gap or generate plausible details. A polished reply saying “your dinner is approved” would still be unsupported: writing an answer and authorizing a claim are different operations.

A false or unsupported generated answer is commonly called a hallucination. Predicting likely text is different from verifying a claim, and training data can contain mistakes. TruthfulQA studied models repeating common falsehoods.

Provide relevant documents, state the task clearly, and verify details that matter. Open cited sources, check calculations, and compare quotations with the original text. Even an answer grounded in a document can misinterpret it.

Frequently asked questions

Sources & further reading

The primary research behind the article, for readers who want to explore the details.

1. Language Models are Few-Shot Learners Brown et al. · 2020
2. Language Models are Unsupervised Multitask Learners Radford et al. · 2019
3. SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing Kudo and Richardson · 2018
4. Attention Is All You Need Vaswani et al. · 2017
5. Training language models to follow instructions with human feedback Ouyang et al. · 2022
6. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks Lewis et al. · 2020
7. TruthfulQA: Measuring How Models Mimic Human Falsehoods Lin, Hilton, and Evans · 2021