What Are Large Language Models?

An answer can get the time right, get the room right and invent the parking arrangements—all in two perfectly ordinary sentences.

To understand how that can happen, we need to separate two questions: How was the text produced? And what supports what it says?

A large language model, or LLM, is a neural-network model trained on large amounts of language data to learn patterns that help it process or generate language. “Large” refers to scale, including the model and its training, rather than a universally agreed minimum size.

This article focuses on the autoregressive language models commonly used to generate text: models that predict the next token from the context available so far. Language models can use other training arrangements too. [1]

We will follow one fictional workshop notice through an answer. No model was tested to produce this example; every line is constructed to make the mechanism and the evidence gap easy to inspect.

First, give the model something to work with

Here is the entire notice in our example:

Harbour workshop
Starts at 18:30.
Room Cedar.
Bring a laptop.

And here is the question:

When and where is the workshop, and is parking free?

The notice and question can be supplied as context: information available to the model for this response. In an application, context can also include instructions, selected conversation history or material retrieved from elsewhere.

That is different from the model’s parameters, the numerical values adjusted during training. Providing a new notice normally changes the input for this answer; it does not retrain the model’s parameters. Models can adapt their response to examples or instructions in context without a parameter update. [1]

The distinction matters. The model may have learned many patterns about event notices during training. But the specific room for this fictional workshop is supplied here, in the notice.

Tokens are the pieces the model works with

Before processing, text is converted into tokens, which have numerical identifiers. Depending on the tokenizer, a token might represent a word, part of a word, punctuation or another text fragment. Token boundaries are model specific; one word is not necessarily one token. [2]

Here is an illustrative split of the beginning of an answer:

“It” · “ starts” · “ at” · “ 18” · “:” · “30” · “ in” · “ Room” · “ Cedar” · “.”

The spaces inside several pieces are intentional. This is a made-up tokenisation for teaching, not the recorded output of a particular tokenizer. Joined together, the pieces say: “It starts at 18:30 in Room Cedar.”

The identifiers are converted into numerical representations the network can process. In a transformer-based model, attention helps combine information from different positions in the available context. That is part of how the model can use a room name supplied earlier when composing a later answer; attention is not a built-in certificate that the answer is true. [3]

How an answer grows

Byte Wonders

Byte asks: “Does it choose the whole answer before the first word?”

In ordinary autoregressive generation, the model repeatedly produces a distribution over possible next tokens. A decoding procedure selects a token, that token joins the context for subsequent generation, and the process continues until a stopping condition is reached. [4]

It is not necessary for a complete answer to exist somewhere, waiting to be uncovered. The growing answer becomes part of what conditions the next step.

Selection is not always “take the single most likely token”. Greedy decoding does that, while sampling selects according to a probability distribution, often with restrictions on the candidate set. Different decoding choices can yield different continuations. [5]

But a token’s probability concerns its fit as a continuation under the model. It is not the probability that the complete factual claim is correct. Fluent continuation and factual verification are different tests.

How does next-token training become something useful?

During autoregressive pretraining, text supplies its own training targets: at a position in a sequence, the actual next token provides the target for prediction. This is a form of self-supervised learning. Repeated updates adjust the model’s parameters. People do not have to write a separate answer label for every such position. [10]

Predicting language across many contexts can support more than finishing a sentence. Models can acquire patterns useful for tasks such as translation, answering questions and working with examples supplied in a prompt. The objective is simple to state; the patterns that help with it can be complicated. [1]

Pretraining is also not the whole story behind an assistant. Post-training can further shape how a model follows instructions or responds to people. One documented approach used demonstrations followed by feedback comparing responses. Such training improved instruction following in the study, but did not eliminate factual mistakes. [6]

This helps explain why “it predicts tokens” is a description of a mechanism, not a complete account of its capabilities—and why sounding like a helpful assistant is not a guarantee of correctness.

Now separate the answer from its evidence

Suppose our illustrative response is:

It starts at 18:30 in Room Cedar. Parking is free.

The sentence structure is unremarkable. The evidence is not.

Byte’s Reality Check

Byte asks: “It got the room right. Does that make the parking bit trustworthy?”

Read each claim against the notice, rather than judging the answer as one confident-sounding whole.

The last claim is unsupported here, not proven false. Perhaps parking really is free; perhaps it costs money; perhaps there is no parking. We cannot settle that from the supplied text.

A better evidence-bounded answer would be:

It starts at 18:30 in Room Cedar. The notice does not say whether parking is free.

That response gives the useful information without converting a gap into an assertion.

LLMs can generate erroneous or false material, often called hallucinations. NIST describes this risk as confabulation. Our invented parking sentence is not evidence that a particular model hallucinated: it is a way to practise spotting a claim whose support is missing. [7]

That distinction is useful in real work. A response can contain both well-supported and unsupported statements. Two correct details do not lend evidence to the third.

Context helps, but it is not a guarantee

If a current parking policy were retrieved and supplied with the notice, the model would have more relevant material available. Retrieval-augmented generation, or RAG, combines retrieval with generation in this broad way. It gives a system access to external information instead of relying only on what is encoded in model parameters. [8]

Still, an available source must be relevant, current and used correctly. A citation next to “parking is free” is useful only if the cited material actually establishes that claim. The same evidence audit still applies.

There are limits to what can be supplied in one request. A model’s context window is its supported token budget for the sequence it can work with; exact limits and accounting depend on the model and system. An application may select, summarise or omit parts of a conversation before passing it to the model.

And fitting information into the available context does not establish that it will be used reliably. In a 2023 study of the models and tasks examined, researchers found that placing relevant information in the middle of long contexts could reduce performance. That is a scoped research finding, not a claim that every model today fails in the same way. [9]

A language model is not the whole application

An LLM supplies a language-processing or generation component. A product built around it may add document retrieval, browsing, calculators, other tools and application-level memory.

Those additions change what the system can do and what information reaches it. They do not mean every response automatically used every available tool. A calculation might come from generated text or from an actual calculator invocation; the distinction matters when checking it.

The useful question is therefore more specific than “Does this AI know the answer?” Ask: What information did this response have access to, what did the system actually do, and which source supports the claim I care about?

For the workshop, that means checking the notice for time and room, then finding the venue’s parking information if it matters. No amount of smooth wording fills in a missing policy.

What to take away

Tokens explain the pieces. Training explains how parameters acquire useful language patterns. Context explains what information is available for this response. Generation explains how the output grows.

None of those, by itself, establishes the truth of every sentence.

Our workshop answer leaves you with a practical habit: trace important claims individually. Supported, unsupported and false are different judgements. Knowing which one you can justify is more useful than treating an entire answer as either trustworthy or useless.

Continue learning

Previous: What Are Neural Networks? — parameters, activations and the difference between a forward pass and a learning update.

For the wider picture: What Is Deep Learning? — learned representations and why the task defines success.

Next: How Does ChatGPT Actually Work? — move from the language-model mechanism to an assistant product.

Explore the AI Fundamentals learning path.

Sources

1. Brown and colleagues (2020), Language Models are Few-Shot Learners. Autoregressive modelling, task performance and adaptation through context without gradient updates. The article describes the mechanism, not current product rankings.
2. Hugging Face LLM Course, Tokenizers. Tokens, subwords and numerical identifiers. Our displayed split is explicitly illustrative.
3. Google Machine Learning Crash Course, Transformers. Embeddings, attention and transformer language-model processing.
4. Hugging Face Transformers, Generation with LLMs. Autoregressive generation from a prompt and preceding output.
5. Hugging Face Transformers, Generation strategies. Greedy selection and sampling; these concern continuation selection, not factual verification.
6. Ouyang and colleagues (2022), Training language models to follow instructions with human feedback. A documented post-training approach and remaining limitations.
7. NIST (2024), Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile, section 2.2. Confabulation and associated reliability risks.
8. Lewis and colleagues (2020), Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. Primary research combining retrieved information with generation.
9. Liu and colleagues (2023), Lost in the Middle: How Language Models Use Long Contexts. Position effects in the studied models and tasks; not a universal current benchmark.
10. Hugging Face Transformers, Causal language modeling. Next-token targets derived from the text itself.