AI glossary

What is Large language model?

Also called LLM, LLMs ·

A large language model (LLM) is a neural network, usually a transformer, trained on huge amounts of text to predict the next token, which lets it answer questions, write, summarise, translate and generate code from plain-language instructions.

How it works

An LLM turns text into tokens, small chunks of words, and processes them with a transformer network that has billions of learned parameters. During pre-training it sees a vast corpus and learns one job: given the tokens so far, predict the next one. Doing that well across code, books and web pages forces the model to absorb grammar, facts, styles and many patterns of reasoning.

A raw pre-trained model only continues text. To become a useful assistant it is usually tuned further with instruction data and human feedback (RLHF or similar methods), so it follows requests, refuses harmful ones and answers in a helpful format. At run time (inference) it generates one token at a time, each choice conditioned on everything in its context window.

Why it matters

  • One model, many tasks. The GPT-3 paper showed that a big enough language model can do new tasks from a few examples in the prompt, without retraining. That is why one LLM can draft an email, explain a contract and write a SQL query.
  • It is the engine of modern AI products. Chat assistants, coding tools and AI agents are built on LLMs plus tools and data.
  • It has real limits. An LLM predicts plausible text, not verified truth, so it can hallucinate. Its knowledge stops at its training cut-off unless you supply fresh information, for example with retrieval.

Example

Ask an LLM "Summarise this 20-page supplier contract and list every termination clause." It reads the contract as tokens in its context window, then writes a summary token by token. For anything you will act on, check the clauses it cites against the original document: the summary is usually good, but not guaranteed.

Questions people ask

What is an LLM in simple words?+

An LLM is a very large text predictor. It has read so much text during training that predicting the next word well requires it to pick up grammar, facts and patterns of reasoning, which it then uses to respond to your prompt.

What is the difference between LLM and AI?+

AI is the whole field of making machines do tasks that need intelligence. An LLM is one type of AI model, specialised in language, and it sits inside the wider areas of machine learning and generative AI.

Is ChatGPT an LLM or generative AI?+

Both. ChatGPT is a generative AI product, and the models that power it are large language models. The app adds a chat interface, tools and safety layers around them.

Related terms

Sources

  1. Language Models are Few-Shot Learners, arXiv (Brown et al., 2020)
  2. Attention Is All You Need, arXiv (Vaswani et al., 2017)

Back to the AI glossary

Go deeper