What is AI hallucination?

How it works

An LLM generates the most plausible next token, not a checked fact. When it lacks reliable knowledge, it still produces fluent text, because fluent text is what it was trained to produce. Researchers usually split hallucinations into two kinds: intrinsic, which contradict the source the model was given, and extrinsic, which add claims the source cannot support.

The 2025 paper "Why Language Models Hallucinate" argues that the problem is reinforced by how models are trained and graded: benchmarks that give credit for right answers and none for "I don't know" reward guessing, much like a multiple-choice exam with no penalty for wrong answers. Rare facts, such as a person's birthday that appears once in the training data, are especially likely to be guessed.

Why it matters

• Trust and liability. Invented case law, fake references and wrong medical or financial figures have real consequences when people act on them.
• Code risk. Coding tools can call packages or APIs that do not exist, and attackers can register those invented package names.
• It cannot be fully removed today, but it can be reduced:
  • ground answers in retrieved documents with RAG and require citations;
  • let the model say it does not know, and tell it to;
  • use tools (search, calculators, code execution) for facts and arithmetic;
  • lower temperature for factual tasks;
  • check outputs with guardrails, tests or human review.

Example

Asked for "three peer-reviewed studies on four-day work weeks with DOIs", a model without search may return three real-sounding titles, journals and DOIs, one of which does not exist. The same model with a search tool and an instruction to cite only what it retrieved is far less likely to invent one, and any it lists can be clicked and checked.