Hallucination

Hallucination is the term used when an AI model states something confidently that is factually wrong or entirely invented. The word is borrowed loosely from psychology, but in AI it describes a specific failure mode: the model is not deliberately lying or guessing, it is doing exactly what it was trained to do -- predict plausible text -- and that process sometimes produces output that sounds authoritative but has no basis in fact.

This is one of the most important limitations to understand about large language models and generative AI more broadly. The models are very good at writing fluently and confidently, which makes hallucinated content difficult to spot without independent checking. It is covered in practical terms in our ChatGPT beginners guide.

How it works#

A language model works by repeatedly predicting the most likely next token given everything that has come before in the conversation. It does not consult a database of verified facts when doing this. All its "knowledge" is baked into its parameters during training on large amounts of text. When a question touches on something the training data covered well, the model tends to produce accurate output. When the question is obscure, ambiguous, or outside what the training data covered, the model carries on generating plausible-sounding text anyway -- because that is the only thing it knows how to do.

This is compounded by the fact that AI models are typically trained to be helpful and confident. Saying "I don't know" is a behaviour that has to be specifically encouraged, and even models trained to do this will sometimes slip into confident-sounding fabrication rather than expressing uncertainty.

Examples#

Invented academic references. A researcher asks an AI to suggest papers on a niche topic in environmental science. The model produces a list of titles, author names, and journal citations that look completely plausible. On checking, several of the papers do not exist: the titles and authors are real-sounding but fabricated. The journals are real, which makes the citations even more convincing at a glance.

Fictitious legal cases. A person researching their rights asks an AI to cite relevant court cases. The model produces case names, dates, and brief summaries. Some are real; others are invented cases that match the naming conventions of genuine rulings so closely that they pass a quick visual check. Relying on these in any formal context would be a serious error.

Why it matters#

Hallucinations are not a minor inconvenience: they are a fundamental property of how current AI models work, and they have real consequences when people treat AI output as a reliable source of truth. Journalists, students, medical professionals, and anyone making decisions based on AI-generated information needs to approach that information with appropriate scepticism.

Practically, the best defence is a verification habit. Treat AI output the way you would treat a tip from a knowledgeable but fallible colleague: useful as a starting point, but worth checking before you act on it. Giving the model the source material in your prompt -- rather than asking it to recall facts -- also reduces hallucination significantly, because the model is now working from text in front of it rather than from memory.

For guidance on getting more reliable outputs from AI tools, see our Claude beginners guide or explore the prompt engineering glossary entry.