An AI hallucination is an output from a language model that reads fluently, factually and convincingly but simply is not true. Invented citations, made-up legal clauses, a programming function that never existed. The awkward part is not the mistake itself but that it looks exactly like a correct answer: no hesitation, no caveat, no signal of uncertainty.
The term is disputed because it makes the model sound human. “Confabulation” would fit better. Hallucination won anyway.
Why models produce this
A large language model predicts the most likely next piece of text. It holds no database of facts and has no switch that separates knowing from guessing. True and plausible are the same thing to the mechanism.
The usual triggers follow from that:
- Gaps in the training data. A niche topic has little material behind it, but the linguistic patterns are enough to assemble an answer.
- The cut-off. Anything after training ended is missing. The model does not know that it does not know.
- Answering gets rewarded. Models are trained to be helpful. In evaluation an answer almost always scores better than a shrug, and that pattern sticks.
- Vague questions. Where a question carries a false premise, the model usually adopts it instead of pushing back.
- Summarising across sources. Condensing several passages produces statements that appeared in none of them.
The forms it takes
| Form | What it looks like | Where it hurts most |
|---|---|---|
| Invented sources | title, author and year look real, the work does not exist | research, legal filings |
| Wrong details | right topic, wrong figure, wrong date, wrong name | quotes, reports, product data |
| Invented code | functions and libraries that do not exist | software development |
| Wrong summary | statements that are not in the document supplied | document analysis, support |
| Confirmed false premise | the wrong assumption in the question gets elaborated | consulting, diagnosis |
The special case in code
Models regularly invent package names that do not exist. The annoyance is not the mistake, which surfaces at install time, but the reaction to it: attackers register frequently hallucinated package names and put malicious code there. The term slopsquatting took hold for this pattern in 2025. Anyone doing vibe coding and accepting dependencies unchecked may end up installing it themselves.
Why summarising exposes it
With a knowledge question you often do not notice the error. With a summary you do, because the source text sits right there and can be compared. That makes summarisation the best test ground: a model that adds things there is doing it elsewhere too, only invisibly.
What actually helps
Hallucinations cannot be switched off, they follow from how the technology works. What you can do is make them less likely and, above all, visible.
- Tie answers to sources. The most effective lever by far. Instead of answering from memory the model works with passages supplied to it, see retrieval-augmented generation.
- Demand evidence. Every statement with a reference. A model that cannot cite has not found anything either.
- Permit not knowing. A prompt that explicitly allows an empty answer noticeably lowers the invention rate. Without that way out, the model always answers.
- Checkable output. Structured formats can be compared against the source data, prose cannot. More on that under prompt engineering.
- A second pass. A separate checking step comparing answer against source catches a share of the cases. No substitute for a person, but a good filter.
- Human sign-off where it counts. Anything that goes outside or triggers a decision gets reviewed. The question is never whether the system makes mistakes, it is what the most expensive mistake costs.
The legal side
Wrong answers from an AI are not a technical curiosity, they are a question of attribution. A Canadian tribunal ruled in 2024 that an airline was bound by the incorrect information its chatbot had given, and rejected the argument that the bot was a separate entity. In the United States, lawyers were sanctioned in 2023 for filing a brief containing invented case law. Such cases have repeated since.
For operators that means: publish an AI answer and you own it like any other statement from your company. The EU AI Act additionally requires that people can tell when they are talking to a system rather than a person. A visible notice on an AI chatbot is therefore an obligation, not a courtesy.
Frequently asked questions about AI hallucinations
Will newer models solve the problem? The frequency drops, the pattern stays. Models with search access and with reasoning steps get factual questions wrong far less often, but they still invent when a question goes beyond what they know. Anyone building an application therefore plans for mistakes rather than hoping they disappear.
Can I forbid invention in the prompt? Only partly. Instructions such as “answer only with evidence from the text supplied”, plus a permitted empty answer, help measurably. A prohibition on its own does not work, because the model cannot tell whether a statement came from the context or from training.
Does a lower temperature help? Somewhat. A low temperature makes output more predictable and keeps it closer to the wording of the context, but it prevents no wrong statement if the context holds no right one. It makes the model more consistent, not more truthful.
How do I spot it as a user? Watch for very specific claims with no source: years, clause numbers, studies, quotations, function names. The more concrete and the less referenced, the more carefully you check. And when a model names its source on request, that is not evidence, it is potentially the next hallucination.
Conclusion
Hallucinations are not a bug that gets fixed one day, they are the flip side of being able to produce plausible language. What makes language models usable is therefore not trust but procedure: tie answers to sources, show the evidence, review wherever it gets expensive. How we build applications so that mistakes surface instead of slipping through is described on our page about custom software development, and our assessment from project practice is in the article AI in software development. Why a chatbot tied to a fixed set of documents says nothing rather than something invented is covered on our page about AI chatbot development. If you want to talk a specific case through, there is a free consultation.