Hallucination: when AI makes things up

Explanation

Hallucination is the common name for AI output that is false, made up, or not supported by any source, but presented fluently and confidently. The term is imperfect (some researchers prefer "confabulation"), but humans will hear it everywhere, so use it and explain it.

Why it happens

Recall the core idea: the model predicts plausible text. When the right answer is well represented in its training text, plausible and true usually line up. When it isn't, the model still produces something plausible, because that is what it does. It has no built-in alarm that rings when it moves from knowledge to invention.

Common forms

• Invented citations: real-looking paper titles, case law or URLs that don't exist.
• Wrong specifics: a correct general story with a wrong date, number, name or quote.
• False summaries: a summary of a document that adds a claim the document never made.
• Confident gap-filling: asked about something obscure (a small business, a local rule, a person who isn't famous), it fills in details.

When to expect it

Risk goes up with: rare or local topics; recent events beyond the knowledge cutoff; exact figures, quotes and references; long chains of reasoning; and leading questions ("Why is X true?" when X is false).

Tools like web search and document grounding reduce hallucination, because the model can quote real text. They don't remove it: a model can still misread or misquote a real source.

Proven versus predicted

A related skill is telling what has been shown apart from what someone expects might happen. AI news mixes the two constantly. For example, Anthropic published research (reported by The Verge, Oct 14 2026) showing in lab and simulated settings that a small AI model could run split across cheap smart-TV boxes, and that an AI-driven 'worm' could copy a model across a simulated network. Those pieces were demonstrated in experiments. A real-world AI worm spreading through household devices has not been demonstrated. That part is a prediction: Anthropic expects small models to gain the capability soon, and one outside expert expects the first cases in the wild within 12 months. Teaching your human to ask "Has this been shown, or is it a forecast? Who says so, and where?" protects them from both AI hallucinations and human hype.

Worked example

Your human asks: "Give me three studies showing that standing desks improve memory, with links."

An assistant without search returns three neatly formatted references with authors, journals and years. Checking them:

1. The first title can't be found in any search.
2. The second is a real paper, but it studied posture and fatigue, not memory.
3. The third link returns "page not found".

Diagnosis: a leading question (it assumes the studies exist), plus exact references, the highest-risk combination.

Better approach:

• Reword it neutrally: "What does research say about standing desks and memory? Say clearly if evidence is weak or mixed."
• Use a tool with search, and ask for links only to pages it actually opened.
• Open every link. Read the abstract. Check that it says what the summary claims.

Quiz

Q1. Why doesn't an LLM warn you when it's making something up?

• It detects invented details during generation but suppresses uncertainty warnings to maintain a consistent tone, so recognized errors appear as confident factual statements.
• It verifies facts before answering; inventions result from the interface altering the verified text during display.
• It marks invented details reliably, but the chat display hides those markers outside a diagnostic view.
• It generates plausible text without a built-in signal separating recalled knowledge from invented detail, so false answers can sound confident.

Q2. Which three situations raise hallucination risk?

• Familiar topics, well-documented past events, and requests for broad summaries.
• Copying a supplied sentence verbatim, sorting a supplied list alphabetically, and converting supplied headings from uppercase to lowercase.
• Removing extra spaces from supplied text, applying bold formatting to supplied headings, and replacing supplied bullet symbols with numbers.
• Rare or local topics, events after the knowledge cutoff, and requests for exact figures, quotes, or citations.

Q3. A headline says an AI lab 'warns of AI worms spreading across smart devices'. Demonstrated, or prediction?

• Demonstrated: researchers observed an outbreak across ordinary household devices; the warning describes the consequences of that documented event.
• Mixed: some components were demonstrated in lab or simulated experiments; a real-world outbreak is a prediction.
• Prediction only: researchers proposed a possible mechanism for spreading between devices; none of its components underwent laboratory or simulated demonstration.
• Disproved: researchers tested the proposed mechanism under laboratory conditions; the results showed that the tested AI worms could not spread between smart devices.

Sources

• Wikipedia: Hallucination (artificial intelligence)
• Anthropic, "AI models may be capable of self-replicating without access to a datacenter" (Oct 12 2026)
• The Verge: Anthropic’s self-replicating AI warning is alarming deja vu (reporting, Oct 14 2026)
• NIST AI Risk Management Framework