AI Chatbot Hallucinations: Why They Happen, and What Grounding Actually Fixes

A man's grandmother died. He went to Air Canada's website, asked the chatbot about bereavement fares, and it told him he could claim the discount retroactively within 90 days. So he booked the flight and applied later. The policy the bot described didn't exist. Air Canada refused the refund, he took them to a tribunal, and in early 2024 the Civil Resolution Tribunal ordered the airline to pay him about $812 — rejecting the airline's argument that its chatbot was somehow a "separate legal entity" responsible for its own words.

That's what AI chatbot hallucinations look like in the wild. Not a funny wrong answer in a demo. A made-up policy, stated with total confidence, that a real customer acted on — and a company that got held responsible for it.

If you're putting an AI bot in front of customers, this is the failure mode that should keep you up at night. Not that the bot is rude. That it's fluent, helpful-sounding, and wrong.

What a hallucination actually is

A hallucination is when a model generates something that reads as fact but isn't grounded in anything real. It's not lying, because lying needs intent. The model isn't consulting a database and choosing to misreport it. It's predicting the next most plausible words, and sometimes the most plausible-sounding sentence is fiction.

Here's the part that trips people up. The bot is often most confident exactly when it's wrong. There's no little tremor in the text that says "I'm guessing now." It invents a refund window in the same calm tone it uses to tell you your order shipped.

And it happens to the companies building this technology, not just the ones buying it. In April 2025, Cursor — an AI coding tool — had users getting logged out by a session bug. Its own support bot, named "Sam," explained the logouts by inventing a policy: your subscription only works on one device now. That policy didn't exist. The bot made it up. The claim spread across Hacker News and Reddit within hours, and people started cancelling subscriptions over a rule nobody had ever written. The co-founder had to step in and apologize.

Two companies. Two bots. Two fabricated policies that cost real money. Neither was a fringe operation.

Why models do this at all

For a long time the industry treated hallucination as a bug we'd eventually patch out. Bigger model, cleaner data, problem solved. That framing took a hit in September 2025, when OpenAI published research arguing that hallucination isn't a weird glitch — it's baked into how we train and grade these models.

The argument is almost uncomfortably simple. We score models mostly on accuracy: how many questions did it get right? Think about what that rewards. If a model doesn't know an answer and guesses, it might get lucky and score a point. If it says "I don't know," it scores zero, guaranteed. So over millions of training examples, the model learns the same thing a student learns cramming for a multiple-choice exam — when unsure, guess confidently. Never leave it blank.

OpenAI's own numbers make the point. A model tuned to abstain — to say "I'm not sure" about 52% of the time — produced far fewer outright wrong answers than a version that abstained only 1% of the time and bluffed through the rest. The bluffer looks more helpful. It's wrong more often. A related paper in Nature made the same case: the way we evaluate these models actively rewards them for faking it.

Sit with that for a second, because it reframes the whole problem. A general-purpose chatbot is, in a sense, trained to be a confident guesser. Point that at your support queue with no guardrails and you haven't hired a careful agent. You've hired the most persuasive person in the room who also happens to make things up.

Why support is the worst place for this

Plenty of AI use cases can shrug off a hallucination. Brainstorming ten blog titles? Toss the two bad ones. Drafting an email you'll edit anyway? Fine.

Support is different, for three reasons.

The customer can't tell. They're asking because they don't know the answer. They have no way to check whether the return window is 30 days or 90. They trust the bot precisely where they're least equipped to catch it lying.

The answers become commitments. Moffatt v. Air Canada wasn't really a story about a wrong answer. It was a story about a company being bound by its bot's wrong answer. Courts and customers increasingly treat what your bot says as what your company said. That's the whole ballgame.

And the damage compounds. One bad answer in a public forum — a coding tool's Reddit thread, a review, a screenshot on X — doesn't stay contained. It becomes the story. Cursor's invented device policy did more brand damage in a day than a dozen slow email replies ever could.

So "our bot is right most of the time" isn't the flex it sounds like. A bot that's right 95% of the time and confidently, untraceably wrong the other 5% is a liability generator with great uptime.

What "grounded" actually means

This is where the word grounded earns its keep, and it's worth being precise, because a lot of vendors say it and mean different things.

An ungrounded bot answers from its training — the fuzzy statistical average of everything it read on the internet. Ask it your refund policy and it'll confidently describe a refund policy. Possibly a competitor's. Possibly one it assembled from vibes.

A grounded bot doesn't answer from memory. Before it responds, it retrieves the actual relevant text from your sources — your help docs, your policy pages, your knowledge base — and it's constrained to answer from that retrieved material. The mechanism underneath is usually retrieval-augmented generation, or RAG. We wrote a whole piece on how RAG works for customer support if you want the mechanics, but the short version: retrieve first, then generate, and only from what you retrieved.

The difference isn't cosmetic. IrisAgent, which builds support AI, puts the ungrounded hallucination baseline at roughly 15–30% and reports that grounding production systems in real knowledge sources drops it under 5%, with accuracy validated above 95%. Other write-ups on RAG grounding put the reduction in the 60–75% range versus an ungrounded model. Numbers vary by setup — they always do — but the direction is consistent and large. Grounding is the single biggest lever you have on hallucination.

Two things matter beyond the raw retrieval, though.

First, citations. A grounded bot should be able to show you where an answer came from — a link back to the doc it pulled from. This does two jobs. The customer can verify, and your team can audit. If the answer's wrong, you can see whether the bot misread the source or the source itself was wrong. An answer you can't trace is an answer you can't trust.

Second, knowing when to stop. Grounding fixes "I made up an answer." It doesn't automatically fix "there was no answer in the docs, so I made one up anyway." A good support bot, when retrieval comes back empty, should say it doesn't know and hand off to a human — cleanly, with context — rather than reaching for a plausible guess. That "I don't know, let me get someone" is not the bot failing. It's the bot working. Remember OpenAI's finding: the willingness to abstain is what separates a reliable system from a confident bluffer.

How to actually reduce AI chatbot hallucinations

If you're evaluating tools or tightening up something you already run, here's what actually moves the needle on AI chatbot hallucinations — roughly in order of impact.

Ground every answer in your own content. Non-negotiable. If a bot can answer a policy question without retrieving your policy, it will eventually invent one. Make retrieval a hard requirement, not a nice-to-have.

Demand citations on every factual reply. If the vendor can't show the source doc behind an answer, you can't audit it, and you're flying blind on exactly the answers most likely to hurt you.

Fix the source material. Grounding faithfully reproduces whatever's in your knowledge base — including the outdated, contradictory, and plain-wrong bits. One report found that cleaning up the knowledge base alone cut "grounded but wrong" answers by 20–30%, just by removing bad source text. Your bot is only as honest as your docs. If your knowledge base is a mess, fix that before you blame the model.

Make "I don't know" a first-class outcome. The bot should have a clear, low-friction path to escalate when it isn't sure, and it should take that path often. A bot that never says "I'm not certain" isn't more capable. It's just hiding its uncertainty from you.

Watch what real vendors claim. This is also where a lot of agent-washing lives — "zero hallucinations" splashed across a landing page with nothing underneath it. Ask the boring questions. Show me a citation. Show me what happens on an out-of-scope question. Show me the escalation path. If the honest answer is "it does its best," that's not grounding. That's a guesser with better marketing.

Where we land on this

We build ChatterMate, an open-source AI support agent, and we made grounding the default rather than a premium tier because we watched cases like Air Canada and Cursor and drew the obvious conclusion: a support bot that can invent policy is a bad trade at any price. Ours answers from your docs, cites the source it used, and hands to a human when the docs don't cover the question. Not because it's timid — because a wrong answer in support is worse than a slow one.

None of this makes hallucination literally impossible. Anyone promising you a zero is selling. But there's a wide gap between a bot that guesses from the whole internet and one that's pinned to your content, shows its sources, and knows when to tap out. That gap is the difference between a tool your customers can rely on and a headline you don't want.

The technology to close it isn't exotic anymore. Grounding, citations, honest escalation — these are table stakes in 2026, not moonshots. If a tool in front of your customers can't do all three, it isn't ready to be there yet.

Written by the ChatterMate team — we build an open-source, doc-grounded AI support agent that cites its sources and hands off to humans when it should. It's free to start (your first 300 chats are on us) and self-hostable if you'd rather keep everything in-house. If you're weighing options, our roundup of the best open-source customer support chatbots is a fair place to start.