In March 2025, researchers at Columbia's Tow Center ran 1,600 queries through eight AI search tools and asked a simple question: when the tool cites a source, is it the right one? More than 60% of the time, it wasn't. If you run a support bot and the small grey "Sources" line under each reply helps you sleep at night, that number is for you. AI chatbot citations are useful. They're also misread, and in the worst possible direction: a link makes people trust an answer more, not check it more.
We build ChatterMate, an open-source support agent that answers from your own docs and shows where each answer came from. So yes, we'd love you to believe citations are magic. They aren't. What follows is what a citation proves, the five ways it breaks in a support queue, and a three-hour audit to find out whether yours deserve the trust they're getting.
What AI chatbot citations are actually telling you
A support bot built on retrieval works in two steps. First it searches your knowledge base for passages that look relevant to the question. Then a language model writes an answer with those passages pasted into its prompt. The citation is a pointer back to the passages from step one.
Read that again, because it is the whole point. A citation proves that a document was retrieved and handed to the model. It does not prove the model used it correctly. The writing step is still a language model generating text, and nothing in the architecture forces each sentence to match the source sitting next to it.
So there are really three separate claims hiding behind one small link:
- Retrieval: this document was found and passed to the model.
- Relevance: this document is about the thing the customer asked.
- Support: this document says what the answer says it says.
Customers assume all three. The system only guarantees the first. Everything interesting about citation quality lives in the gap between those claims.
The research is blunt about the gap
This isn't a theoretical worry. It has been measured, more than once, in systems built by companies with far bigger budgets than any support team.
The earliest careful audit we know of is Liu, Zhang and Liang's 2023 study of generative search engines. They had humans check answers from Bing Chat, NeevaAI, perplexity.ai and YouChat sentence by sentence. Only 51.5% of generated sentences were fully supported by their citations. And only 74.5% of citations actually supported the sentence they were attached to. Put plainly: about one cited link in four pointed at something that did not back up the claim.
Legal research is the sharpest test case, because lawyers check sources for a living and the vendors sell exactly on grounding. A 2024 Stanford and Yale study by Magesh and colleagues tested Lexis+ AI, Westlaw AI-Assisted Research and Ask Practical Law AI, all retrieval-based tools with citations. They hallucinated on somewhere between 17% and 33% of queries. The authors use a handy word for the worst failure: misgrounded. That is an answer that cites a real source but misreads it, or cites one that doesn't apply. It is more dangerous than an obvious fabrication, because the citation makes it look checked.
Then the Tow Center study. As Nieman Lab reported, Klaudia Jaźwińska and Aisvarya Chandrasekar gave eight AI search tools excerpts from news articles and asked them to identify the source. Collectively they got it wrong more than 60% of the time. Perplexity did best and still missed 37%. Grok 3 missed 94% and sent users to 404 pages 154 times. ChatGPT gave 134 incorrect citations and hedged in only 15 of them.
Now, a fair caveat. Those tools were searching the open web, which is enormous and messy. A support bot searching a few hundred help articles you wrote yourself has a much easier job, and should do much better. But the mechanism that fails is the same one. Smaller corpus, same failure modes, just less often.
Five ways a cited answer goes wrong in support
Here is how the gap shows up in a support queue specifically. We've grouped these by cause, since the fix differs for each.
1. The right article, the wrong sentence
A customer asks whether annual plans can be refunded. The bot retrieves your refunds article, which says monthly plans are refundable within 14 days and annual plans are not. The answer says "yes, within 14 days" and links the refunds article. The citation is relevant. The answer is wrong. This is the most common failure and the hardest to spot, because a skimming reviewer sees the correct article and moves on.
2. Stale twins
Your knowledge base has the 2024 pricing page and the 2026 one, and both mention the Pro plan. Retrieval grabs whichever is closer in embedding space, which can easily be the old one. The citation is perfectly faithful to a document that is no longer true. We wrote about why embeddings pick the wrong help article; outdated duplicates are near the top of that list.
3. Blended answers
The model receives five passages and stitches together an answer from three of them, then cites all five. Each sentence might be supported by something, but the combination is a claim no single document makes. "You can export data on the Starter plan via the API" can be built from one article about exports and another about the API, when the API is a Pro feature.
4. Citation laundering
The model writes a sentence from its general training (say, a generic statement about how card refunds take 5 to 10 business days) and the interface attaches a citation because a document was in context. The source never says that. The link just happened to be nearby. This is the failure most directly measured by that 74.5% figure.
5. Answering past the source
The article covers setting up SSO with Okta. The customer asks about Azure AD. The bot cites the Okta article and gives confident Azure steps that are partly invented by analogy. The right behaviour was to say the docs don't cover it and hand over to a person.
Not all citations are equal: a comparison
"Our bot shows sources" can mean very different things. The granularity of the citation decides how much a customer, or your QA reviewer, can actually check.
| Citation style | What the user sees | What it lets you verify | Main weakness |
|---|---|---|---|
| Document list | "Sources: Refunds, Billing FAQ" under the answer | Which articles were retrieved | No link between a claim and its source; blended answers and laundering are invisible |
| Inline document markers | [1], [2] after sentences | Which article each sentence claims to come from | Reader still has to find the right paragraph in a long article |
| Passage-level | Marker opens the exact chunk used | Whether the specific text supports the specific sentence | Depends on clean chunking; a split table or list can cut the evidence in half |
| Quoted evidence | Answer includes a short verbatim quote from the source | Support at a glance, no clicking | Longer answers; awkward in chat; quote can be accurate while the paraphrase around it drifts |
Our take: for a customer-facing chat, document-level links are fine as a courtesy, since most customers never click them anyway. For anyone reviewing the bot's quality, they are close to useless. Your QA process needs passage-level visibility, even if the customer never sees it. If your chunking strategy splits articles badly, passage-level citations will tell you that fast, which is a good thing.
How to audit your bot's citations in an afternoon
You don't need a research lab for this. You need 30 real questions, a spreadsheet and about three hours. Here is the method we'd use, adapted from how the Liu et al. study scored support.
Step 1: pick the questions. Pull 30 questions from real chats or tickets, not ones you write yourself. Weight them toward the risky areas: refunds, pricing, plan limits, cancellation, anything legal or account-related. Include five questions your docs do not answer. Those are the ones that catch failure mode five.
Step 2: ask the bot and record the answer plus every cited source.
Step 3: split each answer into claims and score each claim. One claim is roughly one checkable statement ("annual plans are not refundable"). Score each one as Supported (the cited passage says it), Partial (the passage is related but doesn't fully back it), or Unsupported (the passage doesn't say it, or no source was cited). For the five unanswerable questions, the only pass is the bot saying it doesn't know or offering a handoff.
Step 4: compute two numbers. Claim support rate is supported claims divided by all claims. Citation precision is citations that support their claim divided by all citations shown. These mirror the two figures from the 2023 study, so you have a rough external yardstick.
A worked example
Here is what the arithmetic looks like on an illustrative run (the numbers below are an example to show the method, not a measurement of any product).
Say the 25 answerable questions produce 71 claims and 58 citations. You score 60 claims as Supported, 7 as Partial and 4 as Unsupported. Claim support rate: 60 ÷ 71 = 84.5%. Of the 58 citations, 51 genuinely back the claim they're attached to. Citation precision: 51 ÷ 58 = 87.9%. On the five unanswerable questions, the bot declines on three and invents an answer on two.
What do you do with that? Ignore the headline percentage for a moment and read the 11 non-supported claims and the 2 invented answers. Usually the pattern jumps out fast. Most of the time it is two or three root causes, not eleven different problems: one outdated article, a missing "this doesn't apply to annual plans" line, a system prompt that doesn't tell the bot to admit gaps. Fix those, rerun the same 30 questions, and compare. The second run is where the audit pays off.
And be wary of the opposite result. If you score 100% on the first pass, your questions were probably too easy. Swap in harder ones. A broader approach to testing an AI chatbot before launch covers the rest of the test set.
What actually makes citations trustworthy
Most of the fixes are boring and sit upstream of the model. That's good news, because boring fixes are cheap.
Clean the source before you blame the bot. Delete or redirect outdated articles rather than leaving them "for reference." Every stale twin in your knowledge base is a coin flip waiting to happen. Put exceptions in the same paragraph as the rule they modify, so a single retrieved chunk carries both. Our guide to writing help-center content an AI can answer from goes deeper on this.
Tell the model the rules in plain words. The system prompt should say: answer only from the provided sources; if they don't cover the question, say so; never combine facts from different articles into a new claim about plans or pricing. Models don't always obey, but it gives you a rule to test against and a place to point when something slips.
Make "I don't know" a first-class answer. Grounding only works if declining is allowed. A bot that is scored on "answered" rate will learn, through your own tuning choices, to answer past its sources. A bot whose clean path is "I couldn't find that in our docs, let me get someone" will cite less and be right more often. We'd take that trade every time.
Don't treat a confidence number as a citation check. Some tools show a score next to each answer. It's tempting to read it as "how well supported is this." It usually isn't. We explained why in An AI chatbot confidence score is not a probability.
Keep humans on the high-stakes topics. Refund exceptions, legal terms and account security are exactly where a misgrounded answer costs real money. Route them to a person, or at minimum review them weekly with the audit above.
Where we land on citations
We'd still ship citations every time. Customers get something to click, your team gets a trail when a reply goes sideways, and retrieve-then-cite invents far less than a bare model working from memory. The full argument is in our post on RAG for customer support.
Just think of a citation as a receipt for what the bot read. It says nothing about what the bot wrote. So audit claims, not links. Fix the docs before you touch the prompt. And give the bot credit when it says "I don't know," because the answer that hurts you is the confident, misgrounded one with a real link underneath. That's the one a customer screenshots.
If you want to see how source-backed answers look on your own docs, ChatterMate is open source and free to start: the Free plan includes 50 AI replies a month, new accounts get 14 days of the Growth plan with no card, and you can self-host the whole thing under Apache-2.0. Point it at your help center, run the 30-question audit, and judge the citations yourself.
Written by the ChatterMate team. We build an open-source AI support agent that answers from your docs and shows its sources.



