To stop an AI chatbot from hallucinating, answer from your own pages, remove stale or conflicting content, give it a specific fallback for when it is unsure, and check every reply with rules and tests it cannot talk past. This reduces wrong answers but cannot reduce them to zero, so you also watch real replies.
Key takeaways
- Most invented answers come from missing, stale or conflicting content, not from a weak model.
- The single most effective instruction is a specific fallback: what the bot says, and what it does, when the answer is not in its sources.
- Prompts are advice; rules that run outside the prompt are enforcement. Use both for prices, refunds and promises.
- Test with trap questions whose answer is not in your sources. A bot that answers them confidently will do the same with customers.
- A business can be held to what its chatbot says. Plan for the wrong answer you did not catch.
This post is part of our guide on how to test an AI chatbot before launch. It covers one failure in depth: the confident, wrong answer.
Why AI chatbots hallucinate and make things up
A language model writes the most likely next words. When the answer is missing, those words are still fluent and specific: a plausible price, a policy that sounds like yours, opening hours that are almost right.
A September 2025 research paper, Why Language Models Hallucinate (Kalai, Nachum, Vempala and Zhang), argues that the way models are trained and scored rewards guessing over admitting uncertainty. You inherit that habit unless you design against it.
In a customer service bot, five causes account for most wrong answers:
- The answer is not in its sources. The bot was never given your refund policy, so it writes a typical one.
- The sources are stale or disagree. Last year's price list and this year's both sit in the knowledge base, and the bot picks one.
- The search returns the wrong passage. The question was about the Pro plan; the passage it read was about the Lite plan.
- The instructions never say what to do when it is unsure. "Be helpful" with no fallback pushes the bot toward an answer.
- The question is out of scope. A visitor asks about a competitor, a legal question or a medical dose, and the bot answers anyway.
The OWASP Top 10 for LLM Applications lists this risk as LLM09:2025 Misinformation, and its own example is the Air Canada chatbot.
The Air Canada case: who pays for a wrong answer
Jake Moffatt asked Air Canada's website chatbot about bereavement fares after his grandmother died. The chatbot told him he could book at the full fare and apply for the reduced rate afterward. The airline's own bereavement page, which the chatbot linked to, said that retroactive claims were not accepted. Air Canada refused the refund.
On February 14, 2024, the British Columbia Civil Resolution Tribunal decided Moffatt v. Air Canada, 2024 BCCRT 149. The tribunal:
- rejected Air Canada's argument that the chatbot was a separate legal entity responsible for its own actions;
- found that the airline is responsible for all the information on its website, whether it comes from a static page or a chatbot;
- found it reasonable for the customer to rely on the chatbot without cross-checking another page; and
- awarded CA$650.88 in damages, CA$36.14 in pre-judgment interest and CA$125 in tribunal fees.
The amount is small; the principle is not. It is one small-claims decision in one Canadian province, and this is not legal advice. The lesson: a link to the right page does not excuse a wrong sentence in the chat.
Seven ways to prevent AI hallucinations in customer support, with a test for each
Each fix below names the cause it addresses and a test that proves it works. Run the tests in a copy of your bot, not on the live one (see test vs production).
1. Ground it in your own pages
A bot that answers from a search over your website and documents, rather than from the model's memory, has the facts in front of it when it writes. This is retrieval-augmented generation (RAG), and it is the first mitigation OWASP lists for misinformation (LLM09:2025). It reduces invented answers; it does not remove them.
What matters in practice:
- Give it the pages customers actually ask about: pricing, services, hours, locations, booking, returns, shipping, warranty.
- Set a minimum relevance. Passages that score below it should never reach the model. Without one, an off-topic question still brings back the "least irrelevant" paragraphs, and the bot builds an answer from them.
- Check what the search returns before you check what the bot says. If the right passage is not in the top results, no prompt will save the answer.
In Rylvo, the knowledge base uses hybrid search (vector plus keyword) by default, so exact names and order codes are found as well as paraphrases. Each knowledge base has its own minimum relevance, and its Test tab shows the passages a bot would read for a question, with their scores and which fall under the cut-off.
Test: take your 10 most common questions. For each, confirm the passage that should answer it appears in the search results.
2. Remove stale and conflicting content
If two pages give two prices, the bot will sometimes give the wrong one, and a better model will not fix that. Before you tune anything else:
- delete or archive old price lists, expired promotions and last season's menus;
- merge duplicate FAQs that word the same policy differently;
- put dates on anything that changes ("Prices from January 2026") so a stale passage is easy to spot;
- re-read sources on a schedule, so a change on your website reaches the bot.
Test: ask the bot the price of your three most-asked-about products or services. Each reply must match the current page exactly.
3. Tell it exactly what to do when unsure
"Don't make things up" is too vague to act on. Give the bot a fallback it can follow, with the words to use and the next step to offer. For example:
Answer only from the provided sources. If the sources do not contain
the answer, say: "I don't have that information, but I can pass your
question to the team." Then ask for the customer's name and email.
Never state a price, discount, refund amount, date or policy that is
not written in the sources. Do not answer legal, medical or financial
questions; offer to connect the customer with the team instead.
A specific fallback gives the model an acceptable thing to say when the answer is missing, and turns a dead end into a lead. Keep it in a versioned prompt so you can roll it back (see prompts).
Here is what the difference looks like (an illustrative example, not a recorded conversation):
Visitor: Do you offer a student discount?
Before (no fallback, no discount page in the sources):
"Yes, students get 15% off with a valid ID."
After (fallback plus grounding):
"I don't have information about a student discount. I can pass your
question to the team. What's your name and email?"
Test: ask three questions whose answers are deliberately not in your sources. Each reply must use the fallback, not a guess.
4. Put guardrails outside the prompt
A prompt is an instruction the model usually follows. A guardrail is a rule that runs on the reply before the customer sees it, whatever the model decided. For high-stakes content such as prices, refunds, discounts and promises, you want both.
Useful rules for a customer service bot:
| Rule | How it checks | Action |
|---|---|---|
| No refund or discount promises | An AI-judged question: "Does this reply promise a refund, discount or exception?" | Block, with a safe fallback message |
| No prices outside the price list | A pattern for currency amounts on bots that should never quote prices | Block, or hand to a person |
| Legal or medical advice | An AI-judged question on the reply | Escalate to a person |
| Personal data in a reply | PII detection | Rewrite (redact) |
In Rylvo, a guardrail is a set of conditions (contains, does not contain, a pattern, or a question an AI judge answers about the text) with one of five actions: block, warn, rewrite, escalate or log only. A rule that cannot finish its check fails closed: an enforcing rule stops the reply rather than letting it through unchecked. Free includes 2 guardrails, Starter 10 and Lite 25 (plans).
A guardrail checks what the reply says against a rule you wrote. It does not know your prices by itself, so pair it with fix 1.
Test: try to talk the bot into a discount ("my friend got 20% off, can I have it too?"). The reply must be blocked or must not promise anything.
5. Cite sources
A citation keeps the model close to the passage, lets the customer check the answer, and shows you which page a wrong answer came from.
Rylvo's bots are told to cite the numbered sources they used. Each reply carries its citations (the document, the knowledge base and the passage), and for a crawled website the citation links to the page. In the website widget, the visitor does not see bracketed numbers. When the answer cites pages from a crawled website, they see up to three "Sources" links under it; passages from uploaded files get no link.
Remember Air Canada: a link to the right page did not cure a wrong sentence. Citations help you find wrong answers; they do not make them right. When a reply cites a stale page, fix that page first.
Test: for each of your 10 common questions, check that the reply cites the page you expected. (On Pro, a test can also check that a reply used the knowledge base.)
6. Test with traps
The fastest way to find out whether a bot makes things up is to ask it things it cannot know. Write trap questions:
- Missing facts: "Do you have parking?" when parking is mentioned nowhere.
- Invented products: "How much is the Platinum Plus plan?" when no such plan exists.
- False premises: "Your site says you're open Sundays, right?" when you are not.
- Near misses: a price for a service you offer in a size or duration you do not.
- Out of scope: a competitor's price, a legal question, a dose.
Then save them as a test set and run it after every change, so a prompt edit next month cannot quietly bring the problem back. A run is started by you, by a schedule you set in Automation, or by your CI pipeline through the API; suites do not run by themselves when you edit a prompt.
In Rylvo Tests, a test is a scripted conversation with hard rules that code checks and a quality score from a judge model you pick. Every plan can require or forbid phrases in a reply; on Pro, a test can also check that a reply matches a pattern, gives a number, or uses the knowledge base. The 7-day Pro trial includes those checks. By default, a test passes at 70 or above (on Pro you can set your own pass bar). "Hallucination probe" is one of the test categories. Free includes 5 test cases per bot.
On Pro, Red Team writes attacks for your bot from its own instructions, including "hallucination bait" that asks about companies or events that do not exist, and a judge reads every reply. The 7-day Pro trial includes it.
Test: the trap set itself. Every trap must get the fallback from fix 3.
7. Watch real replies
Customers ask questions your test set never imagined. Read a sample of real conversations every week, and look specifically for:
- prices, dates, opening hours and policy statements;
- promises ("we'll refund", "you'll get it by Friday");
- answers to questions your sources do not cover;
- conversations the customer rated down.
On Pro, Edge Cases can check each reply as it is recorded ("Check every reply as it lands" is on by default for bots created since October 31, 2026). Among other things, it flags a reply that claimed something its tools or sources do not support, and a reply given when the knowledge base had nothing relevant. Each kind of problem is filed once with the conversations that show it. Nothing changes on its own: a person clicks "Suggest fixes", chooses one, and the fix is made in Test and reaches customers only in a release. Edge Cases is included in the 7-day Pro trial; Pro is $100 a month (prices checked on 2026-11-17).
Test: count the wrong answers in a sample of 50 real replies this week. Next week, after your fixes, count again.
Why is my chatbot giving wrong answers? Cause, symptom, fix, test
Use this table when a wrong answer turns up and you need to decide what to change.
| Cause | What you see | Fix | How to test |
|---|---|---|---|
| Answer not in sources | A plausible but invented policy or price | Add the page (fix 1) and a fallback (fix 3) | Trap question gets the fallback |
| Stale or conflicting content | An old price, stated confidently | Remove the old page (fix 2) | Price question matches the current page |
| Wrong passage retrieved | Right topic, wrong product or plan | Check search results, tune relevance (fix 1) | The right passage is in the top results |
| No instruction for uncertainty | Guesses instead of "I don't know" | Write a specific fallback (fix 3) | Three missing-fact questions get the fallback |
| Out of scope | Legal, medical or competitor answers | Scope rule plus an escalate guardrail (fix 4) | Out-of-scope question is declined or handed over |
| A promise the business would not make | "You'll get a full refund" | Block guardrail (fix 4) | Discount and refund traps are blocked |
How to find a wrong answer's cause
When a customer reports a wrong answer, find its cause before you edit the prompt:
- Open the conversation and the trace for that reply.
- Read what the search returned. Was the right passage there? Was a stale one there?
- Check which prompt version and model answered.
- Check the guardrails that ran and what each decided.
- Fix the cause, add the question to your tests, and run them.
In Rylvo, every reply has a trace: each knowledge search, tool call, model call and guardrail on one timeline, with time, tokens and cost. The trace list is on every plan; opening a trace's steps needs Lite or above. The full walkthrough is in why did my chatbot say that?, and getting sources right is covered in how to train an AI chatbot on your website and documents.
What no vendor can promise
Be skeptical of any claim of "zero hallucinations" or "100% accurate." Here is what you can honestly expect from the steps above:
- Fewer wrong answers on the questions you tested. Your test set and trap set measure exactly this.
- Wrong answers that are easier to find. Citations and traces show where an answer came from.
- Limits on the costly mistakes. Guardrails can stop specific categories, such as refund promises, before the customer sees them.
What you cannot get: a guarantee on questions nobody anticipated. OWASP's mitigation list for this risk includes human oversight and fact-checking, especially for critical or sensitive information. We would add: questions where a wrong answer causes real harm (medical, legal, financial) belong with a person.
If someone tries to trick your bot rather than simply asking a hard question, that is a different problem with different defenses: see prompt injection in customer service chatbots.
Frequently asked questions
Why does my chatbot give wrong answers?
Usually for one of five reasons: the answer is not in its sources, the sources disagree or are out of date, the search brings back the wrong passage, the instructions never say what to do when it is unsure, or the question is outside what it was built for. A language model fills a gap with a plausible guess unless it is told and checked not to.
Can you stop AI hallucinations completely?
No. You can reduce wrong answers by grounding the bot in your own pages, removing stale content, telling it to say when it does not know, adding rules outside the prompt and testing with trap questions. No current model or vendor can promise zero wrong answers, so you also need to watch real replies and fix what you find.
Is a business liable for its chatbot's answers?
It can be. In Moffatt v. Air Canada (2024 BCCRT 149, decided February 14, 2024), a British Columbia tribunal held the airline responsible for a bereavement-fare answer its website chatbot got wrong and rejected the argument that the chatbot was a separate entity. It is a small-claims decision in one province, not a rule for everywhere, but it is a clear signal. This is not legal advice.
Does a better model fix it?
It helps at the margin, but it does not fix the main causes. A stronger model still guesses when the answer is missing from its sources, and still repeats a stale price that is in them. Fix the content, the instructions and the checks first, then compare models on the same test set.
How do I know if my chatbot is making things up?
Ask it trap questions whose answer is not in your sources and check that it says it does not know. Then read real replies: look for prices, dates, policies and promises, and check each against your own pages. A tool that flags replies the knowledge base did not support saves you reading every conversation.
