Building an AI chatbot that doesn't make things up

Because it was never given three things: a boundary, access to the sources that contain the answer, and a way to say it does not know.

The outcome is predictable. A bot that states something wrong with total confidence, and is then trusted by nobody. One such answer doing the rounds internally costs more goodwill than the whole build returned.

The fix is rarely a better model. It is reversing the order: first establish what it must answer and what it must never touch, then build.

Start with two lists. Questions it must handle, and — equally important — questions it must never touch on its own.

That second list gets skipped almost every time. Pricing commitments, medical advice, legal liability, anything resembling a promise: those need a handover, not a generated answer.

A chatbot with a sharp boundary is more useful than one with an opinion about everything.

This drives answer quality more than the choice of model does. Clean up the documents, give them structure, and record for every passage where it came from.

It works for manuals, policy documents, contracts, product data, and past tickets. Wally works exactly this way: tax answers grounded in official sources, plus analysis of the invoices and contracts a user uploads.

The boring part of this work is the part that makes the difference. Skip it and you get a chatbot that sounds convincing and is wrong with some regularity.

A chatbot that always says something will eventually say something false. So set an explicit floor: too little grounding in the sources means it says so and hands over.

That feels counterintuitive. You built it to answer, after all. But users forgive a handover; they do not forgive an answer that turns out to be wrong two weeks later.

Put source citation beside it. Every answer points at the document, article, or record it came from, so somebody can check it without calling you.

Build a test set from questions that were actually asked. Including the awkward ones — especially those.

Without one, every adjustment is a guess and you notice a regression when a customer hits it. With one, you can compare models, adjust prompts, and add sources while watching the effect.

Then go live narrowly. Internally first, or a slice of traffic. Widen the audience only once the answers to real questions hold up.

The questions coming in are research you are not paying for. They show which document is missing, which process is unclear, and where your site leaves people stranded.

Capture and cluster them. In practice that analysis often returns more than the chatbot itself, because it exposes problems you would otherwise never have seen.

• Conversations where it did not knowThe most valuable material you have. Review periodically and turn into sources or boundaries.
• Questions that keep recurringAlmost always a sign something is missing from your documentation, not from your chatbot.
• The moments a human takes overThat is where the real complexity of your service lives. Useful to know before your next hiring round.