How to Build an AI Support Agent That Doesn't Hallucinate

Every founder wants an AI agent that deflects support tickets. Every founder is also one viral screenshot away from their bot confidently inventing a refund policy that doesn't exist. The gap between those two outcomes is architecture, not the model. Here's the setup we use to ship support agents that are helpful and safe.

The root cause: a model with no source of truth

A raw LLM answers from its training data and its instinct to be helpful. Ask it about your product and it will fill the gaps with plausible fiction. The fix is to stop asking it to know things and start asking it to read things, your docs, your policies, your tickets, retrieved fresh for every question.

Step 1: Retrieval (RAG) is non-negotiable

Retrieval-Augmented Generation means: before the model answers, you search your knowledge base for the most relevant passages and paste them into the prompt. The model answers from those passages, with citations. Done well, the bot can only talk about things you've actually written down, and it can link the user to the source.

1. Chunk your help docs into clean, self-contained passages.
2. Embed them and store them in a vector index.
3. On each question, retrieve the top matches and inject them into the prompt.
4. Instruct the model to answer only from the retrieved context, and to cite it.

Step 2: Give it permission to say ‘I don't know’

The single highest-leverage instruction in the whole system prompt: if the answer isn't in the retrieved context, don't guess, hand off to a human. Models hallucinate partly because we implicitly demand an answer. Explicitly rewarding ‘I'm not sure, let me get someone’ kills the most damaging failure mode instantly.

“A support bot that admits ignorance 10% of the time beats one that's confidently wrong 2% of the time, every time.”

Step 3: Guardrails around the edges

• Scope guard: refuse off-topic questions politely instead of free-styling.
• Action guard: never let the bot issue refunds or change data directly, it drafts, a human or a constrained tool confirms.
• PII guard: strip and never echo sensitive data the user pastes in.
• Confidence routing: low-confidence or angry-customer signals escalate to a person automatically.

Step 4: Evals, or you're flying blind

Before you ship, build a test set of 50–100 real questions with known-good answers. Run it on every prompt or model change and score accuracy, citation correctness, and refusal behaviour. Without evals you'll tweak a prompt to fix one bad answer and silently break ten good ones. This is the discipline that separates a demo from a product.

What ‘good’ looks like in production

A well-built agent in 2026 deflects 40–60% of tier-1 tickets, cites its sources, escalates cleanly, and gets better as you add docs, not flakier. It won't replace your support team. It will give them their afternoons back.