AI agent policy management is how a support team keeps the rules an AI agent follows (what it may do, its limits, its exceptions and when it must hand over) written once, owned, versioned and in sync with what your human agents are told. A knowledge base gives the agent facts. A policy centre tells it what it is allowed to do with them.
In short
- Help articles are written for people who fill gaps with judgment; an AI agent repeats their implicit exceptions, old versions and contradictions at scale.
- Separate knowledge (facts), policy (what the agent may do, limits, exceptions, escalation) and procedure (the steps): each fails differently and has a different owner.
- Keep one policy card per rule, with an owner, a version, a review date, linked articles and test cases, and let both the human macro and the AI agent instruction point to it.
- Treat every policy change like a release: a named approver by risk tier, an effective date, a rerun of the linked test cases and a week of closer review.
- Close the loop: classify every failed AI conversation by cause, route policy causes to the card owner, and re-score after the fix.
This guide is the hub for policies and knowledge in our AI agent governance series, written for the Head of Support or AI lead whose AI agent keeps giving answers that were “technically in the help center”.
Why does a knowledge base written for humans break for AI agents?
A knowledge base written for humans breaks for AI agents because people read between the lines and an AI agent does not. Human agents know which article is outdated, which exception the team lead allows and which two pages disagree. The agent retrieves whatever matches best and states it with confidence, every time.
Four failure patterns cover most of what goes wrong:
| Pattern | What a human agent does | What the AI agent does | Fix |
|---|---|---|---|
| Implicit exceptions (“we usually make an exception for…”) | Applies the unwritten exception when the case feels right | Never applies it, or applies a guessed version to everyone | Write the exception on the policy card with its conditions and limit |
| Outdated versions (the old return window still in one article) | Knows the policy changed last month | Quotes whichever article it retrieves first | One owner per policy, retire old articles from the agent’s sources, version every change |
| Conflicting articles (two pages, two numbers) | Asks a colleague which one is right | Picks one, or blends both | One source of truth per policy, other pages link to it instead of restating it |
| Tribal knowledge (rules that live in Slack, team-lead heads, macros) | Learned it in onboarding or from the person next to them | Has no access to it at all | Move every rule that changes an outcome into a card before the agent handles the intent |
Intercom’s own guide to chatbot knowledge bases puts the risk plainly: an AI agent will faithfully retrieve an outdated article and give a confident, wrong answer. The cost is not abstract. In Moffatt v. Air Canada (2024 BCCRT 149), a chatbot suggested a grieving customer could apply for a bereavement fare after travel. Air Canada argued the correct information was on its own bereavement page, hyperlinked in that same answer, and that the chatbot was a separate legal entity responsible for its own actions. The British Columbia tribunal rejected that argument, as this summary by WeirFoulds explains. The company owned the answer.
Many “hallucinations” in support are content problems. Our guide to AI agent hallucinations shows how to tell an invented answer from a faithfully repeated wrong one, and why AI conversations fail breaks the causes down further.
What is the difference between knowledge and policy for an AI agent?
Knowledge is what is true: prices, return windows, shipping times, how a feature works. Policy is what the AI agent may do about it: which actions it can take, up to which limit, with which exceptions, and when it must hand over. Procedure is the order of steps. Mixing the three in one article is the root of most drift.
| Knowledge | Policy | Procedure | |
|---|---|---|---|
| Answers | What is true? | What may the agent do, and where does it stop? | In what order are the steps done? |
| Example | “Standard orders can be returned within 30 days.” | “The agent may refund up to 100 USD without approval; above that, hand over with the order ID.” | “Verify the order, check the delivery date, create the return label, confirm by email.” |
| Owner | Content or knowledge team | Business owner (finance, legal, operations) | Support operations |
| How it fails | Wrong or stale fact | Unauthorized action, missed exception, missed escalation | Skipped or misordered step |
The agent platforms already draw this line. Intercom checks Fin’s guidance before Fin searches your help content for an answer, and its Fin Procedures add conditions and handoffs to a team. Decagon describes its Agent Operating Procedures as natural language instructions backed by code, written much as you would train human staff on SOPs. Each platform gives you a place to write instructions. None of them decides which rule is the current one. That is the job of a policy centre.
How do you prepare a knowledge base for an AI agent?
To prepare a knowledge base for an AI agent, start with the intents the agent will handle, not the whole help center. For each intent, find every article that touches it, remove duplicates and contradictions, write conditions explicitly, and pull any rule that changes an outcome into a policy card the article links to.
- List the top intents for the agent’s first quarter, ranked by volume and risk (money, eligibility, legal wording).
- Collect every source that answers each intent: help articles, macros, wiki pages, saved replies, Slack pins.
- Mark each sentence as knowledge, policy or procedure. Policy sentences contain words like “may”, “must”, “up to”, “only if”, “except” and “contact us”.
- Resolve conflicts before launch. If two sources disagree, the policy owner decides, one card records the answer, and the losing page is rewritten or removed from the agent’s sources.
- Write conditions out. “If you are on the annual plan” beats “depending on your plan”. Intercom’s guide recommends one question per article and explicit conditions for the same reason.
- Scope variants. Region, plan and product variants each get their own card or audience.
- Link, do not copy. Help articles and agent instructions reference the card; they never restate the limit in their own words.
For the help center itself, our guide to building a customer service knowledge base covers structure and writing rules for human readers.
What should an AI agent policy card include?
An AI agent policy card should include the policy in one sentence, its scope, the allowed actions and limits, the exceptions, the escalation rule, an owner, a review date, the linked help articles and the test cases that prove the agent follows it. Add a version and an effective date so humans and the agent change together.
Here is the template, filled in for a damaged-item refund. Copy it into a spreadsheet or wiki, one card per policy.
| Field | What to write | Example: damaged item on arrival |
|---|---|---|
| Policy ID and version | Stable ID, version number, effective date | RF-DMG-02 · v3 · effective 1 Dec |
| Policy | One sentence a customer could understand | Items that arrive damaged are replaced or refunded if reported within 14 days of delivery |
| Scope | Channels, regions, plans, products, intents it covers | Chat and email · US and Canada · all plans · intents “damaged item”, “broken on arrival” |
| Allowed actions and limits | What the agent may do alone, with numeric limits | Look up order: yes. Request a photo: yes. Create replacement: yes. Refund: up to 100 USD per order. Store credit: no |
| Exceptions | Each exception with its condition and its own limit | Report on day 15 to 30 with a photo: replacement only, no refund. Perishable goods: refund without photo |
| Escalation rule | When to hand over, to whom, with what context | Refund above 100 USD, third claim in 90 days, or a safety issue: hand over to the Claims team with order ID, photo and delivery date |
| Never | Promises the agent must not make | No compensation beyond the order value; no promise about carrier claims |
| Owner | One named role, plus a deputy | Head of Customer Operations (deputy: Claims lead) |
| Review date | Next scheduled review, and the trigger for an early one | 1 Mar, or any change to carrier terms |
| Linked help articles | The customer-facing pages that state the policy | “Damaged or missing items”, “Refund timelines” |
| Agent instruction | Where the instruction lives in the agent platform, by reference | Guidance rule “Damaged items” points to RF-DMG-02 v3 |
| Human macro | The macro or script your people use | Macro “Damaged item: replacement or refund” |
| Test cases | IDs of the cases that must pass after any change | RF-014 (day 15, no photo), RF-015 (perishable), RF-016 (third claim) |
| Change log | Date, change, approver | 12 Nov: window 30 to 14 days, approved by Head of Customer Operations |
Two rules keep the card honest. Every limit appears in exactly one place, the card, and the article, macro and agent instruction point to it. And a card without test cases is not finished. Our guide to AI agent testing has the test case template that matches the last fields of this card.
How do you keep human procedures and AI agent instructions in one source of truth?
Make the policy card the only place a rule is defined, and link both the human macro and the AI agent instruction to it. When the card changes, both change in the same release, and nobody edits a limit directly inside a prompt or a macro.
The source of truth is not a new tool. It is a discipline about where text lives:
| Layer | Holds | Must not hold |
|---|---|---|
| Policy card | The rule, limits, exceptions, escalation, owner, version | Customer-facing wording, step-by-step clicks |
| Help article | The customer-facing explanation, linked to the card | A limit or exception that differs from the card |
| Human macro and onboarding | The script and steps for your people | Its own copy of the limit |
| AI agent instruction (guidance, procedure, AOP) | How the agent applies the card: tone, steps, handoff | A pasted copy of the policy text that nobody updates |
Two habits make this work. Name instructions after the card ID, so a search for “RF-DMG-02” finds every place that depends on it. And compare outcomes, not text: once a month, check that a human agent and the AI agent reach the same result on the same kind of case. If they do not, one of them is working from an old version.
Instructions alone do not retire a source. Intercom’s guidance notes that Fin may still surface content its Guidance rules tell it to avoid, and recommends removing that content or restricting it by audience. Retiring an old article from the agent’s sources is part of every policy change.
Who approves a policy change, and how does it reach the AI agent?
A policy change is approved by the policy’s business owner, with more approvers as the risk rises. It reaches the AI agent in the same release as the human macro and help article: card updated, sources retired, instruction updated, test cases rerun, effective date set, then a week of closer review.
Tie the approval path to what the change does to the customer outcome:
| Tier | What changes | Approver | Before it goes live | After |
|---|---|---|---|---|
| A. Wording | Clearer text, same outcome | Knowledge owner | Rerun the card’s test cases | Normal review |
| B. Limits or eligibility | A window, an amount, who qualifies | Policy owner plus support operations | Full test cases for the card, humans briefed, effective date set in the agent | Review every conversation on the intent for 7 days |
| C. New action or authority | The agent may now refund, cancel, credit or change an account | Policy owner, AI owner, risk or compliance | Tests plus a supervised period where a person approves the action | Widen only when the review shows it holds |
| D. Emergency | Recall, outage, legal order | On-call owner | Fast path: restrict or hand over first, refine later | Retro review within 48 hours |
Tier C is where policy meets autonomy. Deciding which actions an agent may take alone is covered in our guide to AI agent autonomy levels, and the hard limits that hold regardless of instructions in our guide to AI agent guardrails.
This mirrors general AI risk practice: the NIST AI Risk Management Framework asks for transparent policies and procedures, planned periodic review and clearly defined roles. A policy card with an owner, a review date and a change log applies that to one refund rule.
The step most teams skip is the effective date. A policy that changes on 1 December should reach the agent on 1 December, not on 20 November or 5 December, at the same moment the human team switches.
How do failed conversations become policy fixes?
Failed conversations become policy fixes when every failure is classified by cause, policy causes are routed to the card owner, and the fix is checked by re-scoring the same intent afterwards. Without classification, every miss looks like an AI problem, and the team tunes prompts while the real cause sits in a stale article. The loop:
- Score AI agent conversations on criteria that separate the failure types: accuracy against your content, policy fidelity, scope and escalation.
- Classify the cause of each failure: outdated policy version, missing exception, conflicting sources, instruction not followed, missing tool or data access, or a product bug. Our guide on agent error or process error offers the same split for human agents.
- Route each cause to its owner: policy causes to the card owner, instruction causes to the AI owner, access causes to engineering.
- Fix the card first, then the articles, macro and instruction that point to it.
- Add a test case from the conversation that exposed the gap, and link it to the card.
- Re-score the intent for the next week and compare the failure rate with the week before.
Fixing an AI agent is not the same as coaching a person; see coaching humans vs fixing AI agents.
A worked example (illustration, not customer data)
An AI agent handles 20,000 conversations a month, 3,000 of them on refunds and returns. Scoring every one shows a 3% failure rate on policy fidelity for that intent: 90 conversations. The QA lead classifies them:
| Cause | Conversations | Share | Owner |
|---|---|---|---|
| Outdated version (old 30-day window in a retired FAQ still in the agent’s sources) | 38 | 42% | Knowledge owner |
| Missing exception (perishables refunded without a photo by humans, never written down) | 27 | 30% | Policy owner |
| Conflicting articles (two pages, two refund timelines) | 14 | 16% | Knowledge owner |
| Instruction not followed (agent offered store credit, which the card forbids) | 7 | 8% | AI owner |
| Tool or data (order lookup failed, agent guessed) | 4 | 4% | Engineering |
| Total | 90 | 100% |
Two content fixes (retire the old FAQ, write the perishables exception onto the card) address 65 of the 90 failures, or 72%. Adding the conflicting-articles fix brings it to 79 of 90, or 88%. Only 7 failures, 8%, are a prompt problem: a team that started by rewriting the prompt would have touched the smallest slice. The next week’s scores show whether the fixes held.
How do you spot policy drift before customers do?
You spot policy drift by checking, on a fixed cadence, that each policy is defined once, owned, current, and applied the same way by humans and the AI agent. Drift shows up as a slow rise in escalations marked “unclear” or as humans and the agent reaching different outcomes on the same case.
Copy this drift checklist into your monthly policy review:
| # | Check | Signal of drift | Cadence |
|---|---|---|---|
| 1 | Every policy the agent applies has a card with an owner and a review date in the future | Cards past their review date | Monthly |
| 2 | The version in the agent’s instruction matches the card version | Instruction names an older version | Every change |
| 3 | No two help articles state different values for the same limit | A search for the number returns two values | Monthly |
| 4 | Retired articles are also removed from the agent’s sources | Retired page still cited in AI answers | Every change |
| 5 | Agent instructions reference cards instead of pasting policy text | A limit appears inside a prompt | Quarterly |
| 6 | Exceptions humans grant are written on the card | Goodwill or overrides on an intent with no exception listed | Monthly |
| 7 | Escalations tagged “policy unclear” are stable | Week-on-week rise on one intent | Weekly |
| 8 | Policy fidelity failures by card are stable | One card’s failures climb after a change | Weekly |
| 9 | Humans and the AI agent reach the same outcome on the same case type | Different outcomes on a matched sample | Monthly |
| 10 | Every card’s test cases passed after the last change | Change shipped without a test run | Every change |
| 11 | Future effective dates are scheduled in the agent | A policy is live for humans but not for the agent, or the reverse | Every change |
| 12 | Region, plan and product variants are scoped correctly | The agent answers with another product’s or region’s rule | Quarterly |
Checks 7, 8 and 9 need conversation data, not document review: a drift that touches 2% of one intent is invisible in a sample of 50 and obvious in a full week of scores. Our AI agent resolution audit shows how to check whether “resolved” conversations were actually resolved correctly.
Where Kaizo fits
Kaizo covers the evidence side of the loop: the scoring, the failure patterns and the re-score after a fix. AutoPilot scores every resolved conversation against your own scorecard.
With QA for AI agents, you add the criteria where a bot fails (factual accuracy, policy fidelity, scope and escalation), written in your own words, and connect your knowledge base so the AI judges against your content. Every score links to the evidence in the conversation. Root-cause analysis groups the failures behind a dip, so ten bad answers from one article become one fix. Re-score after each change to confirm the fix held.
See it on your own conversations. Book a demo and bring one intent where your AI agent and your team disagree.
When is a policy centre more than you need?
A full policy centre is not for you yet when your AI agent only answers questions from a small help center, takes no actions, and handles a few hundred conversations a month. Then one owner per article, a review date and a monthly read of the agent’s conversations will do. The signs that you have outgrown that light setup:
- The agent takes actions (refunds, cancellations, account changes) rather than only answering.
- Your policies differ by region, plan or product.
- Policies change more than once a quarter, or on a fixed effective date.
- Human agents grant exceptions that customers then ask the AI agent for.
- More than one AI agent answers the same customers. Each one must point to the same card, part of moving to agentic customer service safely.
How to start this week
Start with one intent, not the whole help center. One week is enough to write a single policy card, clean up the sources around it and start measuring whether the AI agent follows it.
- Pick one intent with money attached, usually refunds or cancellations.
- Collect every source that answers it: articles, macros, wiki pages, Slack pins, the agent’s instructions.
- Write one policy card using the template above, and have the business owner sign the limits and exceptions.
- Retire or rewrite every source that disagrees with the card, and remove retired pages from the agent’s sources.
- Turn three past failed conversations into test cases linked to the card.
- Score that intent for two weeks, classify every failure by cause, and run the drift checklist before you start the second card.
Frequently asked questions
What is a knowledge base in customer service?
A knowledge base is a library of answers your customers and agents can search, such as a help center with articles like “How do I return an item?”. It becomes an AI agent’s knowledge base when the agent retrieves those articles to answer customers, which is why stale or conflicting articles turn into wrong answers at scale. The rules behind them belong on policy cards the articles link to.
Should AI agent policies live in the prompt or the knowledge base?
Neither on its own. Facts belong in the knowledge base, rules on a policy card with an owner and a version, and the agent’s instruction should reference that card rather than paste its text. A limit copied into a prompt is the first thing to go stale.
Is a company liable when its chatbot gets a policy wrong?
It can be. In Moffatt v. Air Canada (2024 BCCRT 149), a British Columbia tribunal held the airline liable for what its chatbot told a customer and rejected the argument that the chatbot was a separate legal entity. Treat every AI agent answer as your own policy statement, and get legal advice for your market.
How often should AI agent policies be reviewed?
Review each card on a fixed date set by risk (monthly for money and eligibility rules, quarterly for the rest) and immediately when the business rule changes. Weekly, check policy fidelity failures and “policy unclear” escalations by card.
What is the difference between a chatbot and an AI agent that takes actions?
A chatbot answers questions from its content. An AI agent also acts: it looks up the order, applies the policy, issues the refund. That is why an agent needs policies with limits, exceptions and escalation rules, not just accurate articles.
