AI Agent Failure Analysis: Hallucination or Missing Policy?

AI agent failure analysis means classifying every failed AI conversation by its cause before anyone fixes it. Most failures that get called “hallucinations” are something else: a missing article, an outdated or ambiguous policy, a retrieval miss, a broken tool or a bad handoff. Each cause has a different owner and a different fix, so the label decides whether the fix works.

In short

• “Hallucination” describes a symptom (a confident wrong answer), not a cause. Treat it as the last explanation, not the first.
• Ten causes cover almost every failed AI conversation in support, from knowledge gaps to customer-side confusion.
• One question settles most cases: was there an approved source that answered this, and did the agent use it?
• Content causes go to the knowledge or policy owner, tool causes to engineering, and only reasoning failures to the team that tunes the AI agent.
• Track the mix by cause per 1,000 conversations every week. Shares alone mislead, because a cause can grow as a share while shrinking in count.

Why do teams call every AI agent failure a hallucination?

Teams call every failure a hallucination because the transcript only shows the output: a wrong answer, stated with confidence. The cause sits upstream, in the content, the policy, the retrieval step, a tool or the handoff rules. Without a step that classifies the cause, the most familiar word wins, and the fix lands on the prompt.

The word has a precise meaning. NIST’s generative AI risk profile defines confabulation, “colloquially also referred to as hallucinations”, as systems that “generate and confidently present erroneous or false content” (NIST AI 600-1, section 2.2). That covers the model inventing something. It does not cover an agent that faithfully repeated a stale refund article, or one that followed a policy nobody wrote down.

The difference matters because the wrong label sends the work to the wrong team. A prompt change cannot fix an article that does not exist. When the support tooling company Pylon categorized roughly 200 tickets its AI agent escalated in one week of its own queue, knowledge gaps “accounted for more than half of escalations” and feature requests were 15% (Pylon, September 2026). That is one company’s data, not a benchmark, but neither cause is a model problem.

This guide is a spoke of our AI agent policy management hub, which covers how policies are written, owned and versioned. Here the focus is the step before the fix: naming the cause correctly.

What are the ten reasons AI conversations fail?

AI conversations in support fail for ten main reasons: a knowledge gap, outdated or conflicting knowledge, a missing or ambiguous policy, a retrieval failure, a reasoning error or hallucination, an action or tool failure, a handoff failure, a scope failure, a product bug or feature request, and a customer-side problem. Each leaves a different trace in the transcript.

The table below is the taxonomy to copy. Use it as the pick list in your review form, one cause per failed criterion.

Tag the failed criterion, not the whole conversation, because one conversation can fail for two causes. Causes 6 and 7 often hide inside containment numbers; our guide to silent AI agent failures covers how to expose them. For human agents, the equivalent split is in our guide on agent error or process error.

How do you tell a hallucination from a missing policy?

You tell a hallucination from a missing policy by checking the source. If an approved, current source answered the question and the agent departed from it, it is a hallucination or reasoning error. If no source covered the case, or the rule exists only in the heads of senior agents, it is a missing policy, and no model change will fix it.

The four pairs that get confused most often:

• Hallucination vs missing policy. The test is “could a new human agent, reading only your written sources, have answered correctly?” If not, the AI agent had nothing to be faithful to.
• Hallucination vs outdated knowledge. If the wrong fact appears word for word in an article, the agent did its job. The article is the failure.
• Hallucination vs retrieval failure. If the right article exists but the answer fits a neighboring topic, the agent answered faithfully from the wrong page. Retrieval-augmented generation, the method many support AI agents use, pairs a model with a search over your content (Lewis et al., 2020), so the search step can fail on its own.
• Hallucinated action vs tool failure. “Your refund is on its way” with no refund in the system is either the agent claiming an action it never took, or a tool call that failed silently. The tool log tells you which. Our guide on the AI agent resolution audit covers how to check claimed actions against backend records.

Our guide to catching AI agent hallucinations covers the routine for true hallucinations once you have isolated them. This page is the filter that decides which ones are true.

The triage decision tree: ten questions in order

The fastest way to classify a failed AI conversation is to answer a fixed list of questions in a fixed order and stop at the first “yes” or “no” that names a cause. The order matters: it rules out customer, product and system causes before blaming content, and blames content before blaming the model.

Copy this into your review form. Answer for one failed criterion at a time.

1. Could a human have understood what the customer wanted from what they wrote? No: customer-side (cause 10). Yes: go on.
2. Was the request one the AI agent should handle at all? No, and it handled it anyway: scope failure (cause 8). Yes: go on.
3. Was the root issue a defect, or something the product does not do? Yes: product bug or feature request (cause 9). No: go on.
4. Did the agent need to take or check an action (lookup, refund, cancel, update), and did that call fail, time out or get refused? Yes: action or tool failure (cause 6). No: go on.
5. Should the conversation have gone to a human, at a different moment, or with better context? Yes: handoff failure (cause 7). No: go on.
6. Does an approved source answer this question? No, and the rule is clear to everyone: knowledge gap (cause 1). No, and the rule depends on who you ask: missing or ambiguous policy (cause 3). Yes: go on.
7. Is that source current, and the only source on the topic? No: outdated or conflicting knowledge (cause 2). Yes: go on.
8. Did the agent use that source for this answer? No: retrieval failure (cause 4). Yes: go on.
9. Does the answer match the source? No: reasoning error or hallucination (cause 5). Yes: go on.
10. Is the criterion itself right? If the agent followed a current, approved source faithfully and still failed, the scorecard criterion may be wrong. Send it to the QA lead for calibration rather than forcing a cause.

Question 10 is the safety valve: a taxonomy that cannot say “our criterion was wrong” blames whoever is left. Count how often you reach question 9. It is the only path that ends in “hallucination”.

Who fixes each cause, and how do you route it?

Each cause has one owner, and the owner is decided by where the fix has to happen: content goes to the knowledge or policy owner, systems to engineering, the product to the product team, and only instructions and reasoning to the AI owner. Route the conversation with the evidence attached, never just a count.

Name each owner as a person, not a team. A routing note needs four things: the cause, two or three example conversations with the failing lines marked, the affected intent with its weekly volume, and the date you will re-score. Without the examples, the receiving team argues about the classification instead of fixing it.

Fixing an AI agent is not coaching a person: the fix must change the content or configuration the agent reads. Our piece on coaching humans vs fixing AI agents covers that split. For causes 6 and 8, the right fix is sometimes a limit, not a better answer. OWASP lists “excessive functionality”, “excessive permissions” and “excessive autonomy” as root causes of damaging agent actions (OWASP LLM06), which is where our guides on AI agent guardrails and AI agent autonomy levels pick up.

How do you run a weekly AI failure triage?

Run AI failure triage as a 45 to 60 minute weekly session: pull the failed conversations, group them by intent, classify the top clusters with the decision tree, route each cause to its owner with examples, and check whether last week’s fixes held. Classify clusters, not single conversations, so the session scales with volume.

A routine that works:

1. Monday: pull the failures. Take every AI conversation that failed a critical criterion last week (accuracy, policy, action, escalation). If you score only a sample, add every conversation with a reopen, a complaint or a repeat contact within 7 days.
2. Group by intent. Refunds, address changes, login problems. Start with the top five intents by failure count.
3. Classify three to five conversations per cluster. Run the ten questions. If they agree, tag the cluster. If they split, the cluster is two problems: split it.
4. Route. One ticket per cause per intent, to a named owner, with the examples and a re-score date.
5. Check last week. For every fix shipped, compare the failure count on that intent before and after. A fix that did not move the count was the wrong fix or the wrong cause.
6. Log it. One row per cluster: week, intent, cause, count, owner, fix, result. This log is your failure mix over time.

The QA lead runs it with the AI owner, the knowledge owner and a support ops lead. Policy, engineering and product join only when their item is on the list. Handoff failures often show only across conversations, which is where customer journey QA helps.

How do you measure the AI failure mix over time?

Measure the failure mix as failures per 1,000 AI conversations, split by cause, every week, next to each cause’s share. Counts per 1,000 show whether a cause is shrinking. Shares show where the remaining work sits. Read both, because fixing one cause makes the others look bigger as a share even when nothing changed.

Pylon saw this in its own data: as its knowledge gaps fell, bug reports went “up from 8.5% of escalations to 19%” (Pylon). A rising share does not by itself mean more bugs: the other slices got smaller.

A worked example (illustration, numbers invented to show the math)

Take an AI agent handling 12,000 conversations a week. In week 1, 480 fail a critical criterion: 40 per 1,000. The team fixes the top knowledge and policy clusters over four weeks. In week 5, volume is the same and 290 fail: 24.2 per 1,000.

Three things to read from it. The three content causes fell from 260 to 95 failures, from 54.2% to 32.8% of the mix. Hallucinations went from 7.3% to 11.4% of failures while their count fell from 35 to 33: a leader reading only shares would conclude the model got worse. And tool failures barely moved (40 to 38), which says engineering has not shipped its fix yet.

The metrics to keep on the dashboard:

• Failures per 1,000 AI conversations, total and by cause.
• Share by cause, to decide where the next week’s effort goes.
• Recurrence: failures in an intent and cause that was marked fixed. Above zero after two weeks means the fix did not hold.
• Time to fix by owner, from routed ticket to re-score.
• Unclassified rate: failures nobody could place. If it rises above a small share, the taxonomy or the criteria need work.

When is this taxonomy not for you?

A ten-cause taxonomy is not for you, or not yet, when your AI agent answers a few hundred questions a month from a small help center and takes no actions. Then three buckets will do: content (causes 1 to 3), AI agent (4, 5, 8, 10) and everything else. Move to the full list when owners outside support start receiving the work.

Signs you have outgrown three buckets: the agent takes actions in other systems, you have more than one AI agent or platform answering customers, policies change more than once a quarter, or two teams disagree every week about whose problem a failure is. If the same intent keeps failing after fixes, run a deeper root cause analysis on that one cluster rather than adding categories.

Where Kaizo fits

Kaizo is QA and governance oversight for support teams where people and AI agents share the work. It covers the evidence side of failure analysis: scoring conversations, surfacing root causes and re-scoring after a fix. It does not write your articles or policies.

AutoPilot scores every resolved conversation against your own scorecard, and on the QA for AI agents side you add the criteria where an AI agent fails and connect your knowledge base so it judges against your content. Every score links to the evidence in the transcript. Insights turns those scores into root causes (the broken process, the product friction, the skill gap), which gives the weekly triage its starting clusters and examples.

See it on your own conversations. Book a demo and bring one intent your team calls a hallucination problem.

How to start this week

Start with last week’s worst AI conversations, not with a new tool. Add the ten causes to your review form, classify a small batch with the decision tree, name an owner per cause and fix the largest content cluster first.

1. Paste the ten-cause table into your review form as a required pick list on every failed criterion.
2. Take last week’s 30 worst AI conversations and run the ten questions on each. Count how many reach question 9.
3. Name one owner per cause, as people, and agree the routing note format.
4. Book a recurring 45-minute triage slot and start the log with this week’s clusters.
5. Pick the largest content cluster and fix it first. Re-score the intent next week. The full policy side of the loop is in our AI agent policy management guide.

Frequently asked questions

Is every wrong answer from an AI agent a hallucination?

No. A hallucination is content the model generates that is not supported by its sources or the conversation. A wrong answer copied from an outdated article, or a guess on a case no policy covers, is a content failure. The test is whether a correct, approved source existed and the agent departed from it.

Does grounding an AI agent on your own content stop hallucinations?

Grounding reduces invented answers, but it does not remove them, and it adds new failure points. The agent can retrieve the wrong article, quote a stale one, or still depart from the right one. That is why retrieval and outdated knowledge get their own causes in the taxonomy, separate from hallucination.

Who should own AI agent failure analysis?

The QA lead usually owns the classification and the weekly triage, because the scorecard and the review evidence sit with QA. The fixes belong to the owners of each cause: knowledge, policy, AI configuration, engineering, support ops and product. One person owning the classification keeps the tags consistent week to week.

How many failed AI conversations should you classify each week?

Classify clusters, not every conversation. Three to five conversations per cluster, across the top five intents by failure count, is 15 to 25 reads a week and usually covers most of the failures. Read more in a cluster when the conversations disagree on the cause.

Should feature requests count as AI agent failures?

Count them, but report them apart from the agent’s own failures. A feature request fails the customer, not the agent, and no content or prompt change fixes it. Counting them shows product how often customers ask, and keeps them from inflating the AI agent’s error rate.