Correct resolution rate is the share of conversations an AI agent (or a human agent) handled that ended resolved, with the right answer, the right action completed and policy followed, and with no repeat contact about the same issue within 7 days. The formula is correct resolutions divided by conversations handled, times 100, estimated from an audited random sample.
In short
- Correct resolution rate = correct resolutions / conversations handled x 100, where “correct” means four checks pass: right answer, allowed by policy, action done in the system of record, no repeat contact within 7 days.
- You estimate it from a random audited sample, so report it with a 95% margin of error: 1.96 x √(p x (1 − p) / n).
- About 385 judged conversations give a margin of roughly ±5 points; about 97 per intent give roughly ±10 points, enough to find a broken intent.
- It is stricter than containment, deflection, first contact resolution and vendor resolution rate, because none of those checks whether the answer was right or the action happened.
- Divide what the AI agent costs by correct resolutions, not billed resolutions, to get the cost per correct resolution.
This page defines the metric and gives you the calculator. The method for judging each conversation (blind review, backend checks, the verdict rule) is in our guide to auditing AI agent resolutions, and both sit under our hub on AI agent evaluation.
What is correct resolution rate?
Correct resolution rate is the percentage of in-scope conversations that an agent, AI or human, closed without a handoff and that pass four checks: the answer was right and complete, it stayed within policy, every promised action is visible in the system of record, and the customer did not come back about the same issue, on any channel, within 7 days.
Each part of the definition does a job:
| Part | Rule | Why it is there |
|---|---|---|
| Handled | Every conversation that entered the agent in the period, minus spam, tests and greeting-only chats | Fixes the denominator, so the rate cannot be raised by excluding hard conversations |
| Resolved | The agent closed it without handing off to a human | A correct handoff is a good outcome, but it is not a resolution by this agent; track it separately |
| Right answer | Matches the policy, price or procedure valid on the conversation date | Catches confident wrong answers the customer accepted |
| Allowed | No exception or promise outside the agent’s written scope | Catches answers that were “helpful” but against policy |
| Action done | Refund, cancellation or update is in the system of record, right amount and item | Catches actions the agent claimed but never completed |
| No repeat contact | No contact from the same customer about the same issue within 7 days, any channel | Catches customers who gave up or came back elsewhere |
One failed check makes the conversation “not correct”. A conversation where a record is missing is “cannot judge” and leaves the sample, the same verdict rule the resolution audit uses.
What is the correct resolution rate formula?
The formula is: correct resolution rate = (correct resolutions / conversations handled) x 100. Because you cannot judge every conversation by hand, you estimate it from a sample: correct share of the judged sample times 100. If the sample only covers vendor-resolved conversations, multiply that share by the vendor’s resolved share.
Two ways to estimate it, depending on what you sample:
- Sample all handled conversations (direct method). Draw at random from everything the agent handled, escalations included. Correct resolution rate = correct / judged. Escalated conversations simply count as “not correct”.
- Sample only vendor-resolved conversations (two-stage method). This is what the resolution audit does. Correct resolution rate = vendor resolved share x correct share of the audited resolved sample. Example: 62% resolved x 70% correct = 43.4%.
The two-stage method is cheaper if you already audit resolutions, but assumes no conversation marked unresolved was in fact a correct resolution. The calculator below uses the direct method.
How do you estimate correct resolution rate from an audited sample?
Judge a random sample, compute the correct share p, then add a 95% margin of error of 1.96 x √(p x (1 − p) / n), where n is the number of judged conversations. With p = 0.48 and n = 385, the margin is about ±5 points, so the true rate lies between roughly 43% and 53%.
This is the most frequently used interval for a proportion, as the NIST engineering statistics handbook notes. The same page prefers the Wilson interval, because the simple formula can produce impossible limits (a lower limit below 0%) while Wilson’s cannot. Use Wilson for any intent with fewer than about 50 judged conversations or a rate near 0% or 100%.
To size the sample before you start, use n = 1.96² x 0.25 / E², where E is the margin you want and 0.25 is the worst case (p = 0.5):
| Margin you need (E) | Judged conversations (n) | Spreadsheet formula |
|---|---|---|
| ±10 points | 97 | =ROUNDUP(1.96^2*0.25/0.10^2,0) |
| ±5 points | 385 | =ROUNDUP(1.96^2*0.25/0.05^2,0) |
| ±3 points | 1,068 | =ROUNDUP(1.96^2*0.25/0.03^2,0) |
Add 3% to 5% for “cannot judge” verdicts. Our page on QA sampling statistics explains why the margin shrinks with the square root of the sample, not in proportion to it.
The correct resolution rate calculator (copy into a spreadsheet)
The calculator has one row per intent, so you can oversample risky intents and still get an honest overall rate. Put the column letters below in row 1, intents from row 2, and copy the formulas down. The numbers are an illustration, not customer data.
Columns and formulas (row 2 shown; row 6 is the total):
| Column | Field | Formula (row 2) |
|---|---|---|
| A | Intent | typed |
| B | Conversations handled (N) | typed, from the agent’s full log |
| C | Sampled at random (n) | typed |
| D | Cannot judge | typed, from the audit |
| E | Correct resolutions in sample | typed, from the audit |
| F | Judged | =C2-D2 |
| G | Correct share (p) | =E2/F2 |
| H | Margin, 95% | =1.96*SQRT(G2*(1-G2)/F2) (in Excel also =CONFIDENCE.NORM(0.05,SQRT(G2*(1-G2)),F2)) |
| I | Lower limit | =MAX(0,G2-H2) |
| J | Upper limit | =MIN(1,G2+H2) |
| K | Estimated correct resolutions | =B2*G2 |
| L | Variance share | =(B2/$B$6)^2*G2*(1-G2)/F2 |
| Total row 6 | Correct resolution rate | =SUM(K2:K5)/B6, margin =1.96*SQRT(SUM(L2:L5)) |
The Excel shortcut in column H works because CONFIDENCE.NORM returns the margin for a given alpha, standard deviation and sample size. Column L weights each intent by its share of volume, which is what makes the overall margin correct when you oversample.
Worked example (illustration). An AI agent handles 20,000 conversations in October. The vendor reports a 62% resolution rate. The team samples 440 conversations across four intents, oversampling cancellations, and judges each with the four checks.
| Intent | Handled (N) | Sampled | Cannot judge | Correct | Correct share | Margin | Range | Est. correct |
|---|---|---|---|---|---|---|---|---|
| Order status | 9,000 | 120 | 2 | 73 | 61.9% | ±8.8 | 53.1% to 70.6% | 5,568 |
| Returns and refunds | 5,000 | 120 | 4 | 46 | 39.7% | ±8.9 | 30.8% to 48.6% | 1,983 |
| Cancel plan | 2,000 | 100 | 3 | 29 | 29.9% | ±9.1 | 20.8% to 39.0% | 598 |
| Billing question | 4,000 | 100 | 1 | 35 | 35.4% | ±9.4 | 25.9% to 44.8% | 1,414 |
| Total | 20,000 | 440 | 10 | 183 | 9,563 |
The result, step by step:
- Overall correct resolution rate: 9,563 / 20,000 = 47.8%.
- Overall margin: 1.96 x √(sum of column L) = ±5.0 points, so 42.8% to 52.8%.
- Gap to the vendor number: 62% reported vs 47.8% correct, a gap of about 14 points.
- Why not 183 / 430? That unweighted 42.6% would over-count cancellations, which were sampled at 5% of their volume while order status was sampled at 1.3%. Weighting by N fixes it.
The intent rows are the to-do list: cancellations and billing questions fail most, so they get the first fixes, or move back to humans until they improve.
How is correct resolution rate different from containment, deflection, FCR and resolution rate?
Containment, deflection, first contact resolution and vendor resolution rate all count conversations that ended a certain way. Correct resolution rate counts conversations that ended well: it is the only one of the five that requires the answer to be right, the action to be done and the issue to stay solved for 7 days.
| Metric | Counts | Checks correctness? | Repeat contact window | Who defines it |
|---|---|---|---|---|
| Containment rate | Conversations that ended without a human | No | None | You or the vendor |
| Deflection rate | Requests that never reached a human queue | No | None | Varies by team; see why deflection rate hides bad conversations |
| First contact resolution (FCR) | Issues solved in the first interaction | Often by survey, not by record | Set by you | You |
| Vendor resolution rate | Conversations the vendor marks resolved | Sometimes, by reading the transcript | Short and channel-bound | The vendor |
| Correct resolution rate | Resolved, right, allowed, action done, no repeat | Yes, against policy and backend records | 7 days, any channel | You |
Vendor definitions differ. Intercom counts an “assumed resolution” when the customer exits “without requesting further assistance” and removes it only if the customer returns to the same conversation (Intercom). Zendesk separates “contained” from “verified” resolutions, where a language model reviews the transcript (Zendesk). Ada counts resolutions judged relevant, accurate and safe by an LLM review (Ada). None of these, as documented, looks for repeat contacts on other channels, and a review of the transcript alone cannot confirm that a refund or cancellation actually posted, which is why you need a number you define yourself.
FCR is the closest cousin: correct resolution rate is FCR with the correctness and action checks made explicit, so it works the same way for an AI agent and a person.
Why should you segment correct resolution rate by intent?
Segment by intent because one blended rate hides where the agent fails, and because the intent mix decides the headline number. An agent that gets more simple order-status questions this month will show a higher rate without getting any better. Per-intent rates show real change and tell you which intents to fix, expand or take back.
Three rules keep the comparison fair:
- Compare like with like. When you compare months, vendors or a human team with the AI agent, compare intent by intent, or reweight both to the same intent mix.
- Size each intent for a decision. About 97 judged conversations per intent give ±10 points: enough to see that cancellations at 30% are broken, not enough to see a 3-point move.
- Watch the risky intents first. Intents where the agent moves money or changes an account deserve more sample, because a wrong action there costs more than a wrong answer about delivery times.
Picking which intents the agent should own in the first place is the subject of which intents to automate first.
How do you calculate cost per correct resolution?
Cost per correct resolution = total cost of running the AI agent in the period / estimated correct resolutions. Use everything you pay to run it (vendor bill, platform fees, internal time for knowledge upkeep and QA), and divide by correct resolutions, not by the resolutions the vendor billed you for.
Continuing the illustration, with an illustrative price of $0.99 per billed resolution:
| Line | Formula | Value |
|---|---|---|
| Billed resolutions | 20,000 x 62% | 12,400 |
| Vendor bill | 12,400 x $0.99 | $12,276 |
| Internal running cost (illustrative) | knowledge upkeep, QA time | $3,000 |
| Total cost | sum | $15,276 |
| Cost per billed resolution | $15,276 / 12,400 | $1.23 |
| Cost per correct resolution | $15,276 / 9,563 | $1.60 (range $1.45 to $1.78) |
The range divides by the lower and upper estimate of correct resolutions (8,565 and 10,560). For a fuller picture, add the human cost of the repeat contacts that false resolutions create, and compute the same figure for your human team (loaded cost of the hours spent / correct resolutions) before you decide what to automate. Our wider set of AI customer service metrics shows where this figure sits next to escalation quality and silent abandons.
When is correct resolution rate not worth the effort?
Skip the sampling when the agent handles a few hundred conversations a month: judge them all and compute the exact rate. The metric also matters less for an agent that only answers simple informational questions with no actions; a repeat-contact count and a light accuracy check may be enough there.
It is also not a way to find individual failures. A sample of 440 gives the rate within about 5 points but will rarely surface a failure that hits 1 customer in 500; finding those needs every conversation scored.
How to start measuring correct resolution rate this week
You need a spreadsheet, one or two reviewers and read-only access to your order or billing system. The first week gives you a first rate with a margin and a list of the intents to fix; the definitions matter more than the decimals.
- Day 1: write down the four checks with one pass and one fail example each, and the 7-day rule.
- Day 2: export last month’s handled conversations with intent labels, and draw a random sample per intent using the size table above.
- Days 3 and 4: judge each conversation, check claimed actions against backend records, and look for repeat contacts on every channel.
- Day 5: paste the counts into the calculator, read the per-intent rates and compute the cost per correct resolution.
Once you know which intents fail, the next step is catching those failures on every conversation instead of a monthly sample. Kaizo’s AutoPilot scores every resolved conversation against your own scorecard, and every score links to the evidence in the conversation; see QA for AI agents for how it works, or book a demo to see it on your own AI agent’s conversations.
Frequently asked questions
What is a good correct resolution rate?
There is no published benchmark, because almost no one defines the metric the same way yet. Track your own rate per intent month over month, and watch the gap to the vendor’s resolution rate. A shrinking gap means fixes are working; a stable high vendor rate with a falling correct rate means the dashboard is drifting from reality.
Can you use correct resolution rate for human agents?
Yes. The four checks apply to any conversation, and the formula is the same. For people, “resolved” means the agent closed the conversation without transferring it. Compare humans and the AI agent intent by intent, never on blended rates, because they usually handle very different mixes.
Why use a 7-day repeat contact window?
Vendor windows are short and usually limited to the same conversation, while many failures show up later: a refund that never arrives, a second charge at the next billing run. Seven days catches most of them. For billing intents, extend the window to the next invoice date.
Can an LLM judge calculate correct resolution rate?
An LLM judge can score the answer and policy checks at volume, if you calibrate it against human verdicts. It cannot see backend records or contacts on other channels by reading the transcript alone, so join it with order data and contact history before you call a conversation correct.
Do “cannot judge” conversations count against the rate?
No. Leave them out of the judged count, report how many there were, and fix the missing data if they pass about 5% of the sample. If they cluster in one intent, that intent’s rate is less reliable than its margin suggests.
