Chatbot to human handoff QA scores the moment a conversation passes between an AI agent or chatbot and a human, in either direction. It is where blended support breaks most often, and per-agent scores miss it because the failure belongs to the transition. Measure it with a few handoff metrics and a scorecard that judges both sides.
In short
- AI-to-human and human-to-AI handoffs fail in different ways.
- AI-to-human fails on late escalation, lost context, an ignored request for a person, or no usable reason.
- Human-to-AI fails when an unresolved case goes back to automation.
- Containment and handle time cannot see it. Measure handoff rate by intent, late handoffs, context carried, repeat questions and resolution after handoff.
- Score both sides of the seam: the bot’s decision to hand off and the human’s pickup.
Why the handoff is its own failure point
When you QA a human agent and you QA an AI agent, you are scoring two halves of a conversation that increasingly is not two halves. In a blended model a customer might start with an AI agent, get escalated to a person, and be routed back to automation for a follow-up, all in one thread. Each participant can score well on their own stretch while the conversation as a whole is a bad experience, because the damage happened at the transition and belongs to neither of them.
That is the blind spot. A per-agent scorecard evaluates what each party did with the conversation while they held it. It has nothing to say about what happened at the moment they passed it on, which is exactly where blended support tends to break. The handoff needs to be scored as a thing in itself, and it is one step of the wider practice of customer journey QA, which scores the whole path a customer takes across bots and people.
What goes wrong in a chatbot to human handoff?
Most bad chatbot to human handoffs come down to a few failures: the bot escalates too late, context is lost so the customer repeats themselves, a request for a human is ignored, the customer loops between bot and queue, the transfer is cold, or the promised person never arrives.
Regulators have noticed. The US Consumer Financial Protection Bureau’s report on chatbots in consumer finance (June 2023) describes “doom loops” when an issue falls outside what the chatbot can do, quotes one complaint (“I am being sent in an endless loop with no way out”) and warns that chatbots which prevent access to live, human support can lead to law violations.
| Failure | What the customer experiences | What it looks like in the transcript | Metric that catches it |
|---|---|---|---|
| Escalating too late | Several rounds of the same unhelpful answer before a person appears | The bot repeats or rephrases the same answer 3 or more times, or the customer asks twice for help | Late handoff rate |
| Lost context | “Can I have your order number?” from the human, after giving it to the bot | The human asks for something already in the thread | Context-carried rate, repeat-question rate |
| “Talk to a human” ignored | The bot answers a different question, or offers the FAQ again | An explicit request for a person with no handoff on that turn | Ignored human request rate |
| Handoff loops | Sent to a queue, then back to the bot, then to a queue again | More than one handoff for the same issue, or the bot re-takes an unresolved case | Handoff loop rate |
| Cold transfer | Silence, then a new name, with no idea why or how long to wait | No message telling the customer a person is taking over, why, and when | Cold transfer rate, time to human |
| Dead end | “Connecting you to an agent” outside opening hours, then nothing | A handoff with no human reply and no fallback offered | Abandonment during handoff |
The dead end is the quiet one. Zendesk’s guide to escalation flows for AI agents puts an availability check before a messaging escalation, with email as the fallback outside working hours. Without that check, the handoff promises a person who never arrives.
AI-to-human: what to score when the agent escalates
When an AI agent escalates to a person, score three things: timing (did it hand off when it stopped making progress), context (did the human get the history and diagnosis) and reason (can the human act on it). Each failure mode deserves its own criterion.
Timing: did it escalate at the right moment
Too late is the common one. An agent that keeps trying after it is clearly stuck runs the customer in circles before handing over a frustrated person. Too early wastes the automation and the human’s time. The criterion is whether the escalation happened at the point the agent stopped making progress, which you can see in the transcript.
Context: did the human get what they need
The worst handoff makes the customer start over. Score whether the escalation carried the history, the diagnosis so far, and what the customer actually wants, so the human opens the case informed rather than cold. A handoff that drops context turns one conversation into two.
Reason: was the escalation intelligible
An escalation tagged only escalated to human tells the human nothing. Score whether the reason for the handoff was captured in a form the receiving person can act on. This is one of the criteria on an AI agent scorecard, but it only becomes visible when you read across the seam.
How should you design a chatbot to human handoff?
Design the handoff around eight rules: honor a request for a person on the first ask, escalate on no progress, send red-list topics straight to a human, check availability, pass a context package, tell the customer what happens next, never bounce an unresolved case back, and record the reason.
- Honor “talk to a human” on the first ask. Intercom’s escalation guidance documentation lists three default triggers for Fin: the customer clearly asks for a human, strong frustration or anger, and a customer stuck in a repetitive loop. Your own guidance and rules can override that default behavior, so check that yours did not switch one off.
- Escalate on no progress. Write a rule you can check in a transcript, such as the same answer given twice.
- Keep a red list. Some intents go to a person every time: disputes, legal complaints, vulnerable customers, anything outside the actions the bot may take alone (an autonomy level decision).
- Check availability. Outside opening hours, offer a callback, email or ticket and say when a human will reply.
- Pass a context package. At minimum: a short summary, the customer’s goal, the steps already tried, verified identity, and the reason for the handoff. Microsoft’s Copilot Studio documentation on handing off to a live agent says the agent can share the full conversation history and all relevant variables, plus an optional private message to the live agent. Your QA checks it arrived and was useful.
- Make it warm. Tell the customer a person is taking over, why, and roughly how long the wait is.
- No bounce-back. The bot does not re-take a conversation a human has not resolved; a second handoff on the same issue goes to a senior queue.
- Log the reason as a field. “Requested human”, “out of scope”, “policy”, “frustration”, “failed action”. Without it you cannot compute handoff rate by reason.
How do you measure AI-to-human handoff quality?
Measure AI-to-human handoff quality per intent: handoff rate, late handoffs, ignored human requests, context carried, repeat questions after handoff, time to human, handoff loops and resolution after handoff. Volume metrics come from your helpdesk; quality metrics need the transcript scored.
The thresholds below are starting points to tune on your own data, not industry benchmarks.
| Metric | Definition | Data source | Starting alert |
|---|---|---|---|
| Handoff rate by intent | Handoffs to a human ÷ AI conversations, per intent | Helpdesk handoff event plus intent tag | Moves more than 5 points week over week on one intent |
| Late handoff rate | Handoffs that came after the escalation point (customer asked twice, or the bot repeated itself 3 or more times) ÷ all handoffs | Transcript scoring | Above 10% |
| Ignored human request rate | Explicit requests for a person not acted on at the first ask ÷ explicit requests | Transcript scoring | Above 2% |
| Context-carried rate | Handoffs where the summary, goal, steps tried and verified identity reached the human ÷ handoffs | Handoff payload or ticket fields, checked in the transcript | Below 90% |
| Repeat-question rate after handoff | Handoffs where the human asked for information already given ÷ handoffs | Transcript scoring | Above 10% |
| Cold transfer rate | Handoffs with no message telling the customer who takes over and when ÷ handoffs | Transcript scoring | Above 5% |
| Time to human | Time from the handoff to the first human reply, as median and 90th percentile | Helpdesk timestamps | 90th percentile above your chat service level |
| Abandonment during handoff | Customers who leave before the first human reply ÷ handoffs | Helpdesk timestamps | Rising for 2 weeks |
| Handoff loop rate | Conversations with more than one handoff for the same issue, or a return to the bot while unresolved ÷ handoffs | Conversation log plus scoring | Above 3% |
| Resolution after handoff | Handoffs resolved with no repeat contact on the same issue within 7 days ÷ handoffs | Helpdesk plus contact history | Falling for 2 weeks |
Two of these split the blame cleanly. When context did not arrive, the fix is the bot or the integration. When context arrived and the human asked anyway, the fix is coaching (see coaching humans versus fixing AI agents). For dashboards and alerts, see AI agent monitoring; for the wider metric set, AI customer service metrics.
The handoff scorecard: criteria for the bot and the human
Score a handoff on two blocks: the bot’s decision (trigger, timing, request honored, context package, customer told, availability) and the human’s pickup (read before replying, no re-ask, picked up in time, owned to resolution). Each criterion is pass or fail, with evidence a second reviewer can point at in the transcript. Copy it into your scorecard as a section that applies only to conversations with a handoff.
| Side | Criterion | Pass | Fail example | Data source |
|---|---|---|---|---|
| Bot | Right trigger | Escalated on a rule you wrote (request, no progress, red-list intent), and never stayed with the bot when a rule was met | Kept a red-list dispute, or escalated a routine order status question it could answer | Transcript plus handoff reason field |
| Bot | Right moment | Handed off at the first point the rule was met | Customer asked for help twice and got the same article three times | Transcript |
| Bot | Request honored | “Talk to a human” acted on at the first ask | “I can help with that!” followed by the FAQ again | Transcript |
| Bot | Context package | Summary, goal, steps tried, verified identity and reason reached the human | Ticket shows only “escalated to agent” | Ticket fields or handoff payload |
| Bot | Customer told | Customer knew a person was taking over, why, and the expected wait | Silence for 9 minutes, then “Hi, how can I help?” | Transcript |
| Bot | Availability checked | Live handoff only when someone is online, otherwise a fallback with a reply time | “Connecting you now” at 2 a.m. with no reply until morning | Transcript plus timestamps |
| Human | Read before replying | First reply refers to what the customer already said | “Hi, what seems to be the problem?” | Transcript |
| Human | No re-ask | No request for information already in the thread | Asks for the order number the bot already verified | Transcript |
| Human | Picked up in time | First human reply within your chat service level | Customer waited past the target and left | Helpdesk timestamps |
| Human | Owned it | Resolved, or set a clear next step, without sending it back to the bot unresolved | Closed with “the assistant can help you with the rest” | Transcript plus contact history |
| Human | Flagged a bad handoff | Tagged the handoff when the context was missing or wrong | Worked around a broken summary without telling anyone | Ticket tag |
Two scoring rules keep it honest. Treat “request honored” and “owned it” (no bounce-back) as auto-fail criteria: a customer who asked for a person and did not get one is a failed conversation however polite the bot was. And attribute each failed criterion to its side, so the bot block feeds the AI agent owner and the human block feeds coaching. For the escalated ticket once a person holds it, see scoring an escalated ticket.
A worked example: what handoff metrics show that samples miss
Illustration: the numbers are invented to show the math. A team runs 20,000 AI agent conversations a month; 3,000 end in a handoff, a 15% handoff rate. They score every handoff with the scorecard above.
- Late handoffs: 480 of 3,000, or 16%, mostly in refunds and delivery disputes.
- Context carried: 2,550 of 3,000, or 85%. 450 handoffs reached the human without the summary.
- Repeat questions: 690 handoffs, or 23%: 400 where the context never arrived, 290 where it arrived and the human asked anyway.
- Ignored requests: 900 customers asked for a person; 90 of them, or 10%, were not handed off on the first ask.
- Resolution after handoff: 2,340 of 3,000, or 78%, with no repeat contact within 7 days.
The split tells the team where to act. The 400 re-asks with no context are an integration fix. The 290 with context present are a coaching topic. The 90 ignored requests go to the owner of the bot’s escalation rules, this week.
Now compare a 2% random sample: 400 conversations, about 60 with a handoff (15%), roughly 10 late handoffs and about 2 ignored requests (90 of 20,000 is 0.45%; 0.45% of 400 is 1.8). Two cases read as an anecdote. Ninety, grouped by intent, read as a broken rule.
See it on your own conversations. Kaizo scores 100% of your support conversations and turns the findings into coaching. Book a demo.
Human-to-AI: the direction nobody scores
A human-to-AI handoff fails when a person passes a case back to automation before it is resolved, delegates a task the AI cannot pick up, or triggers a follow-up that contradicts what the human just did. The reverse handoff is newer, less examined and easy to miss.
| Human-to-AI handoff | What good looks like | The failure to score for |
|---|---|---|
| Returning a case to automation for follow-up | The AI has the resolution and the next step is genuinely routine | The human hands back an unresolved case to clear their queue |
| Delegating a sub-task mid-conversation | The AI can complete the specific task and hand back cleanly | The AI cannot pick up the thread and the customer loses continuity |
| Post-resolution automated follow-up | The follow-up matches what was actually resolved | The automation contradicts or forgets what the human just did |
When AI agents also hand off to each other, the same seams multiply; our guide to AI agent orchestration covers agent-to-agent handoffs.
How to score a handoff in practice
To score a handoff in practice, evaluate the whole thread across the transition, cover every handoff rather than a sample, and attribute each failure to the side that caused it.
Evaluate the whole thread
Score continuity across the seam: did context survive, did the customer avoid repeating themselves, did the second party pick up where the first left off. This is impossible if your QA looks at the AI portion and the human portion as separate records, which is how most tooling stores them.
Cover every handoff, because they are rare per pair but common overall
Any single agent pair may hand off infrequently, so a sample per agent misses handoffs almost by construction. Across the operation they are everywhere. Scoring 100% of conversations is what makes handoff failures visible as a pattern rather than as the occasional complaint. At UiPath, Kaizo automated 100% of QA with 200% ROI and an 8% lift in quality score.
Attribute each failure to one side
Was it the AI’s fault for escalating badly, or the human’s for handing back a mess? Score the bot block and the human block as separate criteria, and point every finding at the exact turns where continuity broke, so you fix the routing rule, the context package or the coaching topic. The full workflow is in how to QA AI agents and chatbots.
When handoff scoring is not for you
Skip a dedicated handoff scorecard when your bot hands off a few dozen conversations a month: read every one by hand instead. It is also premature when the bot has no written escalation rules, because “right trigger” and “right moment” need a rule to score against, or when the handoff is a link to a contact form: there is no seam until the bot and the human share a conversation.
How to start measuring handoff quality this week
Start with a manual pass: count last month’s handoffs by intent, read 30 of them end to end, tag each against the scorecard above, and split the re-asks by cause. A few hours give you a baseline before you automate anything.
- Pull last month’s handoffs from your helpdesk and count them by intent.
- Read 30 handoff conversations end to end and tag each against the scorecard above.
- Split the re-asks into “context missing” and “context present”. Send the first list to the bot owner and the second to your team leads.
- Write down your escalation rules and add a reason field to the handoff.
When the manual pass proves the pattern, automate it. Kaizo’s AI agent QA scores AI agent conversations on the same scorecard as your team’s and checks the handoff as criteria on it: did the AI agent escalate on the right trigger, pass on the context, and did the human agent pick it up. AutoPilot scores every conversation the moment it resolves. To score the whole path, see customer journey QA.
Frequently asked questions
What is handoff QA?
Handoff QA is evaluating the moment a conversation passes between an AI agent and a human, in either direction. It treats the transition as its own unit of evaluation, because a handoff can fail (lost context, late escalation, an unresolved case handed back) even when both the human and the AI score well on their own segments.
What should you score in an AI-to-human escalation?
Three things: timing (did it escalate at the point it stopped making progress, rather than too late or too early), context (did the human receive the history and diagnosis so the customer does not start over), and reason (was the escalation captured in a form the receiving person can act on).
What goes wrong in a human-to-AI handoff?
The main failures are a human handing an unresolved case back to automation to clear their queue, delegating to an AI that cannot pick up the thread and so breaks continuity, and post-resolution automated follow-ups that contradict or forget what the human just did. This direction is newer and rarely scored.
Why do per-agent QA scores miss handoff problems?
Because the failure lives between the records. A per-agent scorecard evaluates what each party did while holding the conversation, not what happened at the transition. Scoring the handoff requires reading the whole thread across the seam, which is why it needs the conversation stored and evaluated as one unit rather than two.
What is the difference between a warm and a cold handoff?
In a warm handoff the customer is told a person is taking over, why, and how long the wait is, and the human receives a summary before replying. In a cold handoff the conversation lands in a queue: the customer waits without knowing what happens, and the human starts from nothing.
What is a good chatbot to human handoff rate?
There is no single good rate, because it depends on the intent: 40% can be right for billing disputes and alarming for password resets. Track the rate per intent, watch for changes, and judge it together with late handoffs and resolution after handoff.






