Skip to content

Best practice

Customer Service Quality Assurance: How to Run a QA Program

Customer service quality assurance for support leaders: build a scorecard, calibrate reviewers, coach from scores and cover every conversation with AI.

· Updated · 19 min read

Reviewed by Dominik Blattner, founder of Kaizo

Part of: How to Build a Customer Service QA Program: A 7 Step Playbook

Customer service quality assurance (QA), also called customer care quality assurance, is the process of scoring support conversations against a defined standard and using the results to coach agents. Running a QA program means writing a scorecard, calibrating the people who score, coaching from the results and reporting on trends. Manual review reaches a small sample, so modern programs use AI to score every conversation across chat, email, tickets and calls.

In short

  • Customer service QA scores conversations against your scorecard, then coaches agents from the results.
  • Manual review reaches a small sample of conversations, so the score describes a sample, not the team.
  • A working QA program has five parts: a scorecard, monitoring, calibration, coaching and reporting.
  • AI scoring makes full coverage practical on chat, email and tickets, with humans kept for calibration and coaching.

What is customer service quality assurance?

Customer service quality assurance (QA) is the process of evaluating support conversations against a defined quality standard, then using the results to coach agents and improve service. It checks whether agents resolve issues, follow process, communicate clearly and deliver a consistent experience across chat, email, tickets and messaging.

In practice, a QA analyst or team lead scores a sample of conversations each week against a rubric, and those scores feed coaching, compliance and performance reviews. QA is how a support team keeps quality high as it grows. Ongoing review of conversations is also called quality monitoring.

The problem is that manual scoring only ever reaches a fraction of conversations. That gap is exactly what AI-powered QA and modern customer service QA software are built to close.

Customer service QA vs call center QA

Customer service QA and call center QA share the same method: define quality, score conversations, coach from the results. Call center QA grew up in voice, where reviewers listened to a sample of recorded calls. Customer service QA is broader and increasingly digital-first, covering chat, email, tickets and messaging as much as calls.

What differs is the channel mix. In an August 2025 Gartner survey of 265 service leaders, self-service and live chat were on track to overtake phone and email as the most valuable service technologies within two years. A QA approach built only for voice leaves a growing share of the work unmeasured.

Many contact centers still review calls in one tool and chat in a spreadsheet. The better practice is one scorecard for every channel the team handles, so quality is judged against the same definition and the results are comparable. This guide takes the digital-first view. For the voice side, see our guide to call center quality assurance.

Agent analysis across call center and digital channels

Why does customer service quality assurance matter?

Customer service quality assurance matters because it is the only systematic way to know whether customers get the standard you promise. It catches poor interactions before they cause churn and links service quality to CSAT and retention.

  • Protects the customer experience by catching poor interactions before they cause churn.
  • Drives consistency so every customer gets the same standard regardless of agent or channel.
  • Powers coaching by turning real conversations into specific development for each agent.
  • Ensures compliance where a missed step carries real risk.
  • Links service to outcomes by connecting quality scores to CSAT and retention.

The stakes are not abstract. In the PwC 2025 Customer Experience Survey of 5,511 US consumers, 52% said they had stopped using or buying from a brand after a bad experience with its products or services.

Customer service QA challenges in large support teams

Every one of these depends on coverage. When QA sees only a small sample of conversations, the insight rests on evidence that may not be representative, and the senior time spent grading is time not spent coaching. Manual QA does not just limit visibility, it consumes exactly the people you most want coaching.

What a customer service QA program includes

A complete customer service QA program has five parts: a scorecard, a monitoring method, calibration, coaching and reporting.

The scorecard defines a good conversation, the monitoring method (manual, automated or both) decides which conversations get scored, calibration keeps scoring fair, coaching turns scores into improvement, and reporting ties quality to outcomes.

Miss any one and the program stalls. A scorecard without coaching does not change behavior, and monitoring without calibration produces scores no one trusts.

What are the types of customer service QA?

There are four types of customer service QA: manual QA, where a reviewer grades a sample by hand; automated QA, where AI scores every conversation; peer and self review; and customer-driven QA based on CSAT and surveys. Most programs combine them.

  1. Manual QA: a reviewer grades a sample by hand. High context, very low coverage.
  2. Automated QA: AI scores every conversation against the scorecard. Full coverage and consistency.
  3. Peer and self review: agents review their own or colleagues’ conversations to build a shared quality culture.
  4. Customer-driven QA: CSAT, surveys and feedback bring the customer’s own view of quality into the picture.

Automated QA provides the coverage, human review adds calibration and nuance, and customer feedback validates it all against real experience.

How do you build a customer service QA program?

To build a customer service QA program, define quality, turn it into a weighted scorecard, choose how conversations are monitored, calibrate reviewers and the AI, then coach and report.

  1. Define quality for your team. Decide what great looks like across resolution, compliance, communication and efficiency. Involve agents and team leads so the definition reflects real work, not a manager’s assumption.
  2. Build a weighted scorecard. Translate that definition into criteria, weighted by what matters most, and set explicit auto-fail rules for compliance-critical steps.
  3. Choose a monitoring method. Manual alone caps you at a small sample, so most scaling teams automate the scoring and keep humans for calibration and nuance.
  4. Calibrate. Run calibration sessions so reviewers, and the AI, score the same conversation the same way, then reconcile the differences.
  5. Coach and close the loop. Route findings into per-agent coaching and track improvement at 30, 60 and 90 days so scores turn into behavior change.
  6. Report and refine. Watch quality trends, tie them to CSAT, and update the scorecard as products, policies and customer needs evolve.

For a longer walkthrough of each step, see how to build a QA program from scratch. The analysts who run it day to day are described in the QA analyst role.

Customer service QA management strategies

What goes on a customer service QA scorecard?

A customer service QA scorecard lists the criteria a good conversation must meet, grouped into weighted categories such as resolution, compliance, communication and efficiency, with auto-fail rules for compliance-critical steps. A typical weighting looks like this:

CategoryExample criteriaExample weight
ResolutionIssue fully resolved, correct information35%
ComplianceRequired steps and disclosures followed25%
CommunicationClear, empathetic, on-brand tone25%
EfficiencyResolved without unnecessary back-and-forth15%

The scorecard is the heart of the program, because every score, calibration session and coaching conversation refers back to it. A quick QA checklist for any conversation: Was the customer verified? Was the issue resolved? Was the information accurate? Was the tone empathetic? Were next steps clear? Was policy followed? For a deeper build, see our guide to the QA scorecard.

What is a good customer service QA score?

There is no universal pass mark for a customer service QA score, because it depends on how demanding your scorecard is. Most mature programs set a target band that fits their scorecard and treat scores below it as a coaching trigger rather than a verdict on the agent.

Two things matter more than the headline number. The first is auto-fail criteria: a compliance breach should fail the whole conversation. The second is the trend per agent and team over time, especially whether it correlates with CSAT.

Calibration is what makes any score trustworthy. Without it, a “good score” is just one reviewer’s opinion.

Customer service QA best practices and common mistakes

The best customer service QA programs score for coverage rather than small samples, keep the scorecard focused, calibrate regularly, coach rather than police, tie quality to CSAT and apply one standard everywhere.

  • Score for coverage, not just samples. A small sample hides systemic problems.
  • Keep the scorecard focused on what predicts customer outcomes.
  • Calibrate regularly to keep scores fair and trusted.
  • Make QA a coaching engine, not a policing tool. Agents engage with a program built around development; the method is in how to run a QA program agents trust.
  • Tie quality to CSAT to prove the connection to customer outcomes.
  • Standardize across teams, channels and BPOs with one shared standard.

What are the most common customer service QA mistakes?

The most common customer service QA mistakes each erode either the accuracy of the score or the trust agents place in it:

  • Grading a tiny sample and treating it as representative.
  • An overloaded scorecard that dilutes the signal that matters.
  • Skipping calibration, which erodes agent trust.
  • Scoring without coaching, so nothing improves.
  • Using QA to punish rather than develop.
  • Ignoring digital channels and measuring only calls.

Common customer service QA errors in large teams

How do you turn QA scores into coaching?

You turn QA scores into coaching by linking each finding to a scorecard behavior, showing the agent the conversations behind it and tracking whether scores move at 30, 60 and 90 days.

The problem with manual QA is that preparing coaching eats the very time leads should spend coaching. Automated QA flips this. Because every conversation is scored, the system surfaces each agent’s specific strengths and gaps and drafts coaching, so the lead arrives with the evidence already gathered.

The best teams make coaching continuous and tie it to the behaviors the scorecard measures. See how AI coaching automates that loop.

Coaching and talent retention in customer service

Which metrics measure customer service QA?

The core customer service QA metrics are the quality score trend, its correlation with CSAT, resolution and response times watched alongside quality, coaching impact per agent and coverage.

MetricWhat it tells you
Quality (QA) score trendWhether quality is rising across the whole team, ideally on every conversation
QA-to-CSAT correlationWhether your scores reflect what customers actually experience
First contact resolution and response timesWhether speed is coming at the expense of quality
Coaching impactScore movement per agent over 30, 60 and 90 days
CoverageThe share of conversations actually scored, to quote next to every score
Scorecard automation rateThe share scored without human grading, as a maturity measure

See our guide to customer service metrics and KPIs for the full set.

How does customer service QA improve CSAT and other core metrics?

Customer service QA improves core metrics by showing which behaviors drive them and focusing coaching there. Because scores tie back to CSAT, you can prove the link between quality work and customer outcomes rather than assert it.

  • CSAT. Scoring every conversation reveals the behaviors that drive satisfaction, so coaching targets what raises the score.
  • First contact resolution. Full coverage exposes resolution gaps a small sample misses.
  • Agent ramp time. New agents are coached from real conversations immediately.
  • Retention. Fair, development-focused QA raises agent engagement and lowers churn.

Sampling vs 100% coverage in customer service QA

Sampling means a reviewer grades a small, hand-picked share of conversations; 100% coverage means automated QA scores every conversation in scope. Sampling still has a job in calibration and disputes, but only full coverage describes the whole team.

Sampling (manual QA)100% coverage (automated QA)
Share of conversations scoredA few per agent per weekEvery conversation in scope
What the score describesA few chosen conversationsThe whole team
Rare failures (compliance, wrong answers)Usually missedFlagged when they happen
ConsistencyVaries between reviewersOne standard, checked by calibration
Reviewer timeHours of grading weeklyRedeployed to calibration and coaching
Coaching inputPrepared by handDrafted per agent from all conversations
Where it still fitsDisputes, deep reviews, calibration setsDay-to-day measurement

A human read of a hand-picked set is how you calibrate the AI and settle disputes. It just cannot be the measurement itself. Automated QA removes the manual grading, not the human. More on what full coverage changes in what 100% QA coverage means.

Leveraging adaptive AI in customer service QA

How customer service QA evolved from samples to full coverage

Customer service QA has moved through three eras: manual sampling in spreadsheets, QA software with digital scorecards and calibration, and AI-powered QA that scores every conversation. Each step made scoring more consistent, but only the third changed coverage.

A fourth shift is here too: AI is starting to handle conversations directly. Gartner found that 85% of service leaders planned to explore or pilot customer-facing conversational GenAI in 2025 (survey of 187 leaders, 2024). Those AI interactions need grading on the same standard, which makes complete coverage more important than ever.

How to scale customer service QA without hiring more analysts

You scale customer service QA without adding analysts by letting AI score every conversation and sending people only to the conversations that need judgment: auto-fails, low scores, disputes and calibration sets.

The math of manual QA explains why hiring does not scale. Take a team where each agent handles 50 tickets a day, or 250 a week: an analyst who scores five per agent covers 2% of that agent’s work, and doubling coverage means doubling the QA team. With automated scoring, coverage stops being a staffing question. Analysts move to calibration, which keeps the AI’s scores aligned with your own reviewers, and to coaching. For teams that grow quickly, see QA when your support team scales.

What modern customer service QA looks like in practice

Modern customer service QA scores every conversation automatically and moves the QA team from grading to calibration and coaching. One Kaizo customer shows the shift.

SteelSeries, with a 30-person support team, cut its QA time in half with AI summaries, and senior time moved from inspection to improvement. The QA team did not disappear. It shifted to calibration and coaching, the higher-value work manual grading never left time for. Read how SteelSeries cut its QA time in half.

Can you trust AI to score customer service conversations?

Yes, when human oversight is designed in. Support leaders trust automated QA when the AI is calibrated against their own reviewers, overrides improve future scoring, scores are checked against CSAT, and the team controls how much is automated.

  • Calibration sessions to align the AI with your team’s standards, the highest-trust signal there is.
  • Reviewer corrections at scale, so every human override improves future scoring.
  • External validation, tying QA scores to CSAT so higher scores demonstrably track happier customers.
  • A controllable automation dial, so you decide how much is automated and raise it as confidence grows.

For how to test this, read how accurate AI QA is.

How do you build the business case for customer service QA?

The business case for upgrading customer service QA has to convince three people: the Head of QA, the Head of Support or Operations, and the team leads. Starting with one team makes the before-and-after measurable for all three.

  • The Head of QA cares about coverage and fairness: from a sample to every conversation, and no reviewer bias.
  • The Head of Support or Operations cares about ROI and time: reviewer hours reclaimed and how fast the investment pays back.
  • Team leads care about coaching: less prep, more time developing agents.

Measure the first team, then extrapolate reclaimed hours and score gains across the operation.

Customer service QA software: what to look for

Before you shortlist vendors, read our best customer service QA software, compared. Good software scores every conversation against your own scorecard, explains each score, supports calibration and human review, turns findings into coaching and connects to your helpdesk.

  • Coverage. Can it score every conversation unattended, or does the AI only assist a manual sample?
  • Your scorecard, not a template. Custom criteria, weights and auto-fail rules, scoped per team, queue or client.
  • Explained scores. Every score should come with a comment a reviewer can check against the conversation and overturn.
  • Calibration and human review. Reviewers and the AI calibrated on the same conversations, with human review where it matters.
  • Coaching built in. Findings turn into per-agent coaching with tracked impact.
  • Integrations. Connects to your helpdesk (Zendesk, Salesforce and more) without a long build.
  • Digital-first coverage of chat, email and messaging, not a voice product with chat bolted on.

Test coverage, scorecard flexibility and explained scores in a demo rather than taking them on trust: ask to see one score taken apart against the conversation it came from.

Customer service QA by industry and team size

Customer service QA follows the same method in every industry and team size, but the priority changes: growth for SaaS, peaks for ecommerce, compliance for regulated teams, one standard for BPOs and large operations.

  • SaaS and tech support. Keep quality steady while headcount scales fast, and ramp new agents from real conversations.
  • Ecommerce and retail. Protect CSAT through seasonal spikes when manual coverage collapses.
  • Fintech and regulated support. Compliance coverage on every conversation, not a sample.
  • BPOs and outsourcers. Standardize quality across clients with unified evaluations and prove SLAs. See Kaizo for BPOs.
  • Marketplaces and travel. Maintain consistency across high volume and many interaction types.

How does customer service QA work in a large, multi-site operation?

In a large operation, customer service QA has to hold one standard across sites, languages, BPO partners and AI agents. That means one core scorecard with scoped variations, calibration across sites rather than within them, and reporting that compares like with like.

  • More than one site or language. Reviewers in each site drift toward local norms. Run cross-site calibration on a shared set of conversations each month, and keep criteria wording identical in every language version of the scorecard.
  • BPO partners. Each partner should be scored on your scorecard, not its own, so you can compare vendors and hold them to the SLA with evidence. Our guide to BPO quality assurance covers the contract side.
  • AI agents. Conversations handled by an AI agent belong on the same scorecard as human ones. Kaizo scores the conversations Zendesk AI agents handle side by side with human ones.

Foot Locker Europe is an example of what that visibility does at scale. When volumes more than doubled in 2020, its European care team used Kaizo on top of Zendesk to see performance at management and agent level across several languages, and cut resolution time by 75% and reply time by 37% between 2020 and 2021, with CSAT up 13%.

Agent workforce in customer service operations

How does customer service QA support compliance?

In regulated industries, customer service QA is about risk as much as experience. A missed identity verification, an omitted disclosure or incorrect regulated guidance can carry real consequences, and a sample leaves most conversations unchecked.

Automated QA checks every conversation for the required steps and flags violations when they happen, turning compliance from a quarterly spot check into continuous assurance. See how this works for compliance and risk teams.

Regulators look for this. The UK FCA’s review of the Consumer Duty support outcome (first published March 2025) lists call listening and speech analytics among the ways firms monitor support, and 13% of the firms it surveyed said they carried out no quality assurance on their support channels at all.

If you support customers in the EU or UK, QA itself processes personal data, so GDPR applies to the program. Keep only the conversation data the scorecard needs and delete it when it is no longer necessary (GDPR Article 5). And keep a person in decisions about agents: Article 22 gives people the right not to be subject to a decision based solely on automated processing that significantly affects them, so an AI score should inform a coaching or HR decision, not make it.

Signs your customer service QA needs an upgrade

Your customer service QA needs an upgrade when reviewers reach only a small share of conversations, agents dispute scores, team leads spend hours grading, or QA cannot keep pace with headcount.

  • Reviewers reach only a small percentage of conversations, and the sample may not be representative.
  • Agents push back on scores as subjective or inconsistent between reviewers.
  • Team leads spend hours each week grading and preparing coaching by hand.
  • You are scaling headcount and QA cannot keep pace without hiring reviewers.
  • You run multiple teams, channels or BPO partners and struggle to apply one standard.
  • Compliance rests on a sample, leaving most conversations unchecked.

If three or more of these are true, manual QA is already the bottleneck and it is time to automate.

When is automated customer service QA not the right fit?

Automated customer service QA is not the right first step for every team. If you handle a few dozen conversations a week, a lead can read them all and a spreadsheet scorecard is enough. If you have no agreed scorecard yet, write and calibrate one first, because automation only scales the standard you already have.

The same honesty applies to channels. If most of your volume is phone, check how any QA tool handles calls before you buy, and do not assume a text-first product covers them.

How should QA cover AI agents and digital channels?

Customer service QA now has to cover support going digital-first and AI agents handling conversations directly. The answer is one scorecard for every channel and for both human and AI conversations, scored side by side.

For digital-first teams, that combination of full coverage and one standard is fast becoming the norm. For the AI side, see how to QA AI agents.

How does QA data feed Voice of the Customer?

QA data feeds Voice of the Customer because every scored conversation is also a signal about what customers struggle with, where processes break and which issues drive the most contacts.

In Gartner research from 2024, among service leaders who rate their QA program highly valuable, 52% say that is mainly because of voice-of-the-customer insight, against 19% who point to rep performance.

With full coverage you can spot the top contact drivers, feed product and policy teams real evidence, and connect service quality to retention. When you score every conversation rather than a sample, that signal becomes representative enough to act on, which makes QA the front door to conversation intelligence.

Where to start with customer service QA

Start with the scorecard: write down what a good conversation looks like, calibrate two reviewers on the same ten conversations, and fix every criterion they read differently. Then decide how much of the scoring to automate.

To see what scoring every conversation looks like, book a live demo: we walk you through Kaizo with realistic sample conversations, scored, explained and coached.

Frequently asked questions

How do you scale customer support quality without adding more QA analysts?

Let AI score every conversation against your scorecard, and send analysts only to the conversations that need a human: auto-fails, low scores, disputes and calibration sets. Coverage then stops depending on headcount. Analysts spend their time keeping the AI calibrated against their own judgment and coaching agents, which is the work that raises quality, instead of grading a queue by hand.

How do you ensure consistent QA scoring across multiple support teams?

Use one core scorecard for every team, with identical criteria wording, and add team-specific criteria only on top. Run calibration across teams on a shared set of conversations each month, and rewrite any criterion that reviewers score differently. Track reviewer agreement as a number, so drift between teams shows up before agents notice it.

How many conversations should you review per agent?

Manual programs often manage a few conversations per agent per week. For an agent who handles 250 tickets a week, five reviews is 2% of their work, which leaves a wide margin of error. Automated QA removes that limit by scoring every conversation. The question then shifts from how many to sample to what to do with complete coverage.

What is quality assurance as a service?

Quality assurance as a service means an outside provider runs part or all of your QA: its reviewers score your conversations against a scorecard, report the results and sometimes coach agents. It adds capacity without hiring, but it is still sampling. Agree on your own scorecard and calibrate with the provider, or the scores will reflect their standard rather than yours.

How do you measure customer service quality?

You measure customer service quality with a weighted QA scorecard applied to conversations. Then you track the quality score trend, its correlation with CSAT and the impact of coaching, and you quote coverage next to every score. Automation lets you measure every conversation rather than a sample, so the score describes the whole team.

What is the difference between QA and QC in customer service?

QA (quality assurance) is the ongoing process of defining a standard, evaluating conversations against it and improving them. Quality control (QC) is more of a final check on output, catching defects after the fact. In support, QA is the broader, continuous discipline that includes scoring, calibration and coaching; see quality assurance vs quality control.

In KaizoAssignmentsAssignments give manual reviewers an even, consistent distribution of work, targeted at the conversations where a human look actually changes something.See Assignments

On this page

See this in a live demo

We will walk you through Kaizo with realistic sample conversations, so you see the example scored, explained and coached.

Trusted by global support teams

  • Foot Locker
  • Canva
  • GetYourGuide
  • Instacart
  • BlaBlaCar
  • UiPath