Skip to content

Bananaflow MCP is live.

Read the docs

Speech Analytics for Call Centers in India

Jaseem Abbas, founder of Bananaflow, in glasses and a dark top against a pale background.Jaseem Abbas

Published
Cover for "Speech Analytics for Call Centers in India": a white line diagram of every call being heard: five sound waves in a column, five recorded calls, each send a line that converges on a lit node under a lit pill reading Every call, not just a few, and a bold arrow runs on to a list panel labelled Search, count, score, drawn over a soft, out-of-focus delphinium in muted sky blue, with a sage mint haze at the far left and a dusty rose glow at the far right.

Speech analytics is software that turns recorded calls into text and data, so you can search, count and score every call instead of the few a person can hear. This guide covers call center speech analytics for Indian teams: what it measures, where mixed Hindi-English calls break it, how to test it on your own calls, how to roll it out and what Indian privacy rules mean for it.

Here is the problem in one scene. Imagine an 80-seat broadband support floor in Visakhapatnam that turns on speech analytics. In week one, a search for the word "cancel" finds very few cancellation calls. The tool is not broken. Callers are saying "band karna hai" and "disconnect", and the phrase list has neither.

The floor in Visakhapatnam is an example we made up.

What is speech analytics?

Speech analytics is software that turns recorded calls into text and data, so you can search, count and score every call instead of the few a person can hear. In a call center it is used to check quality, find compliance gaps, learn why people call and spot trouble such as long silences.

Two pieces of technology sit underneath:

  • Automatic speech recognition (ASR) turns the audio into words. Its main quality number is the word error rate, which is the share of words it gets wrong.
  • Natural language processing (NLP) reads those words to find phrases, topics and sentiment.
A call recording becomes a transcript of who said what, then tags, topics and sentiment, then scores and alerts, then reports and coaching.

How is speech analytics different from call analytics and voice analytics?

Call analytics counts what happened around a call without listening: volumes, durations, missed calls, peak hours. Voice analytics, in the usual use of the term, looks at how something was said, such as pitch and tone. Speech analytics looks at what was said. A real floor often needs all three, and the words people use for them overlap, so ask a vendor which one it means.

Our guide to call center metrics covers the counting side. Speech analytics explains why a number moved.

What does speech analytics do?

Speech analytics does six things, and each depends on something below it. If the transcript is wrong, everything built on it is wrong too. The table below is our own framework, not a standard. An AI agent can also add a summary of each call.

  • Capability: Transcription

    What it gives you
    Searchable text of every call
    What it needs
    Speech recognition that handles your languages
    Indian-call caveat
    Code-switching raises errors, and the script (Devanagari or Roman) changes what a search finds
  • Capability: Keyword and phrase tracking

    What it gives you
    Did the agent say the recording line? Did callers say refund, cancel or fraud?
    What it needs
    A phrase list
    Indian-call caveat
    Callers say refund, paisa wapas and the Devanagari spelling: list every variant
  • Capability: Topics and call reasons

    What it gives you
    Why people call, by week
    What it needs
    A labelled list or clustering
    Indian-call caveat
    Mixed-language topics split into duplicates unless you merge them
  • Capability: Sentiment

    What it gives you
    The caller's mood per call or per turn
    What it needs
    A text model, sometimes a tone model
    Indian-call caveat
    Research shows mixed-language sentiment is harder
  • Capability: Silence and overtalk

    What it gives you
    Dead air, long holds, interruptions
    What it needs
    Timestamps, plus a way to tell the speakers apart
    Indian-call caveat
    A transcript-only tool sees gaps, not tone
  • Capability: Quality scoring

    What it gives you
    A score per call against your scorecard, with the lowest calls sent to a person
    What it needs
    A written scorecard and calibration
    Indian-call caveat
    Scorecard and calibration are covered in our call audit guide

What is call transcription?

Call transcription is the step that turns a recording into text. It is the base of everything else, and it is also where Indian calls hurt most. A transcript is also useful on its own: a manager can search for a phrase across a week of calls instead of listening to them.

For how recording software should make calls findable, see our guide to call recording software in India.

How does it tell the agent from the caller?

It needs to know who spoke when. This task is called speaker diarization, defined in a 2021 review by Park and co-authors as "a task to identify 'who spoke when'", which the review says is integrated with speech recognition to give speaker-specific information (A Review of Speaker Diarization: Recent Advances with Deep Learning). If both voices are on one mixed audio channel and the tool cannot separate them, talk ratio, overtalk and per-speaker sentiment are unreliable. Ask whether your recording keeps the agent and the caller on separate channels.

What does sentiment analysis tell you?

Call sentiment analysis labels how positive or negative a caller sounds, per call or per turn. Treat it as a way to sort calls for a human to review, not as a verdict. A caller who says "thanks, bahut helpful" in a flat voice may be sarcastic. A text-only tool cannot hear that.

What is automated quality assurance?

Automated quality assurance scores calls against your scorecard without a person listening to each one, then sends the lowest scores to a reviewer. It does not replace your quality analysts. It changes what they listen to. Our guide to AI versus manual call audits has the full comparison, and we do not repeat it here.

Speech analytics vs conversation intelligence: what is the difference?

The two terms overlap, and the difference is mostly who buys them and what they listen to. Speech analytics usually means analysing call-floor recordings for quality, compliance and call reasons. Conversation intelligence is the broader term, and the pages that rank for it are mostly about sales meetings and customer conversations. A search for conversation intelligence platforms returns sales tools, not call-center tools.

Both are different from conversational AI. IBM defines conversation intelligence as using AI tools to analyse and extract insights from business conversations, and says it "doesn't engage with users directly", while conversational AI systems "actively engage in real-time conversations with users" (IBM Think on conversation intelligence). In short, analytics listens afterwards and conversational AI talks during the call. Our guide to what conversational AI is covers the second.

So if you run a call floor, search for speech analytics. If you sell through meetings, conversation intelligence is the term you will see. The tools differ less than the names do, and the checks in this guide apply to both.

Why do Indian calls trip speech analytics up?

Indian calls trip speech analytics up because speakers mix languages inside a sentence, the audio is often noisy, and most speech tools were built and tested on cleaner, single-language data. Research gives a few numbers. They come from research systems and test sets, not from the product you will buy, so read them as a direction. Phone audio adds to the problem, because a call is narrowband and loses sound detail that a clean recording keeps; see Hindi speech to text on calls. For transcribing calls in each Indian language, see transcribing Indian-language calls.

  • Paper: Chadha et al., Code Switched and Code Mixed Speech Recognition for Indic languages (2022)

    What it measured
    Word error rate on code-switched speech
    Result
    21.77 percent on Hindi-English and 28.27 percent on Bengali-English. Combining language identification with single-language models improved error by 50 percent across languages
  • Paper: Bhogale et al., Vistaar: Diverse Benchmarks and Training Sets for Indian Language ASR (2023)

    What it measured
    A collection of 59 benchmarks across Indian languages and domains
    Result
    The authors argue that diverse benchmarks are needed to evaluate and improve speech recognition for Indian languages
  • Paper: Javed et al., IndicVoices (2024)

    What it measured
    A dataset of 7,348 hours from 16,237 speakers across 22 languages
    Result
    74 percent of it extempore and 17 percent conversational, the rest read aloud
  • Paper: Patwa et al., SemEval-2020 Task 9, SentiMix (2020)

    What it measured
    Sentiment on code-mixed tweets, with about 20,000 Hinglish examples
    Result
    The best system scored 75.0 percent F1 on Hinglish and 80.6 percent on Spanglish
  • Paper: We care: Improving Code Mixed Speech Emotion Recognition in Customer-Care Conversations (2023)

    What it measured
    Emotion recognition on natural customer-care calls
    Result
    The authors note that customer-care audio is often noisy, with speakers regularly switching languages, and that most emotion datasets use actors in noise-free settings

The sources, in order, are Chadha et al., Vistaar, IndicVoices, SemEval-2020 Task 9 and "We care".

Three points follow from them.

  • Accuracy depends on language and domain. The Vistaar authors' own argument is that one benchmark is not enough. For you it means a vendor's headline number, measured on someone else's audio, does not tell you how it will do on your calls.
  • Conversational speech is the hard kind. IndicVoices notes how much of its data is extempore or conversational. A call is closer to that than to someone reading a script.
  • Sentiment on mixed text is far from perfect even when the text is clean. The Hinglish result is on tweets, not calls, and it is from 2020. It is a caution, not a prediction for your floor.

Our guide to Hinglish voice agents explains why mixed speech is hard in the first place. The same errors that confuse a voice agent carry into speech analytics on recorded calls.

Devanagari or Roman: why does the script change what you find?

A transcript of the same call can come out in Devanagari, in Roman letters or in a mix, and a phrase search matches letters, not meaning. If your list has "refund" but the transcript has the Devanagari spelling, the search misses it. The same goes for "paisa wapas" against its Devanagari spelling. Build every phrase list in the forms callers really use, in both scripts, and re-check it each month, because transcripts shift when the tool is updated.

What can a transcript not tell you?

A transcript cannot tell you tone of voice, line noise, a sigh or a raised voice. A tool that reads only the text sees the words, the timing and the gaps. That is a general limit, not a flaw in one product. If tone matters to you, ask a vendor whether it analyses the audio too, and test that claim the same way as the rest.

How do you test accuracy on your own calls?

You test accuracy by comparing a tool's transcript with a person's, on your own calls, for each language mix you handle. Do not rely on a vendor's number, because accuracy depends on language and domain. The steps below are our suggestion, not a standard, and the sample sizes are a starting point.

  1. Pick the calls. Take 30 to 50 real calls per language mix you handle: English, Hindi, Hinglish and your main regional language. Include noisy ones.
  2. Transcribe two minutes of each by hand. Use a person who speaks the language.
  3. Count the errors. Word error rate is the wrong, missing and extra words divided by the words spoken. Compare word by word.
  4. Test your phrases. List the 20 phrases that matter most, such as the recording line, the cancellation right and "refund". Count how many the tool found in the calls where they were said. This is phrase recall.
  5. Weigh the two. Phrase recall matters more than overall error rate for compliance checks. A transcript with a few wrong filler words is fine. One that misses the recording line is not.
  6. Check who said what. Look at whether the tool put each line with the right speaker.

If you buy software, ask the vendor to run the same test on your calls before you sign. Our guide to call recording software in India has a wider buying checklist, and we do not repeat it.

What can a call center use speech analytics for?

A call center can use speech analytics wherever the answer is hidden in many calls. Each use below is advice, and each is only as good as the transcript under it.

  • Quality at full coverage. Score every call against your scorecard and send the lowest scores to a person. See call audit in a BPO for the scorecard.
  • Compliance phrases. Check that the agent said the recording line, the cancellation right or, for an insurer, the free look period that a welcome call explains. Our guide to IRDAI rules for insurance sales calls lists what to check.
  • Call reasons and repeat callers. Count why people call, by week. A topic that rises is a product or process problem, not an agent problem. Our guide to call center metrics explains first call resolution.
  • Dead air and long holds. Find the moments with long silences, then look at what came before. Our guide to average handle time covers why handle time grows.
  • Cancellations and churn. Find calls where people ask to leave, whatever the agent logged. Our guide to call disposition codes shows how to compare codes against what was said.
  • Coaching. Use real calls as examples, good and bad.
What speech analytics flags, a missed compliance line, a long silence, rising negative sentiment or a repeated complaint topic, and the action each should trigger.

What does this look like on a real floor?

Back to the Visakhapatnam floor, which is made up. In week one, the search for "cancel" misses calls where callers said "band karna hai" and "disconnect". In week two, the team adds both, and cancellation calls turn out to be about three times what the agents' own call codes showed. Most of them follow a 20-second silence after the agent says "let me check your plan". The fix is a faster plan lookup, not more agent coaching. The numbers are invented to show the method.

How do you roll out speech analytics, step by step?

Roll out speech analytics by starting with one question, sampling, calibrating against your people and only then scaling. The seven steps below are our recommendation, sized for an Indian floor, and not a standard.

  1. Pick three questions. For example: why do callers cancel, do agents say the recording line, where does dead air happen. One question per week of attention beats a wall of dashboards.
  2. Check your notice and purpose. Make sure what callers are told covers analytics, as the next section explains.
  3. Run the accuracy test from the section above, before you commit.
  4. Build phrase lists in callers' own words, in both scripts.
  5. Score a week of calls and calibrate. Have two quality analysts score a sample, compare their scores with the tool's and adjust the scorecard until they agree. Our guide explains how to run call calibration.
  6. Act on one finding a week. A finding that nobody acts on is a dashboard, not an improvement. Re-measure after the change.
  7. Set retention and access for transcripts, not only for audio.

Do not scale until steps three and five agree: the transcript is good enough for the phrases you need, and the scores match your analysts. Scaling a tool you have not calibrated just scores more calls wrongly.

What do Indian privacy rules mean for speech analytics?

Speech analytics is a new use of recordings, so it raises a purpose question under India's Digital Personal Data Protection (DPDP) Act. A recording of an identifiable caller is personal data. This is general information, not legal advice, so check it with your own counsel.

Three points from the Act, whose text is on the Ministry of Electronics and IT website:

  • Purpose (section 6). Consent must be "free, specific, informed, unconditional and unambiguous", for "the specified purpose", and limited to the personal data necessary for it. If your notice says calls are recorded for quality and training, ask whether it clearly covers mining them for sales insights or churn signals.
  • Retention (section 8). Personal data must be erased when consent is withdrawn or when it is reasonable to assume the purpose is no longer served, unless a law needs it kept. A transcript is personal data too, so give it a retention rule, not only the audio.
  • Processors (section 8). You must keep reasonable security safeguards, including over processing done on your behalf by a data processor. An analytics vendor is usually such a processor.

For dates and duties, read our guide to the DPDP rules for call centers. For the recording notice, read is call recording legal in India. No tool makes analytics compliant by itself.

Even researchers treat call audio as sensitive. The CallCenterEN paper describes 91,706 real English call center conversations, about 10,448 hours, from India, the Philippines and the US, and says the audio is not in the public release "due to biometric privacy concerns" (CallCenterEN). That is a research choice, but it is a useful prompt for your own policy.

How Bananaflow fits

Call QA reviews the calls Bananaflow's own AI voice agent takes.

For those calls, it reads the finished call's transcript step by step and returns a score from 1 to 10, tags that quote the moment, the caller's sentiment and a short summary, or the fields of your own scorecard. Its default tags include dead air, a looping agent, a frustrated caller and hearing problems. It runs after the call ends. See Call QA for details.

Every call the agent takes also leaves a transcript, a recording, minutes used, extracted variables and an outcome code, and analytics shows KPIs and trends with a CSV export. If you send call events to BigQuery, it receives diagnostic events. Bananaflow supports English plus 10 Indian languages (languages).

What should you remember about speech analytics?

Speech analytics turns every call into text and data, and it is only as good as the transcript beneath it. On Indian calls that means mixed languages, noise and two scripts, so test on your own calls and check your key phrases. Start with one question, calibrate against your quality analysts, act on one finding a week and check that your consent notice and retention cover the transcripts too.

FAQ

Frequently asked questions

  • What is speech analytics?

    Speech analytics is software that turns recorded calls into text and data, so you can search, count and score every call instead of the few a person can hear. It transcribes the call, then tags phrases, topics, sentiment and silence, and rolls the results into reports and alerts.

  • What is the difference between speech analytics and conversation intelligence?

    The two overlap. Speech analytics usually means analysing call-floor recordings for quality, compliance and call reasons. Conversation intelligence is the broader term and is most often sold for sales meetings and customer conversations. Neither talks to the caller. Conversational AI is the one that talks, and the two are often confused.

  • Can speech analytics handle Hinglish?

    Partly, with more errors than on single-language calls. Published research systems on Hindi-English speech reported word error rates of about 22 percent, and sentiment on mixed Hindi-English text is still far from perfect. Do not trust a vendor figure. Test any tool on 30 to 50 of your own calls and check the phrases you care about.

  • Can speech analytics detect tone of voice?

    Only if the tool analyses the audio itself. A tool that reads a transcript sees the words, the gaps and the timing, but not a sigh, a raised voice or a bad line. Ask any vendor which of the two it does, and do not assume sentiment from text equals tone.

  • What is automated quality assurance?

    Automated quality assurance scores calls against a written scorecard without a person listening to each one. The lowest scores go to a human reviewer. It works only as well as the scorecard and the transcript beneath it, so calibrate it against your own quality analysts before you rely on it.

  • Is speech analytics the same as conversational AI?

    No. Speech analytics listens to calls after the fact and produces data about them. Conversational AI talks to a caller during the call. A floor that uses an AI voice agent can use both, with analytics reading what the agent said.

  • Which calls does Bananaflow Call QA review?

    Bananaflow's Call QA reviews the calls its own AI voice agent takes, and it reads the finished call's transcript after the call ends.

Written by

  • Jaseem Abbas, founder of Bananaflow, in glasses and a dark top against a pale background.

    Jaseem Abbas

    Founder, Bananaflow

    Jaseem founded Bananaflow to build voice AI that understands how people in India really speak, including the switch between languages in the middle of a call.

Similar articles

Bring us one process. We'll pilot it on your real calls, with your team as the fallback.

Plan a pilot