ASR and STT: speech recognition on phone calls
Jaseem Abbas
- Published
How does ASR work on a phone call in India?
ASR takes the caller's audio, a few moments at a time, and writes out the words it hears, so that software can act on what was said. The same job is called STT, short for speech to text; the two names are used for the same thing, and this page uses both.
Picture a caller in Nagpur who rings an online store's helpline and says: "मेरा order कल तक आना था, order ID 48213 है". The ASR writes the Hindi words in Devanagari and the English words, "order" and "ID", in English, and the digits as digits, so the agent can look the order up. The caller, the store and the order are made up for this example.
Phone calls are a hard case. NIST's OpenASR20 challenge (read 15 November 2026) tested ASR on "conversational telephone speech" in ten languages, Tamil among them, and the paper itself describes the telephone as a challenging kind of data.
How is ASR accuracy measured?
ASR accuracy is measured as word error rate, or WER: the errors in the ASR's output, divided by the number of words a person says were spoken. NIST's OpenASR20 paper states it exactly: WER "is computed as the sum of deletion, insertion, and substitution errors in the ASR output compared to a human reference transcription, divided by the total number of words in the human reference transcription".
In plain terms, a deletion is a spoken word the ASR missed, an insertion is a word it added, and a substitution is a word it got wrong. If a caller says 10 words and the ASR drops one and gets one wrong, WER is 2 ÷ 10, which is 20%. The lower the number, the better. Compare WER only on the same kind of audio: a quiet room and a noisy road give different results.
Are ASR and STT the same thing?
Yes: ASR and STT name the same job, turning speech into text. ASR is the term used in research and by engineers, and speech to text is the plainer product name. A third term, voice recognition, can also mean recognising who is speaking, which is a different task.
How does AI change ASR?
ASR is the ears of an AI voice agent: whatever it misses, the agent never hears. A voice agent works in a loop: the caller speaks, ASR writes it down, a language model decides what to say, and a voice speaks the reply. Voice activity detection works just before it, marking when the caller starts and stops talking, and text to speech works just after, speaking the reply.
On Bananaflow, an agent hears English and 10 Indian languages: Hindi, Bengali, Gujarati, Kannada, Malayalam, Marathi, Odia, Punjabi, Tamil and Telugu. It follows the caller's language, and it hears Hindi mixed with English as it was said, with Indian words in their own script and English words in Latin letters. A Dictionary of brand names and terms helps the agent hear the words that matter to your business. For the full picture, read how Bananaflow agents hear English and 10 Indian languages, and for how Hindi sounds to speech recognition on a call, Hindi speech recognition on phone calls. For when the agent decides a caller has finished, see endpointing and turn detection.
Why does ASR mishear names and numbers?
ASR mishears names and numbers because they are the words it has the fewest clues for. A common word is predicted from the sentence around it; an order ID or a surname is not. Accents, a noisy line and speaking fast make it harder. A good agent reads key details back before it acts, and a Dictionary of your own brand names and terms helps with the ones that repeat.
FAQ
Frequently asked questions
What is the full form of STT?
STT stands for speech to text. It is software that turns spoken words into written text, and it is the same job as ASR, automatic speech recognition.
What is the full form of ASR?
ASR stands for automatic speech recognition: software that listens to speech and writes down the words. In phone calls it lets a voice agent, a transcript or a quality check work with what the caller said.
Are ASR and STT the same?
Yes. They name the same job. ASR is the term engineers and researchers use; speech to text is the plainer product name. Voice recognition can also mean working out who is speaking, which is a different task.
What is word error rate?
Word error rate (WER) scores ASR accuracy: deletions, insertions and substitutions in the output, divided by the number of words in a human reference transcript. NIST uses it as the main metric in its ASR challenges.
Which languages does a Bananaflow agent hear?
English and 10 Indian languages: Hindi, Bengali, Gujarati, Kannada, Malayalam, Marathi, Odia, Punjabi, Tamil and Telugu. The agent follows the caller's language and hears Hindi mixed with English as it was said.
Can I help the agent hear my brand names?
Yes. On Bananaflow you add your brand names and terms to a Dictionary, and the agent uses it to hear them correctly. Test the names by voice in your browser before going live.
How is ASR different from voice activity detection?
Voice activity detection only marks when someone is speaking; ASR writes down what was said. On a voice agent call, detection comes first, then ASR, then the language model and the voice.
