Word error rate (WER): meaning and formula
Jaseem Abbas
- Published
What is the word error rate formula?
Word error rate is the number of word mistakes divided by the number of words in the correct text. The mistakes are of three kinds: a wrong word put in (a substitution), a word missed (a deletion) and an extra word added (an insertion). So:
WER = (substitutions + deletions + insertions) / words in the reference
The scoring tool from the US National Institute of Standards and Technology, sclite in the NIST SCTK toolkit (read 16 November 2026), "compares the hypothesized text (HYP) output by the speech recognizer to the correct, or reference (REF) text". It lines the two up, which it calls alignment, and then reports each kind of error as a share of the reference words, for example "# Substituted words / # Reference words * 100". Add the three shares and you have the word error rate. The reference is the transcript a person has checked; the hypothesis is what the speech recognizer wrote.
How do you calculate WER, with a Hinglish example?
Count the three kinds of mistake after the texts are lined up, then divide by the words in the reference. Here is a made-up call line, written the way Hinglish is often typed.
- Reference: mujhe kal subah dus baje call karna (7 words)
- Recognizer output: mujhe kal subah das baje karna please
Three things differ. "dus" came out as "das", one substitution. "call" is missing, one deletion. "please" was never said, one insertion. So the rate is (1 + 1 + 1) / 7 = 3 / 7, or 42.9%. The sentence and the output are made up for this example, and the number says nothing about any real recognizer.
Because insertions count too, the rate can go above 100%, when a recognizer adds more wrong words than the reference holds.
Why does normalisation change the number?
WER compares text letter by letter, so any difference in how the two texts are written counts as an error, even when the meaning is the same. If the reference says "dus baje" and the recognizer writes "10 baje", that is a substitution, though both mean ten o'clock. The same happens with capital letters, punctuation, and "call" typed in Roman letters against "कॉल" in Devanagari.
Normalisation means cleaning both texts the same way before scoring. NIST's sclite does one form of it by default: by its options page, unless you ask for case-sensitive alignment, "all input is mapped to a single case before scoring" (sclite options, read 16 November 2026). Everything else, such as numbers, script and spelling, is a choice you make and must write down. Two WER numbers are only comparable if they were cleaned the same way and scored against the same references.
For Hinglish this is the hard part, since there is no single correct spelling of "dus" or "das". See code-switching for why callers mix languages mid-sentence.
What is character error rate, and why use it for Indian languages?
Character error rate, or CER, is the same formula counted in characters instead of words. It gives partial credit: a word that is one letter wrong costs one character, not one whole word. That helps when spelling varies, as it does in romanised Hinglish, or when a word in an Indian script differs from the reference by one sign.
sclite supports this: its -c option will "chop up the words into separate characters before doing the alignment" (sclite options). The same page notes that this was added to score Mandarin by character. If you score Indian scripts by character, decide first what counts as one character, and state it, since a single written letter can be made of more than one Unicode code point.
How do you run a fair WER test on Indian calls?
Use your own recordings, write the references by hand, normalise both sides the same way, and test names and numbers separately. A single score hides these. Our guide to benchmarking speech-to-text on Indian calls sets out the full method, and how to test Whisper on Hindi phone calls applies it to one open model.
On Bananaflow, the agent hears and speaks English plus 10 Indian languages, and follows the caller's language. Read how a Bananaflow agent hears and speaks English and 10 Indian languages.
FAQ
Frequently asked questions
What is the full form of WER?
WER stands for word error rate. It is the share of words a speech recognizer gets wrong when its text is compared with a correct reference transcript.
What is the formula for word error rate?
WER = (substitutions + deletions + insertions) divided by the number of words in the reference. A substitution is a wrong word, a deletion is a missed word and an insertion is an extra word. Multiply by 100 for a percentage.
Can WER be higher than 100%?
Yes. Insertions count as errors, so a recognizer that adds many extra wrong words can make more errors than the reference has words. The formula allows it.
Why does normalisation change the WER?
Because WER treats any difference in writing as an error. "10 baje" against "dus baje", or a capital letter against a small one, counts as a mistake. Cleaning both texts the same way first stops that, so scores can be compared.
What is character error rate (CER)?
CER is the same calculation done on characters instead of words. A word that is one letter wrong costs one character, not a whole word. It is useful where spelling varies, as in romanised Hinglish.
What is a good word error rate?
There is no single good number, because it depends on the audio, the language, the reference transcripts and how both texts were cleaned. Lower is better, but compare only scores from the same test. Test on your own call recordings.
