How Pronunciation Assessment Works (and How Accurate It Is)

A pronunciation checker doesn't listen the way a teacher does. It uses a speech model to work out which sounds you produced. It lines them up against the sounds you were supposed to say. Then it scores how well each one matches. For whole sentences the best systems now correlate with experts at about 0.74–0.78, close to how well human judges agree with each other. For single sounds they're much weaker.

Contents

Why I wanted to know what's inside

I specialised in experimental phonetics and once wrote programs for automatic phonetic transcription. At Taalhammer we now use a commercial pronunciation-scoring service in the app. Before showing its numbers to learners, I wanted to know exactly what they mean. This article is the answer: how the technology works, what data it learns from, who sells it and how far you can trust it. If you're after practical tips on recording yourself, see hearing yourself: the oldest pronunciation trick.

How does pronunciation assessment work?

Almost every scorer follows the same chain. You read a sentence you were given, and the system compares your recording with what that sentence should sound like.

1. Acoustic model. A neural network trained on many hours of mostly native speech looks at your audio every 10 milliseconds. For each slice it estimates how much it sounds like each speech sound.
2. Expected sounds. The text you were asked to read is turned into phonemes with a pronunciation dictionary: fast becomes F AE S T.
3. Forced alignment. The system finds where in your recording each expected sound sits. It doesn't ask "what did you say?" but "where is the F, where is the AE?"
4. A score for each sound. For every aligned slice it asks how convinced the model is that this was the right sound rather than some other one.
5. Scores like a teacher. A final model, trained on human expert ratings, turns the per-sound numbers, pauses and pitch into word, fluency and overall scores.

One consequence surprises people. In this "scripted" mode the machine listens for the sentence it expected. Say a different, perfectly pronounced word and it scores as a bad version of the expected one. That's why vendors offer a separate unscripted mode, which first has to guess what you said.

What is Goodness of Pronunciation (GOP)?

Step 4 has a name: Goodness of Pronunciation. Silke Witt and Steve Young at Cambridge defined it in 2000. It's "the duration normalised log of the posterior probability" that the speaker said the expected sound, given that stretch of audio (Witt & Young, 2000). In plain terms: how likely is the expected sound compared with the best competing sound, averaged over its duration.

Their best system agreed with human judges at 0.72 on individual sounds. The judges agreed with each other at 0.78. That's a 26-year-old result, and the gap between machine and human at the level of single sounds has never fully closed.

GOP has kept evolving:

• Neural networks replaced the older statistical acoustic models. A Microsoft Research Asia paper on neural-network GOP (Hu et al., 2015) won the ISCA best-paper award.
• Alignment-free GOP. Modern speech models trained with CTC produce "spiky" outputs that don't segment cleanly. Researchers at NTNU in Trondheim proposed a version of GOP that considers all possible segmentations instead of one (Cao, Fan, Svendsen & Salvi). A CTC-based GOP also won the 2025 Norwegian children's pronunciation challenge, NOCASA.

We use the same idea ourselves, in a different direction. To check whether a text-to-speech voice pronounces a minimal pair correctly, we ask a speech model which of the two spellings fits the audio better. We describe that in how we test text-to-speech voices for language learning.

What data trains a pronunciation scorer?

This is the part nobody explains, and it decides everything. A scorer needs two kinds of data:

• Lots of native speech, so the acoustic model knows what correct sounds look like. This is plentiful.
• Learner speech rated by experts, so the final model can imitate a teacher's judgement. This is scarce and expensive.

So the standard recipe trains the acoustic model on large native corpora and then a small scorer on a small set of human-rated learner speech. The best-known datasets:

Dataset	What it is	How it's labelled
TIMIT	630 native American English speakers, read sentences	hand-checked, time-aligned phonetic transcription
LibriSpeech	~1,000 hours of native English audiobooks	text only; used to train acoustic models
L2-ARCTIC	24 non-native speakers of English (Hindi, Korean, Mandarin, Spanish, Arabic, Vietnamese)	in 3,599 sentences, annotators marked 14,098 substituted, 3,420 deleted and 1,092 added sounds
speechocean762	5,000 English sentences from 250 Mandarin-speaking learners, half of them children	5 experts scored every sentence: each sound 0–2, each word 0–10, the sentence for accuracy, fluency, completeness and prosody
NB Tale	Norwegian speech database, 380 speakers	includes 120 speakers of Norwegian as a foreign language, annotated sound by sound
TeflonNorL2 / NOCASA	10,334 recordings of children saying 205 Norwegian words	experts rated each 1–5

Two details show how hard the labelling is. The speechocean762 experts first voted on the "correct" pronunciation of each word, so that a British vowel wouldn't be marked wrong against an American reference. And the scores are skewed: most sounds and words score high, so models see few clear examples of errors.

Small data shows. In the Norwegian NOCASA challenge, the best system recognised the right 1–5 rating with only 44.8% balanced recall (NOCASA).

From GOP to end-to-end models and audio LLMs

Newer systems learn more of the chain directly. Two results show where the field stands:

• GOPT (MIT, 2022) feeds GOP features for each sound into a small Transformer and predicts every score at once. On speechocean762 it correlates with the experts at about 0.61–0.68 per sound and 0.74 per sentence, but only 0.29 on word stress (Gong et al., 2022).
• Models that name the wrong sound. Fine-tuned self-supervised speech models (wav2vec 2.0) catch individual mispronunciations with an F1 of about 0.61 on L2-ARCTIC (Xu et al., 2021).

And audio LLMs? In a 2025 study, Microsoft researchers asked an audio language model to grade word pronunciation with no training for the task. Its correlation with the experts was −0.03: no signal at all. Fine-tuned on scored data it reached 0.77–0.78 per sentence but only 0.39 per sound (Microsoft, 2025). Our own test of OpenAI's audio models on Norwegian minimal pairs pointed the same way: close to guessing.

Who offers pronunciation assessment

Provider	What you get	Languages	What's public about the model
Microsoft Azure Pronunciation Assessment	accuracy, fluency, completeness and an overall score; word errors (omitted, inserted, mispronounced); a score for every phoneme	33 locales incl. Norwegian Bokmål; prosody scoring and "which sound did you actually say" only for US English	architecture not published; Microsoft authors report 0.62 per word and 0.78 per sentence against experts on speechocean762 (docs)
Speechace	sound, syllable and word scores, stress, fluency, mapping to IELTS/CEFR	English, French, Spanish variants	proprietary; vendor claims 0.9 correlation with raters, no paper (plans)
SpeechSuper	sound to paragraph scores, stress, fluency	8 languages	not published
ELSA	its own engine inside its app	English	not published
Langcraft	speech-to-IPA with timestamps and detected substitutions, insertions and deletions	"40+ languages" claimed	a Berlin start-up founded in 2026; no papers or public prices (langcraft.world)

Google, Apple and Amazon offer speech recognition, but we found no public pronunciation-scoring API from them.

What we use: Taalhammer uses Azure Pronunciation Assessment. Azure gives each sound an accuracy score but no "right, substituted, dropped or added" label, so we derive one. A sound below 80 on Azure's 0–100 scale counts as wrong; whole words that Azure reports as omitted or inserted mark all their sounds. Azure itself flags a word as mispronounced below 60, and recommends setting thresholds to fit the learner, more leniently for children.

How accurate is automatic pronunciation assessment?

Comparison	Agreement with experts
Human judges with each other, single sounds (Witt & Young)	0.78
GOP in 2000, single sounds	0.72
GOPT 2022, single sounds / whole sentence	0.61–0.68 / 0.74
Azure on speechocean762, word / sentence	0.62 / 0.78
Fine-tuned audio LLM, single sounds / sentence	0.39 / 0.77–0.78
Audio LLM without task training, words	−0.03

The pattern is consistent. Sentence scores are trustworthy; single-sound verdicts are shaky, for machines and humans alike. Scores also depend on the reference accent, the learner's native language, how strict the original raters were, and whether the speaker is a child.

There's a deeper caveat. A high "accuracy" score mostly measures how close you sound to the native reference, not how well you're understood. Munro and Derwing showed this in a classic study. Accent, ease of understanding and actual intelligibility are related but partly independent, so speech can be heavily accented and perfectly intelligible (Munro & Derwing, retrospective). Many teachers follow John Levis's argument that the goal is to be understood, not to sound native. A 70 can be perfectly clear.

Why your own ear isn't enough

So why use a machine at all? Because we're poor judges of our own speech:

• You hear a different voice. When you speak, sound also reaches your inner ear through your skull, which boosts the low frequencies. A recording lacks that part, so it sounds thinner than the voice you know (Acoustical Society, 2013).
• Weak speakers overrate themselves, strong ones underrate themselves. In a study of 134 learners, self-assessment of accent and comprehensibility was mostly inaccurate in exactly that pattern (Trofimovich et al., 2016).
• Your ear steers your mouth. When researchers shifted speakers' vowels in their headphones, the speakers unconsciously changed how they articulated to compensate (review of Houde & Jordan's experiment). The catch: you can only correct what your ear can detect.

That last point is why minimal pairs matter. If you can't hear the difference between two sounds, you won't notice when you get it wrong. More in hearing yourself.

Does it actually help you learn?

Yes, with conditions. A meta-analysis of 86 studies found a large effect of pronunciation teaching (d = 0.89). The effect was larger with feedback (Lee, Jang & Plonsky, 2015). For practice with speech recognition specifically, a 2024 meta-analysis found a medium effect (g = 0.69). It worked best with explicit feedback on individual sounds (0.86) and on individual sounds rather than intonation and rhythm (0.82 vs 0.37). Practising with a partner beat practising alone (0.89 vs 0.44) (Ngo, Chen & Lai, 2024).

How to use a pronunciation score

1. Read the sounds, not the single number. Which sounds keep scoring low? That's your list.
2. Treat a word score as a signal, not a verdict. Single-sound judgements are the least reliable part of the system.
3. Train your ear first. Practise the problem sounds as minimal pairs, then record yourself.
4. Aim to be understood. Accent and intelligibility aren't the same thing.

FAQ

How do pronunciation checkers work?

They line up your recording with the sounds of the sentence you were asked to read. Then they score how well each sound matches a model of native speech. A final model, trained on expert ratings, turns those scores into word and sentence ratings.

How accurate are AI pronunciation scores?

For whole sentences the best systems correlate with experts at about 0.74–0.78, close to the 0.78 that human judges reached with each other in a classic study. For individual sounds agreement is lower, roughly 0.6–0.7, and some models do much worse.

What data are pronunciation assessment models trained on?

Acoustic models learn from large amounts of native speech such as audiobooks. The scoring layer learns from smaller sets of learner speech rated by experts, such as speechocean762 or L2-ARCTIC.

Is Azure pronunciation assessment good?

It covers 33 locales including Norwegian Bokmål and, according to Microsoft researchers, correlates with experts at 0.78 at sentence level on a standard English test set. Prosody scoring and alternative sound suggestions are available for US English only.

Can an app tell if I have an accent?

It can tell how far your sounds are from its native reference, which is close to measuring accent. It can't directly tell whether people understand you, and that's what matters most.