We measured our own dictation accuracy. Here are the numbers.

Halopen’s on-device mode transcribed 8 of 10 clips exactly, at 0.30× real time on an M4 Max, with a mean word error rate of 6.0% once number formatting is ignored and the personal dictionary is applied — or 17.6% scored strictly against the spoken form, which is the same run and the same transcripts. Why one figure is not enough is the first thing this page explains. The two it missed were a French sentence and a compound product name. Every number here comes from a script in our repository, and you can rebuild the audio yourself with a command that ships on your Mac.

We published this because the dictation roundups that rank for “best dictation app” mostly do not test. One widely-cited guide says so in its own disclosure: it compared products from their documentation and pricing pages rather than benchmarking them. That is an honest thing to admit, and it leaves a gap. Here is our half of it.

We measured our own app only. Nothing here scores another product, so nothing here can misstate one.

One number would have lied

Word error rate compares a transcript to a reference, word by word. Ours is written the way the sentence was spoken, which creates an odd result: when the clip says “claude opus four point seven” and the model writes Claude Opus 4.7, strict scoring calls that a 60% failure — while a writer would call it correct.

So we report three numbers instead of one:

Scoring	Mean WER	What it counts
Strict	17.6%	Every difference from the spoken form, including 4.7 for “four point seven”
Ignoring number and clock formatting	6.6%	Digits and “a.m.” treated as the same as their spoken words
With the personal dictionary applied	6.0%	As above, plus the multi-word dictionary a real user has

The normaliser that produces the middle column has tests that make sure it cannot flatter the result: “Maria” for “Meria” and the mis-decoded French clip must still count as errors, and they do.

Every clip, and what came back

Measured 2026-09-15 on an Apple M4 Max, macOS 26.6.2, using Whisper Large v3 on-device. Model load, paid once per launch, was 1.6 seconds.

Clip	Audio	Transcribe	Real-time factor	WER (+dictionary)	What came back
1	1.2s	0.75s	0.63×	0%	Charlevoix, Michigan.
2	1.6s	0.72s	0.45×	0%	Claude Opus 4.7
3	2.7s	0.77s	0.28×	0%	Let me check the metadata in the Supabase function.
4	2.2s	0.73s	0.33×	0%	The brown fox jumps over the lazy dog.
5	1.9s	0.75s	0.40×	50%	Bonjour. Comment à l’es-vous.
6	2.4s	0.76s	0.31×	0%	Uh so like, you know, the thing is…
7	6.0s	0.87s	0.15×	10%	…verify the whisper kit integration before I ship the privacy mode.
8	4.8s	0.87s	0.18×	0%	Cafe Meria opens at 7 a.m. on weekdays and we serve cold brew until 2 p.m.
9	4.6s	0.81s	0.18×	0%	Send Dallas a quick note about the affiliate code and ask him to share it on Instagram.
10	6.3s	0.86s	0.14×	0%	…makes voice dictation race against the IDE.

Longer clips are relatively faster: the two six-second takes transcribed in under a second each, while a 1.2-second clip took 0.75s. Most of the cost is fixed, not per-word.

The two real misses

Clip 5, French. The reference is “Bonjour, comment allez-vous”; it came back “Bonjour. Comment à l’es-vous.” This is not a mystery: on-device recognition transcribes in your Mac’s first preferred language instead of detecting the language of each take. Dictating another language means reordering your languages in System Settings. Cloud transcription behaves differently here.

Clip 7, a compound name. “WhisperKit” came back as “whisper kit”. The words are right and the joining is wrong — a small thing in prose, an annoying thing in a technical sentence.

And one that fixed itself. Clip 8 contains “Cafe Meria”. The model heard Maria, the way most people would. Because “Cafe Meria” is a multi-word entry in the personal dictionary, the app corrected it after transcription — which is exactly what that feature is for, and why the last column reads 0%. Single-word dictionary entries are not applied on-device; why dictation misspells the names you added explains the mechanism.

Reproduce it without our code

The clips are speech synthesis, which is the part that makes this independently checkable: say ships with macOS, so you can rebuild byte-comparable audio and score any dictation tool you like against the same sentences.

# One clip; repeat for each sentence below.
say -v Samantha -r 175 -o clip.aiff "the brown fox jumps over the lazy dog"
afconvert -f WAVE -d LEI16@16000 -c 1 clip.aiff clip.wav

The ten references, in order: Charlevoix Michigan · claude opus four point seven · let me check the metadata in the supabase function · the brown fox jumps over the lazy dog · Bonjour, comment allez-vous (voice: Audrey) · uhhh so like, you know, the thing is · I’m working on the dictation app and I need to verify the WhisperKit integration before I ship the privacy mode · Cafe Meria opens at seven AM on weekdays and we serve cold brew until two PM · Send Dallas a quick note about the affiliate code and ask him to share it on Instagram · The cursor jumps to a new line every time the autocomplete fires which makes voice dictation race against the IDE.

Word error rate is edit distance over words, divided by the number of reference words, after lowercasing and removing punctuation.

What these numbers are not

• Not live microphone accuracy. Synthesised speech has no room tone, no breath, no accent, no plosives. Real dictation is harder. This measures the recognition path under clean conditions, which is a floor, not a promise.
• Not a rating. Ten clips on one Mac, one run. A different Mac, a different room or a different voice will produce different numbers.
• Not cloud mode. These are on-device results only. Cloud transcription uses a different model with different behaviour, and we have not published numbers for it.
• Not a comparison. We did not install, run or score another product, and we are not going to publish other companies’ numbers we cannot keep current.

If you want to check it against your own voice instead, that is the honest test anyway: download it, turn off Wi-Fi, and dictate a sentence with a name and a number in it. Our cloud-versus-on-device guide covers what changes when you do.