Your dictation cleanup should have to show its work

Dictation has two steps that can go wrong, and only one of them tells you.

When transcription mishears you, the damage is obvious. You said --watch and the screen says “dash dash watch.” It is wrong in a way you cannot miss, so you fix it and move on.

When the cleanup pass edits you, the damage is invisible. It returns a clean, plausible, well-formed sentence — which is exactly what a correct result looks like. Say “I think the retry logic might be racing” and get back “The retry logic is racing.” One word shorter, considerably more confident, and now asserting the thing you deliberately declined to assert. Nothing on screen marks it. The failure mode of a tidying pass is not garbage, it is fluent text that means something slightly different from what you said — and fluency is the property that stops you checking.

So the second step is the one that needs supervision. Whether the tool you use has any is worth five minutes of checking, and there is a test for it at the end of this piece.

The asymmetry, stated plainly

A transcription error is a visible error. A cleanup error is a silent one.

Everything about how these tools are built points at the first problem. Word Error Rate is the number everyone quotes, the benchmark everyone runs, and the thing that has genuinely become very good. There is no equivalent number for the second problem. No standard benchmark asks did this edit change what the sentence claims — so it is not measured, not reported, and not competed on. A step nobody scores is a step nobody has to defend.

It is both. It is a genuinely useful feature that is also the only point in the pipeline where your meaning can change without leaving a mark.

Three things that make a rewrite accountable

Not “turn cleanup off.” Cleanup is useful, and verbatim is not automatically better — it is unedited, which is a different claim. The question is what has to be true before an automated edit is allowed to reach your cursor.

Three things, in increasing order of how much they cost to build.

1. A veto — something that can overrule the edit without you

The cheapest supervision is a second opinion that runs every time and does not need your attention.

Halopen’s cleanup step is checked by something other than itself. After the model returns, a deterministic comparison runs over the pair — the transcript that went in against the text that came back — and measures loss: what fraction of distinct words vanished, how far the sentence count moved, and how many content words of four letters or more disappeared entirely. It is arithmetic, not a second opinion from another model, which is the point: it cannot be talked round and it cannot hallucinate agreement.

When it trips, the cleaned text is discarded and your verbatim words go to the cursor instead. No dialog, no interruption: you get your own sentence, which is the safe outcome, and being interrupted to be told that a safety net worked is not a feature. The rejection is recorded in Halopen’s audit log rather than announced, so it is there when you go looking and absent when you are working.

This runs by default. It is the layer that matters most precisely because it does not depend on you watching.

It also has a limit worth stating outright, because it is the limit that explains everything after it. A loss check catches deletion. It does not catch softening. The retry-logic example at the top of this piece — “I think the retry logic might be racing” arriving as “The retry logic is racing” — does not trip it, and that is deliberate rather than an oversight. Hedges and intensifiers sit on an explicit exclusion list, because an earlier version that counted them as lost meaning fired falsely three times out of four, and a guard that cries wolf at that rate is a guard everybody disables in a week.

So the honest description is narrow: the veto catches the cleanup removing substance, not the cleanup changing your stance. Which is precisely why there is a second layer, and why it is the one you have to opt into.

There is a second veto on the same step: a deadline. If the cleanup does not return in time your verbatim text is injected rather than held hostage to polish, and the fallback is recorded as soft_timeout so it shows up in the log rather than passing unnoticed. Never trade a word for latency is a written rule in this codebase, and a slow tidying pass is not worth a stalled sentence.

2. An appeal — a way to see the edit and refuse it

Turn on the verbatim diff and cleanup stops being a thing that happens to you. Each pass opens side-by-side: what you said on one side, what the model wants to send on the other, with three choices. Accept takes the edit. Edit lets you fix it first. Reject sends your original words.

Two details that took some arguing to get right:

It auto-accepts after four seconds if you do nothing. A confirmation dialog that blocks until you click is not a safety feature, it is a toll booth, and a toll booth on every sentence gets switched off within a day — which leaves you with no supervision at all. So the default action is the one you almost always want, and the window is short.

Unless the veto in the previous section fired. Then there is no countdown and Reject is highlighted instead, because the app has just been told this particular edit probably went sideways, and “probably went sideways” is not something to auto-accept after four seconds.

The diff ships off. That is a deliberate choice rather than a hedge: most cleanups are fine, and a modal on every dictation is its own kind of tax. It is one switch, and worth turning on for a week early — it is the fastest way to learn what your cleanup mode actually does to your writing.

3. A receipt — a record you can read later

Even a perfect diff only helps at the moment it appears. The question that arrives a day later is different: what happened to that paragraph?

Halopen keeps an audit log on your own disk. Each cleanup’s row holds the transcript that went in, the text that came back, and the outcome — carrying the guard’s reason when the guard is what rejected it. So a sentence that came out wrong on Tuesday is answerable on Wednesday, from a file you own, by reading the two versions side by side rather than trying to remember which one you saw.

Halopen’s local log stores transcripts and supported app events. A local dictation entry identifies its transcription route; it is not a monitor of every app or Mac network request. Account, licensing, updates, usage and enabled diagnostics have separate network behavior. See the privacy page for processing and retention details.

The honest counter-argument

All of this is overhead for a step that is usually right.

That is true, and it is why only the veto runs by default. If cleanup were wrong often the answer would not be better supervision — it would be a better cleanup, or none at all. Supervision is the right response to one specific shape of problem: rarely wrong, invisibly wrong, and expensive when it is. One bad edit in a hundred sounds like a rounding error until it is the one that turned a hedge into a commitment in a message to your team. It is the shape people underestimate, because a failure you cannot see feels like a failure that does not happen.

There is a second cost worth naming, and it is not a bug. A visible diff will show you that your cleanup mode is doing more than you assumed — and once you have watched a Polish pass rewrite three sentences you were perfectly happy with, the honest response is usually to change modes rather than to change the setting. That is the diff working as intended. It is still a week of being mildly annoyed at your own software.

How to check the tool you already use

You do not need Halopen to run this. Three questions, in the order they matter:

1. Does the cleanup have a veto? Dictate something a tidying pass will struggle with — a half-finished sentence, a technical token, two languages in one clause. If what arrives is confidently wrong rather than obviously wrong, nothing is checking the checker.
2. Can you see the edit? Look for a diff, a preview, or an undo that restores the transcript rather than clearing the field. If the only undo you have is deleting the text and saying it again, the original was never kept.
3. Can you read it back tomorrow? If there is no log, then every answer about what your tool did to your words is a reconstruction from memory — and the whole problem with a silent edit is that there is nothing in your memory to reconstruct from.

Twelve sentences that make a silent edit visible

“Dictate something a tidying pass will struggle with” is easy to say and hard to act on, so here is the concrete version. Say these into whatever you use, then read what arrives. Each one is ordinary speech that a cleanup pass is tempted to correct, and each has a predictable “corrected” form — which is what makes the result countable rather than a matter of impression.

say this	the tidy version
It works just as good.	just as well
Me and him went to the store.	He and I went
I could care less.	I couldn’t care less
We was running late.	We were running late
I should of checked it first.	should have checked
There’s three files left.	There are three files
I’m gonna refactor the parser.	I am going to refactor
This is real important.	This is really important
Um, I think the config is wrong. — say “the” twice	(filler and stutter removed)
Irregardless of the deadline.	Regardless of the deadline
Who did you give it to?	To whom did you give it
It don’t matter.	It doesn’t matter

Count how many came back as you said them. Two caveats, because a test you can misread is worse than no test. “Should of” is not a fair one — it is acoustically identical to “should have”, so writing “should have” is a transcription choice rather than an edit. And removing “um” and a stutter is the most defensible edit on the list; score it separately rather than holding it against a tool.

The number itself matters less than the direction. A tool that returns eleven of twelve as spoken has a different philosophy from one that returns two — and neither is wrong in the abstract. What is wrong is not being able to find out which one you are using, or not being offered the choice.

Our own score, so you can hold us to the same test

Asking you to run a test we have not published our own result for would be cheap, so here it is.

Halopen 1.8.29, Verbatim mode, macOS 26.6.2, built-in microphone. Three runs of the same clip, identical results each time: ten of the ten scorable sentences came back exactly as spoken.

Two of the twelve are not scorable, and they are excluded for both of the reasons this page already gave you rather than because of how they came out:

• “I should of checked it first” came back as “should have” on one run and “should’ve” on the other two. As noted above, that is a homophone — the audio for “should of” and “should have” is the same audio — so it is a transcription choice, not an edit, and it cannot tell you anything about a cleanup pass.
• The doubled “the” came back as “I think that the config is wrong.” That is a mishearing, not a tidy-up, and it is the test’s fault rather than the app’s: synthesised speech does not render a stutter cleanly enough for the item to decide anything. We are re-cutting that clip.

So: ten for ten on what the test can decide, and two items where the test cannot decide. Not a perfect score — a test with two broken items and a fixed one.

Reproduce it without our code. The clip is built by macOS’s own say, from the sentences in the table above:

say -v Samantha -r 170 -o fidelity.aiff "It works just as good. Me and him went to \
the store. I could care less. We was running late. I should of checked it first. \
There's three files left. I'm gonna refactor the parser. This is real important. \
Um, I think the the config is wrong. Irregardless of the deadline, who did you give \
it to? It don't matter."

Play that into whatever you use — through your speakers into the microphone, the way you would speak it — then count how many of the ten scorable sentences arrive as written, scoring “should of” and the doubled “the” on their own. The same 18-second file measures any dictation tool, ours included, and nothing about the method depends on trusting us.

What this measurement is not. It is twelve sentences, one synthesised voice, one clip, one Mac, and we ran it on our own product. Clean synthetic audio isolates the edit policy, which is the point, but it is easier than a real voice in a real room — an app that preserves your words here might still garble them there. Treat it as a floor on the question “does this tool rewrite me”, not as a rating of accuracy in general. Our separate word-error-rate measurement, method and failures included, is in we measured our own dictation accuracy.

A tool that passes all three has earned the right to edit you. That is a reasonable bar, and it is lower than the one we hold transcription to.

---

Halopen is native macOS dictation, verbatim by default, with cleanup you can veto, inspect and audit. Try the time-limited no-card trial — halopen.com.

Current trial duration and paid plans