The headline is one gauge
The demo at eg2.haku.sh reports 12.6 seconds for a one-hour file on a MacBook Air with an M4 chip. Its page identifies the file as one hour of English audio from a 1923 book. It also reports a first-run model download of 1.72 GB.[1]
That is a sharp demo. It is not a general transcription benchmark. One machine, one file, one language, and one reported duration tell us throughput. They do not tell us how the tool handles overlapping speakers, room noise, product names, code, or the cold start after a clean install.
Keep the stopwatch. Add a transcript.
Fast and wrong is still fast
FluidAudio's published Core ML benchmark shows why speed and accuracy belong on the same card. On an Apple M5 Pro, its three English models process a 60-minute Earnings-22 file at a reported 335 to 480 times real time. The same table reports word error rates from 13.5 to 17.2 percent for that file.[2]
The faster Phonon-2 row has the highest error rate of the three on that recording. The Ultra row is slower and more accurate. That is not a defect. It is the tradeoff a product team needs to see before choosing a model.
FluidAudio also reports much lower word error rates on LibriSpeech test-clean. Clean read speech and an earnings call are different workloads. A single headline can hide that difference.
Transcription and retrieval are separate jobs
The demo uses EmbeddingGemma 2 after transcription. Google's model card says the 740 million parameter model maps text, code, images, video, and audio into one 768-dimensional space. It supports an 8,192-token context and optional 128, 256, and 512-dimensional outputs.[3]
That can make a local audio archive searchable. It does not make the transcript accurate. Speech recognition produces words. An embedding model places content near related content. Test both stages with separate fixtures.
Google warns that 128-dimensional output loses substantial multimodal quality and should be tested on the intended workload. It also warns against float16 because the model can return NaN values or quietly degraded embeddings. Those implementation details belong in the same review packet as search quality.
Run the file you hate
Do not start with a polished audiobook. Use a recording that contains the failures your team will notice. Include two people speaking at once, an internal acronym, a filename, a version number, and a correction spoken midway through a sentence.
Save the reference transcript. Count substitutions, deletions, and insertions. Measure model download, first load, warm runs, peak memory, and output stability. Then run the search queries people will use. A local tool earns a place in the cockpit when it finds the right minute and leaves enough evidence to correct the wrong word.