Skip to content
Vulpine
Esc
↑↓navigate↵open⌘Jpreview
On this page

Speech recognition

The speech to text that runs on the Apple Neural Engine: a Rust runner for Parakeet TDT 0.6B v3 through CoreML, the encoder on the Neural Engine and the decode loop on the CPU.

Living doc, checked against the code 2026-11-14

coreml_stt — Parakeet TDT 0.6B v3 on the Apple Neural Engine

A clean-room Rust runner for Parakeet TDT 0.6B v3 through Apple CoreML. The acoustic encoder runs on the Neural Engine and the transducer decode loop on the CPU. It uses no GPU ops, so STT does not contend with TTS and end-of-utterance detection for the GPU during barge-in.

use coreml_stt::{ComputeUnits, Recognizer};

let mut rec = Recognizer::load(bundle_dir, ComputeUnits::CpuAndNeuralEngine)?;
let out = rec.transcribe(&pcm_16k_mono_f32)?;
println!("{}", out.text);
for t in &out.tokens {
    println!("{:.2}-{:.2} {:?}", t.start_s, t.end_s, t.text);
}

CLI:

fox9-coreml-stt --bundle <dir> transcribe <wav...> [--json]
fox9-coreml-stt --bundle <dir> serve
fox9-coreml-stt --bundle <dir> partials <wav...> [--interval-ms 500] [--first-tick-ms N] [--texts] [--json]
fox9-coreml-stt --bundle <dir> placement
fox9-coreml-stt --bundle <dir> bench --clip <wav> --warm 5

--compute-units cpu_only|cpu_and_neural_engine|cpu_and_gpu|all goes before the command.

  • transcribe --no-seam-gap-repair turns gap repair off.
  • --silence-aligned-windows starts each window at the quietest frame near the hop.
  • --zero-token-retry re-decodes a window that reads nothing over speech as two halves (off by default).
  • partials replays a clip as an open segment and commits words the way commit.rs does.
  • --first-tick-ms places the first tick. The live segmenter ticks first at pre-roll + its 600 ms minimum segment.
  • --texts adds three things to the --json row: each tick’s committed prefix, the first non-empty one (the first partial a router reads) and the final text.

serve (src/serve.rs) loads the bundle once, then decodes JSON-line requests on stdin: {id, wav_path | pcm_f32le_path, sample_rate}.

  • A {"op":"ready","protocol":1,"load_ms":…} line comes first.
  • Each request gets one line, in order: {id, text, words[{w, t0, t1}], decode_ms, audio_s}, or {id, error{kind, message}} with kind one of bad_request, unsupported_sample_rate, audio_unreadable, decode_failed.
  • {"op":"quit"} or end of input ends it.

The decode is transcribe’s, with gap repair on. meetctl (tools/meet) decodes a call’s windows on one serve child per run by default.

Bundle it expects

Recognizer::load takes either the bundle root (containing parakeet-v3/) or parakeet-v3/ itself:

file in out
preprocessor.mlmodelc audio_signal [1,240000], audio_length [1] mel [1,128,1501], mel_length [1]
encoder.mlmodelc mel, mel_length encoder [1,1024,188], encoder_length [1]
decoder.mlmodelc targets [1,1], target_length [1], h_in/c_in [2,1,640] decoder [1,640,1], h_out, c_out
joint-decision.mlmodelc encoder_step [1,1024,1], decoder_step [1,640,1] token_id, token_prob, duration
parakeet_vocab.json — {"<id>": "<piece>"} × 8192

Set VULPINE_COREML_STT_ENCODER to swap the encoder file (e.g. encoder-lut6.mlmodelc) without a rebuild.

The vocabulary has 8192 entries and no explicit blank token: blank is index 8192 (Vocab::blank_id returns len). A len - 1 blank silently corrupts every decode.

The front end is CoreML too. preprocessor.mlmodelc maps raw 16 kHz audio straight to the encoder’s [1,128,1501] mel input, so there is no hand-written mel to disagree with the encoder’s training. The upstream repo’s MelEncoder.mlmodelc is a fused mel+encoder (595 MB), not a front end; this crate does not use it.

Weights license — attribution required

The models are not part of this crate and are not vendored. They come from FluidInference/parakeet-tdt-0.6b-v3-coreml under CC-BY-4.0, converted from nvidia/parakeet-tdt-0.6b-v3, also CC-BY-4.0.

Speech recognition uses Parakeet TDT 0.6B v3 by NVIDIA, licensed CC-BY-4.0, converted to CoreML by Fluid Inference.

That attribution must travel with anything that ships or serves these weights. The upstream README’s prose says “Apache 2.0”, but its model-card front matter, its HF tag and the base model say cc-by-4.0. Treat CC-BY-4.0 as binding. The crate’s own code is MIT, like the rest of the workspace.

Measured numbers

On an M3 MacBook Air (16 GB, cpu_and_neural_engine, the int8 Encoder_v2), 1376 of the encoder’s 3398 ops run on the ANE and none on the GPU. Pooled WER over 140 reference clips is 10.07 %, and peak host RSS is 0.074 GB.

Timestamps

TokenSpan::end_s is start_s + duration × 80 ms, the model’s own predicted duration for that token. A span ends where the token ends, so blank frames are left out; it does not run to the next emission. One encoder frame is 80 ms (10 ms mel hop × 8 subsampling).

Long form

The encoder’s window is a fixed 240 000 samples (15 s). Longer audio is cut into 15 s windows overlapping by 1 s. At each seam the two windows are stitched on the longest token run in the overlap whose clocks agree within 0.5 s, so a word straddling the cut survives exactly once. With no agreeing run, the earlier window owns audio up to the midpoint of the overlap. Gap repair is on by default: any gap over 1.5 s between tokens is decoded again in a fresh window centered on it. chunk.rs holds the rules.

Caveats

  • Host RSS is not the whole cost. 0.074 GB is the host process, without CoreML/ANE weight residency. Budget ~0.7 GB for the system, and budget the first-compile peak, not the steady state.
  • The first ever load pays a full ANE compile. CoreML caches compiled models per executable name under ~/Library/Caches/<executable>/com.apple.e5rt.e5bundlecache/<os build>/, so a rename or an OS update re-pays it. Pre-warm at deploy, never at the first utterance.
  • macOS only; the library is a no-op stub elsewhere.

Was this page helpful?