---
title: "Speech recognition"
description: "The speech to text that runs on the Apple Neural Engine: a Rust runner for Parakeet TDT 0.6B v3 through CoreML, the encoder on the Neural Engine and the decode loop on the CPU."
type: "living"
source: "crates/coreml_stt/README.md"
verified: "2026-11-14"
group: "The voice path"
---

> **Living doc**, checked against the code 2026-11-14

`coreml_stt` — Parakeet TDT 0.6B v3 on the Apple Neural Engine

A clean-room Rust runner for **Parakeet TDT 0.6B v3** through Apple CoreML.
The acoustic encoder runs on the Neural Engine and the transducer decode loop
on the CPU. It uses no GPU ops, so STT does not contend with TTS and
end-of-utterance detection for the GPU during barge-in.

```rust
use coreml_stt::{ComputeUnits, Recognizer};

let mut rec = Recognizer::load(bundle_dir, ComputeUnits::CpuAndNeuralEngine)?;
let out = rec.transcribe(&pcm_16k_mono_f32)?;
println!("{}", out.text);
for t in &out.tokens {
    println!("{:.2}-{:.2} {:?}", t.start_s, t.end_s, t.text);
}
```

CLI:

```sh
fox9-coreml-stt --bundle <dir> transcribe <wav...> [--json]
fox9-coreml-stt --bundle <dir> serve
fox9-coreml-stt --bundle <dir> partials <wav...> [--interval-ms 500] [--first-tick-ms N] [--texts] [--json]
fox9-coreml-stt --bundle <dir> placement
fox9-coreml-stt --bundle <dir> bench --clip <wav> --warm 5
```

`--compute-units cpu_only|cpu_and_neural_engine|cpu_and_gpu|all` goes before
the command.

- `transcribe --no-seam-gap-repair` turns gap repair off.
- `--silence-aligned-windows` starts each window at the quietest frame near
  the hop.
- `--zero-token-retry` re-decodes a window that reads nothing over speech as
  two halves (off by default).
- `partials` replays a clip as an open segment and commits words the way
  `commit.rs` does.
- `--first-tick-ms` places the first tick. The live segmenter ticks first at
  pre-roll + its 600 ms minimum segment.
- `--texts` adds three things to the `--json` row: each tick's committed
  prefix, the first non-empty one (the first partial a router reads) and the
  final text.

`serve` (`src/serve.rs`) loads the bundle once, then decodes JSON-line
requests on stdin: `{id, wav_path | pcm_f32le_path, sample_rate}`.

- A `{"op":"ready","protocol":1,"load_ms":…}` line comes first.
- Each request gets one line, in order: `{id, text, words[{w, t0, t1}],
  decode_ms, audio_s}`, or `{id, error{kind, message}}` with `kind` one of
  `bad_request`, `unsupported_sample_rate`, `audio_unreadable`,
  `decode_failed`.
- `{"op":"quit"}` or end of input ends it.

The decode is `transcribe`'s, with gap repair on. `meetctl` (tools/meet)
decodes a call's windows on one `serve` child per run by default.


## Bundle it expects

`Recognizer::load` takes either the bundle root (containing `parakeet-v3/`) or
`parakeet-v3/` itself:

| file | in | out |
|---|---|---|
| `preprocessor.mlmodelc` | `audio_signal [1,240000]`, `audio_length [1]` | `mel [1,128,1501]`, `mel_length [1]` |
| `encoder.mlmodelc` | `mel`, `mel_length` | `encoder [1,1024,188]`, `encoder_length [1]` |
| `decoder.mlmodelc` | `targets [1,1]`, `target_length [1]`, `h_in`/`c_in [2,1,640]` | `decoder [1,640,1]`, `h_out`, `c_out` |
| `joint-decision.mlmodelc` | `encoder_step [1,1024,1]`, `decoder_step [1,640,1]` | `token_id`, `token_prob`, `duration` |
| `parakeet_vocab.json` | — | `{"<id>": "<piece>"}` × 8192 |

Set `VULPINE_COREML_STT_ENCODER` to swap the encoder file (e.g.
`encoder-lut6.mlmodelc`) without a rebuild.

The vocabulary has 8192 entries and no explicit blank token: blank is index
8192 (`Vocab::blank_id` returns `len`). A `len - 1` blank silently corrupts
every decode.

**The front end is CoreML too.** `preprocessor.mlmodelc` maps raw 16 kHz
audio straight to the encoder's `[1,128,1501]` mel input, so there is no
hand-written mel to disagree with the encoder's training. The upstream repo's
`MelEncoder.mlmodelc` is a *fused* mel+encoder (595 MB), not a front end; this
crate does not use it.

### Weights license — attribution required

The models are **not** part of this crate and are not vendored. They come from
[`FluidInference/parakeet-tdt-0.6b-v3-coreml`](https://huggingface.co/FluidInference/parakeet-tdt-0.6b-v3-coreml)
under **CC-BY-4.0**, converted from
[`nvidia/parakeet-tdt-0.6b-v3`](https://huggingface.co/nvidia/parakeet-tdt-0.6b-v3),
also **CC-BY-4.0**.

> Speech recognition uses Parakeet TDT 0.6B v3 by **NVIDIA**, licensed
> CC-BY-4.0, converted to CoreML by Fluid Inference.

That attribution must travel with anything that ships or serves these weights.
The upstream README's prose says "Apache 2.0", but its model-card front
matter, its HF tag and the base model say `cc-by-4.0`. Treat CC-BY-4.0 as
binding. The crate's own code is MIT, like the rest of the workspace.

## Measured numbers

On an M3 MacBook Air (16 GB, `cpu_and_neural_engine`, the int8 `Encoder_v2`),
1376 of the encoder's 3398 ops run on the ANE and none on the GPU. Pooled WER
over 140 reference clips is 10.07 %, and peak host RSS is 0.074 GB.


### Timestamps

`TokenSpan::end_s` is `start_s + duration × 80 ms`, the model's *own*
predicted duration for that token. A span ends where the token ends, so blank
frames are left out; it does not run to the next emission. One encoder frame
is 80 ms (10 ms mel hop × 8 subsampling).

## Long form

The encoder's window is a fixed 240 000 samples (15 s). Longer audio is cut
into 15 s windows overlapping by 1 s. At each seam the two windows are
stitched on the longest token run in the overlap whose clocks agree within
0.5 s, so a word straddling the cut survives exactly once. With no agreeing
run, the earlier window owns audio up to the midpoint of the overlap. Gap
repair is on by default: any gap over 1.5 s between tokens is decoded again in
a fresh window centered on it. `chunk.rs` holds the rules.


## Caveats

* **Host RSS is not the whole cost.** 0.074 GB is the host process, without
  CoreML/ANE weight residency. Budget \~0.7 GB for the system, and budget the
  **first-compile peak**, not the steady state.
* **The first ever load pays a full ANE compile.** CoreML caches compiled
  models per *executable name* under
  `~/Library/Caches/<executable>/com.apple.e5rt.e5bundlecache/<os build>/`, so
  a rename or an OS update re-pays it. Pre-warm at deploy, never at the first
  utterance.
* macOS only; the library is a no-op stub elsewhere.
