Reach: voice
A key paired by reach-pair.v1 whose row grants voice. That is a client row: client rows get voice by default, and a row paired without it pairs again to gain it.
Living doc, checked against the code 2026-11-14
reach-voice.v1 — one protocol 5 session over reach as a control stream and Opus datagrams
- Status: accepted; public, built.
publicis its label in the API catalog. - Scope: the bytes between a paired client and the gateway host’s helper on ALPN
vulpine/voice/1. The gateway behind the helper sees an ordinary protocol 5 WebSocket client with 24 kHz PCM16 audio. The helper’s bridging and pacing are described only where a client can observe them. - Schema: reach-voice.v1.schema.json covers every JSON frame of this layer (the hello,
the accept, the refusal and
media_mode). The protocol 5 frames are in frontend.v5.schema.json. - Off by default. The helper serves it only with
[reach] enabled = trueandvoice = true, in a build with thevoicefeature andhelper_wsset. A client that is refused, or meets the ALPN unserved, falls back to reach-ws.v1.
Who may open it
brain and agent-signer rows never hold it,
and no delegation may name it: an agent has no microphone to lend. The accept rule closes any other key before a
byte, with the codes in reach-pair.v1’s accept rule (4 for a row
without voice).
One connection
| part | opened by | carries |
|---|---|---|
| the control stream | the client: its first bidirectional stream | len32 frames: the hello, the answer, then the session’s protocol 5 JSON both ways |
| datagrams | either side | one media frame each: a 12-byte header and one Opus packet |
| media streams | the sender of a direction, when it moves off datagrams | unidirectional; one per direction; u16 length-prefixed media frames |
The session ends when the client finishes the control stream or closes the connection, when the gateway closes its
WebSocket session, or on a control frame that is not UTF-8 text. The helper then closes the connection with code 0.
The control stream
Every frame is a big-endian u32 length and that many bytes of UTF-8 JSON. A reader refuses a frame over 1 MiB
(1,048,576 bytes) from its header alone, before it reads the body.
-
The hello (client, first frame, within 10 seconds):
{"v":1,"codecs":["opus"],"frame_ms":[20,10],"rate_hz":24000,"channels":1,"reference_channel":false,"max_datagram":1200}max_datagramis the client’s datagram limit on this connection (0when it has none). The helper decidesmediafrom the connection’s own limit, not from this member. With no stream, or no frame, within the 10 seconds the helper closes the connection with code1and sends no answer. -
The answer (helper, one frame). Either an accept:
{"v":1,"codec":"opus","frame_ms":20,"uplink_kbps":24,"reply_kbps":32,"fec":true,"epoch":0,"media":"datagram"}mediaisdatagram, orstreamwhen the connection cannot carry datagrams of at least 128 bytes.epochis the reply epoch the session starts at. Or a refusal. The helper finishes its side of the stream and waits up to 10 seconds for the client to close, then closes the connection with code1:{"v":1,"refuse":"audio-codec"}word why hellothe first frame is not JSON, or not a hello versionvis not1audio-codecno opusincodecs, orrate_hzis not 24000, orchannelsis not 1frameframe_msdoes not offer 20gatewaythe helper could not open its loopback session to the gateway within 3 seconds grantreserved; the helper never sends it: a missing grant is refused at the handshake with code 4A client falls back to
vulpine/ws/1on any refusal, including a word it does not know. -
The session. Each later frame is one protocol 5 JSON frame, exactly the text a WebSocket text frame would carry, in the same order, both ways. The gateway’s
helloarrives first. The client’s protocol 5 hello must offerinput_pcmandoutput_pcmat 24000 Hz, mono, in 20 ms frames, because the helper decodes the mic to and encodes the reply from 24 kHz mono PCM16. Audio never rides the control stream: protocol 5’s binary PCM frames become media frames (below).
The frame rule: whose frame is it
The control stream carries two kinds of JSON after the answer. A receiver sorts each frame:
- A frame with a top-level
"type"member is protocol 5, whatever else it holds. It is carried byte for byte.start_sessionnames the TTS voice in a top-level"voice"member (a string ornull) and is still protocol 5. - Otherwise, a frame with a top-level
"voice"member is the voice layer’s own. It is never passed to the gateway or to a loopback client. The one such frame in v1 ismedia_mode. A voice-layer frame this build does not know is dropped. - Anything else is carried as protocol 5.
Voice-layer frames never carry "type". An older helper lacks rule 1 and drops a start_session that names a voice,
even as null. A client that must also work with such a helper leaves voice out of start_session, as the phone’s
link does.
media_mode
{"voice":"media_mode","dir":"down","to":"stream","seq":1234}
The receiver of a direction whose datagrams stopped asks the sender to move that direction (dir: up is the mic,
down the reply) onto its media stream (to: stream; datagram is reserved). seq is the first frame the receiver
lacks. The sender resends what it still holds from there (the last 32 frames, 640 ms) and carries on from its next
seq on the stream. Datagrams are not tried again for that direction within the session. A sender also moves by
itself when a datagram send fails for any reason but the datagram being too large. Vulpine’s own client asks after
100 ms without a reply datagram inside a live reply; the helper asks after 300 ms without a mic datagram.
Media frames
A datagram is one media frame: the header, then one Opus packet of 1 to 1,275 bytes. A frame that is shorter, or that
breaks a rule below, is dropped as if lost. On a media stream each frame is a big-endian u16 length, then the same
header and packet.
| bytes | field | |
|---|---|---|
| 0 | kind |
1 mic (client to helper), 2 the mic’s echo reference (client to helper), 3 reply (helper to client) |
| 1 | flags |
bit 0 last (the last frame of this epoch), bit 1 fec (the packet carries in-band FEC), bit 2 dtx. Any other bit set: the frame is dropped |
| 2-3 | epoch |
big-endian u16. Serial arithmetic (RFC 1982): wraps after 65,535 |
| 4-7 | seq |
big-endian u32, per direction and kind, never reset within a connection |
| 8-11 | media_ts |
big-endian u32: samples at 24 kHz from the epoch’s start. Set it; v1 receivers do not depend on it |
The codec. Opus, 24 kHz, mono, 20 ms frames (480 samples), in-band FEC on. The helper encodes the reply at 32 kbit/s, and Vulpine’s client encodes the mic at 24 kbit/s (the accept names both). Each Opus decoder starts 156 samples (6.5 ms) late: a receiver that rebuilds PCM drops that lookahead at each epoch’s start.
Epochs. A receiver keeps a fence per kind. A frame of an older epoch than the fence is dropped on arrival and is not a gap. A newer epoch raises the fence: the first frame of a new epoch may beat the control frame that announces it.
- The reply epoch is the helper’s. It bumps on every protocol 5
turn_startedandaudio_reset. Onaudio_resetthe helper drops every reply frame it still holds before it writes the reset on the control stream, so no frame of the cut reply is sent after it. The client’s fence drops the ones that raced it. - The mic epoch is the client’s. It bumps it on every barge-in. The helper starts the new epoch clean.
The uplink. The helper decodes each mic frame the moment it is next in seq. Only on a gap does it wait, at most
20 ms, for the missing frame or the next frame’s FEC. Then it rebuilds or conceals the gap and moves on. A hole over
50 frames (1 s) resyncs without filling. Kind 2 is fenced and counted; the helper does not pass it to the gateway.
The reply. The helper still sends each protocol 5 audio_chunk event on the control stream, so a client can cut
the decoded reply back into the same chunks and emit them in the gateway’s order. The chunk’s samples arrive as reply
frames under the reply epoch. Only turn_audio_completed ends that audio: it pads the last partial frame and adds
one silent frame flagged last. turn_completed finishes text generation, not speech; carry it unchanged without
ending the reply encoder or the client’s jitter epoch. More audio_chunk frames can follow it. The helper paces the
reply: the first 200 ms of an epoch at once, then 1.25 times real time, never more than 400 ms ahead of real time.
What Vulpine’s own client does (not normative)
vulpine-reach voice relay serves one loopback WebSocket session to an unchanged Client Core. The phone (phone_core)
does the same in process. Its downlink jitter buffer starts at 40 ms on a direct path and 60 ms relayed, stays between
20 ms and 120 ms (200 ms relayed), grows one frame at once on a late frame and shrinks at most one frame a second. It
writes one voice_media trace row per session and one media_mode row per switch: closed words and counts, never
audio, a word, an address or a key.
Compatibility
- A change to the hello, the answer, the header, the flags or the frame rule takes
vulpine/voice/2. So does a new voice-layer frame a v1 peer would need to understand: v1 peers drop the ones they do not know. - The
vmember is 1 for this ALPN. An unknownvis refused (version). - A new refusal word is safe: a client falls back on any word.
See also
- API surfaces: this ALPN’s row and its stability label.
- ADR 0053.
- runtime/reach: the helper’s switches and rollback.