Skip to content
Vulpine
Esc
↑↓navigate↵open⌘Jpreview
On this page

Reach: voice

A key paired by reach-pair.v1 whose row grants voice. That is a client row: client rows get voice by default, and a row paired without it pairs again to gain it.

Living doc, checked against the code 2026-11-14

reach-voice.v1 — one protocol 5 session over reach as a control stream and Opus datagrams

  • Status: accepted; public, built. public is its label in the API catalog.
  • Scope: the bytes between a paired client and the gateway host’s helper on ALPN vulpine/voice/1. The gateway behind the helper sees an ordinary protocol 5 WebSocket client with 24 kHz PCM16 audio. The helper’s bridging and pacing are described only where a client can observe them.
  • Schema: reach-voice.v1.schema.json covers every JSON frame of this layer (the hello, the accept, the refusal and media_mode). The protocol 5 frames are in frontend.v5.schema.json.
  • Off by default. The helper serves it only with [reach] enabled = true and voice = true, in a build with the voice feature and helper_ws set. A client that is refused, or meets the ALPN unserved, falls back to reach-ws.v1.

Who may open it

brain and agent-signer rows never hold it, and no delegation may name it: an agent has no microphone to lend. The accept rule closes any other key before a byte, with the codes in reach-pair.v1’s accept rule (4 for a row without voice).

One connection

part opened by carries
the control stream the client: its first bidirectional stream len32 frames: the hello, the answer, then the session’s protocol 5 JSON both ways
datagrams either side one media frame each: a 12-byte header and one Opus packet
media streams the sender of a direction, when it moves off datagrams unidirectional; one per direction; u16 length-prefixed media frames

The session ends when the client finishes the control stream or closes the connection, when the gateway closes its WebSocket session, or on a control frame that is not UTF-8 text. The helper then closes the connection with code 0.

The control stream

Every frame is a big-endian u32 length and that many bytes of UTF-8 JSON. A reader refuses a frame over 1 MiB (1,048,576 bytes) from its header alone, before it reads the body.

  1. The hello (client, first frame, within 10 seconds):

    {"v":1,"codecs":["opus"],"frame_ms":[20,10],"rate_hz":24000,"channels":1,"reference_channel":false,"max_datagram":1200}

    max_datagram is the client’s datagram limit on this connection (0 when it has none). The helper decides media from the connection’s own limit, not from this member. With no stream, or no frame, within the 10 seconds the helper closes the connection with code 1 and sends no answer.

  2. The answer (helper, one frame). Either an accept:

    {"v":1,"codec":"opus","frame_ms":20,"uplink_kbps":24,"reply_kbps":32,"fec":true,"epoch":0,"media":"datagram"}

    media is datagram, or stream when the connection cannot carry datagrams of at least 128 bytes. epoch is the reply epoch the session starts at. Or a refusal. The helper finishes its side of the stream and waits up to 10 seconds for the client to close, then closes the connection with code 1:

    {"v":1,"refuse":"audio-codec"}
    word why
    hello the first frame is not JSON, or not a hello
    version v is not 1
    audio-codec no opus in codecs, or rate_hz is not 24000, or channels is not 1
    frame frame_ms does not offer 20
    gateway the helper could not open its loopback session to the gateway within 3 seconds
    grant reserved; the helper never sends it: a missing grant is refused at the handshake with code 4

    A client falls back to vulpine/ws/1 on any refusal, including a word it does not know.

  3. The session. Each later frame is one protocol 5 JSON frame, exactly the text a WebSocket text frame would carry, in the same order, both ways. The gateway’s hello arrives first. The client’s protocol 5 hello must offer input_pcm and output_pcm at 24000 Hz, mono, in 20 ms frames, because the helper decodes the mic to and encodes the reply from 24 kHz mono PCM16. Audio never rides the control stream: protocol 5’s binary PCM frames become media frames (below).

The frame rule: whose frame is it

The control stream carries two kinds of JSON after the answer. A receiver sorts each frame:

  1. A frame with a top-level "type" member is protocol 5, whatever else it holds. It is carried byte for byte. start_session names the TTS voice in a top-level "voice" member (a string or null) and is still protocol 5.
  2. Otherwise, a frame with a top-level "voice" member is the voice layer’s own. It is never passed to the gateway or to a loopback client. The one such frame in v1 is media_mode. A voice-layer frame this build does not know is dropped.
  3. Anything else is carried as protocol 5.

Voice-layer frames never carry "type". An older helper lacks rule 1 and drops a start_session that names a voice, even as null. A client that must also work with such a helper leaves voice out of start_session, as the phone’s link does.

media_mode

{"voice":"media_mode","dir":"down","to":"stream","seq":1234}

The receiver of a direction whose datagrams stopped asks the sender to move that direction (dir: up is the mic, down the reply) onto its media stream (to: stream; datagram is reserved). seq is the first frame the receiver lacks. The sender resends what it still holds from there (the last 32 frames, 640 ms) and carries on from its next seq on the stream. Datagrams are not tried again for that direction within the session. A sender also moves by itself when a datagram send fails for any reason but the datagram being too large. Vulpine’s own client asks after 100 ms without a reply datagram inside a live reply; the helper asks after 300 ms without a mic datagram.

Media frames

A datagram is one media frame: the header, then one Opus packet of 1 to 1,275 bytes. A frame that is shorter, or that breaks a rule below, is dropped as if lost. On a media stream each frame is a big-endian u16 length, then the same header and packet.

bytes field
0 kind 1 mic (client to helper), 2 the mic’s echo reference (client to helper), 3 reply (helper to client)
1 flags bit 0 last (the last frame of this epoch), bit 1 fec (the packet carries in-band FEC), bit 2 dtx. Any other bit set: the frame is dropped
2-3 epoch big-endian u16. Serial arithmetic (RFC 1982): wraps after 65,535
4-7 seq big-endian u32, per direction and kind, never reset within a connection
8-11 media_ts big-endian u32: samples at 24 kHz from the epoch’s start. Set it; v1 receivers do not depend on it

The codec. Opus, 24 kHz, mono, 20 ms frames (480 samples), in-band FEC on. The helper encodes the reply at 32 kbit/s, and Vulpine’s client encodes the mic at 24 kbit/s (the accept names both). Each Opus decoder starts 156 samples (6.5 ms) late: a receiver that rebuilds PCM drops that lookahead at each epoch’s start.

Epochs. A receiver keeps a fence per kind. A frame of an older epoch than the fence is dropped on arrival and is not a gap. A newer epoch raises the fence: the first frame of a new epoch may beat the control frame that announces it.

  • The reply epoch is the helper’s. It bumps on every protocol 5 turn_started and audio_reset. On audio_reset the helper drops every reply frame it still holds before it writes the reset on the control stream, so no frame of the cut reply is sent after it. The client’s fence drops the ones that raced it.
  • The mic epoch is the client’s. It bumps it on every barge-in. The helper starts the new epoch clean.

The uplink. The helper decodes each mic frame the moment it is next in seq. Only on a gap does it wait, at most 20 ms, for the missing frame or the next frame’s FEC. Then it rebuilds or conceals the gap and moves on. A hole over 50 frames (1 s) resyncs without filling. Kind 2 is fenced and counted; the helper does not pass it to the gateway.

The reply. The helper still sends each protocol 5 audio_chunk event on the control stream, so a client can cut the decoded reply back into the same chunks and emit them in the gateway’s order. The chunk’s samples arrive as reply frames under the reply epoch. Only turn_audio_completed ends that audio: it pads the last partial frame and adds one silent frame flagged last. turn_completed finishes text generation, not speech; carry it unchanged without ending the reply encoder or the client’s jitter epoch. More audio_chunk frames can follow it. The helper paces the reply: the first 200 ms of an epoch at once, then 1.25 times real time, never more than 400 ms ahead of real time.

What Vulpine’s own client does (not normative)

vulpine-reach voice relay serves one loopback WebSocket session to an unchanged Client Core. The phone (phone_core) does the same in process. Its downlink jitter buffer starts at 40 ms on a direct path and 60 ms relayed, stays between 20 ms and 120 ms (200 ms relayed), grows one frame at once on a late frame and shrinks at most one frame a second. It writes one voice_media trace row per session and one media_mode row per switch: closed words and counts, never audio, a word, an address or a key.

Compatibility

  • A change to the hello, the answer, the header, the flags or the frame rule takes vulpine/voice/2. So does a new voice-layer frame a v1 peer would need to understand: v1 peers drop the ones they do not know.
  • The v member is 1 for this ALPN. An unknown v is refused (version).
  • A new refusal word is safe: a client falls back on any word.

See also

  • API surfaces: this ALPN’s row and its stability label.
  • ADR 0053.
  • runtime/reach: the helper’s switches and rollback.

Was this page helpful?