Hardpoint: text to speech
The tts hardpoint slot 1.0: the built-in qwen3-worker, what a user speech output does (a Wyoming TTS server on loopback: synthesize in, audio-start, audio-chunk, audio-stop out, 16-bit PCM), its probe (one fixed sentence rendered to bytes, never played, timed to first audio against first_text_to_first_audio's p90) and its fallback; catalog/hardpoints/wyoming-kokoro is the first-party entry, bound with [hardpoints] tts = "wyoming-kokoro".
Living doc, checked against the code 2026-11-14
hardpoint.tts.v1 — the tts slot: what a speech-output implementation must do to carry a session’s voice
- Status: 1.0, built. The slot’s interface contract under hardpoint.v1 and ADR 0059 §2. A minor version only adds.
- Owner: the gateway, at session bind. Budget leg:
first_text_to_first_audioin turn-budget.v1 (p50 400 ms, p90 850 ms).
Built-ins
| id | what it is |
|---|---|
qwen3-worker |
the default, the TtsAdapter: the gateway’s own worker streaming Qwen3-TTS from the TTS server just models installs (VULPINE_TTS_HTTP_URL, which just bootstrap writes), or without that server the Python worker in runtime/qwen3_tts_worker under an immutable VULPINE_QWEN_UV |
A built-in qualifies by its worker’s ready receipt at start, so it costs nothing at bind.
What an implementation does
A tts implementation speaks one reply segment at a time. For each request it answers 16-bit little-endian PCM at
the rate it names and then says it is done, or it answers an error. The gateway turns that into the Qwen worker’s
frames: audio_start, audio_chunk with its output PCM, audio_done, error. Playback, barge and the turn budget
see no difference. Requests render one at a time, in order, because the session has one playback queue. A cancel
drops every render in flight.
Transports this build drives
| transport | wire | endpoint |
|---|---|---|
wyoming |
One TCP connection per request. The gateway sends synthesize {text}; the server answers audio-start {rate, width, channels}, any number of audio-chunk (PCM payload) and audio-stop, or error {text, code} (its words are carried, bounded, into the failure). width must be 2. |
tcp://<loopback ip>:<port> |
wyoming, streaming text |
When the server’s describe answers info with supports_synthesize_streaming: true on a TTS program, a reply is one connection: synthesize-start, then one synthesize-chunk {text} per sentence as the reply’s text is cut (each ends in a blank line, which Home Assistant’s sentence detector takes as a sentence end at once), then the compatibility synthesize {text} and synthesize-stop when the reply’s text is complete. The server answers each sentence as its own audio-start .. audio-stop, or the whole stream as one span (xn-ptts: its audio then rides the reply’s first request), and ends with synthesize-stopped. A stream with no event and no new text for 1 s is ended the same way, inside a span too (a later sentence opens a new one); once the stop is sent, 10 s of quiet is a hung server. VULPINE_WYOMING_TTS_STREAMING=0 keeps the plain row. |
the same |
Any other transport is refused transport_not_allowed. The endpoint must be a loopback address: a user
implementation stays local.
Qualification probe
At every bind that names it, the gateway renders the fixed sentence Hardpoint check. through the implementation and
counts the bytes. Nothing is played. The probe passes when audio comes back within 5 s. The measured ms is the time to
the first audio chunk, compared with the leg’s p90. Over it, the implementation stays bound and the Admin marks it
over budget. A failed probe binds the manifest’s fallback (qwen3-worker), reported degraded {reason: qualification_failed}. With fallback = "none" the bind is refused.
Voice check
A server can answer every request with audio and still not speak its configured voice. macOS, for one, renders an Eloquence voice in its fallback voice, Samantha, and reports no error. An entry may name the voices its server falls back to (optional, at most four):
reference_voices = ["alloy"]
After a passed qualification the gateway checks the voice in the background; the session never waits for it.
- It asks the server’s
describefor its configured voice. A server that advertises exactly one voice names it; the OpenAI-speech shim advertises its--voice. - It renders
Hardpoint check.in the configured voice, the way every reply is rendered. - It renders the same sentence in each reference voice of another name (
synthesize {text, voice: {name}}), and stops at the first match.
Two renders match on the same bytes or on the same fingerprint: the same rate, the trimmed length within 1%, and a 32-bin loudness envelope within 2 of 63 per bin. Nothing is played. The Admin’s Hardpoints card shows the outcome under the slot’s state:
| verdict | meaning |
|---|---|
own_voice |
the configured voice differs from every reference voice compared, or is a reference voice itself |
fell_back |
the server names its configured voice, and it renders the same as a reference voice of another name |
same_as_reference |
it renders the same as a reference voice, and the server does not say which voice it is configured with |
unchecked |
no_reference_voices (the entry names none), or render_failed (a render failed, or 20 s passed) |
A voice that is a reference voice under another name reads as fallen back: preowned’s Samantha is alloy. The check
costs one render per voice on the server (about 0.3 s each on preowned), after the bind and off the turn.
Naming the voice
A loadout may name the voice its tts implementation speaks in (optional):
[hardpoints]
tts = "xn-ptts"
tts_voice = "operator"
- On the wire. The gateway sends it as Wyoming’s own
voice: {name}: in the qualification probe (synthesize {text: "Hardpoint check.", voice: {name}}), in every plain reply, in a streamed reply’ssynthesize-startand its compatibilitysynthesize, and in the voice check’s configured render. Withouttts_voicenothing changes: no voice field, the server’s own voice. - Qualification. A server that does not have the voice answers
error(xn-ptts:unknown voice '<name>'). The probe fails with those words and the bind falls back to the declared built-in,degraded {reason: qualification_failed}. - Bounds. 1 to 64 of
[A-Za-z0-9_.-], and only beside a[hardpoints] ttsCatalog implementation. A built-in takes its voice from the persona, so the loadout is refused (voice_invalid). - What it is not. A name, never a path. The voice’s file (xn-ptts:
<name>.safetensorsin--voice-dir) and its pace (--voice-condition <name>:duration_delta=<x>) are the server’s launch settings. A voice file made from someone’s recordings never reaches the gateway, the trace or the Admin. - Readback. The Admin’s bind row carries
voicewhile the implementation carries the slot: the card readsxn-ptts (yours) · voice operator. The voice check reports it asconfigured, also when it has no reference voice to compare.hardpoint-check --voice <name>probes an entry in a voice.
The first-party entry
catalog/hardpoints/wyoming-kokoro/hardpoint.toml names runtime/wyoming-kokoro (kokoro-onnx on 127.0.0.1:10210,
optional). A loadout binds it with:
[hardpoints]
tts = "wyoming-kokoro"
User entries
catalog/hardpoints/xn-ptts/hardpoint.toml is a user entry: xn-ptts’s own ptts-openai-server with
--wyoming-addr 127.0.0.1:10220 --lang en, bound with no shim. The operator builds and runs it; Vulpine does not
install it. A trained Phonon voice file binds through it with tts_voice and a per-voice pace patch, as
its README describes. The server holds a streamed sentence under 20
characters until more text or the stop, so a short first sentence waits up to the 1 s idle window.
VULPINE_WYOMING_TTS_STREAMING=0 speaks it plain, at once.
A server that speaks OpenAI’s POST /v1/audio/speech rather than Wyoming binds through vulpine-openai-speech
(crates/openai_speech_shim). It is a loopback Wyoming server. It asks the server for response_format=pcm, passes
each piece of body on as one audio-chunk the moment it lands, and hangs up on the server when the gateway does. Two
entries use it:
catalog/hardpoints/preowned-speech/hardpoint.toml: macOS system voices, one whole buffer per reply, port 10240.catalog/hardpoints/qwentts-cpp-server/hardpoint.toml: qwentts.cpp’stts-serveron CUDA, Vulkan or the CPU, streamed while the model runs, port 10241.