The spoken voice
Swap Vulpine's spoken voice, add a mood voice, and roll back. just models installs the qwentts.cpp tts-server on :8850 with qwen-talker-0.6b-base-Q8_0 (tools/install/models.toml); the gateway registers its voice there as operator, ICL.
Living doc, checked against the code 2026-11-14
Which clip is live is host state; the re-check below reads it.
The path
The gateway runs runtime/qwen3_tts_worker/qwen3_tts_worker.py and hands it the sealed voice reference (WAV +
transcript) as inherited descriptors (ws/adapters.rs, build_tts_adapter; ARCHITECTURE’s TTS section). The worker
has two modes behind the same frame protocol:
- qwentts-http (with the installed server). With
VULPINE_TTS_HTTP_URLset, the gateway passes--http-urland--http-voice(VULPINE_TTS_HTTP_VOICE, defaultoperator). The worker health-checks the server (GET /health), registers the sealed reference under that name (POST /v1/audio/voices), registers the mood bank in the background, and streams PCM from the server. It loads no model. - MLX (the fallback). Without
VULPINE_TTS_HTTP_URLthe worker loads Qwen3-TTS through MLX in-process and clones the sealed reference per request.
The server’s voice registry is memory-only; the worker recovers on its own:
- On
unknown voicefor the default voice, it re-registers the sealed reference and retries once. - A mood voice that answers empty or unknown is re-registered, or the request falls back to the default voice. Then the rest of the bank refills. Mood never fails a turn.
- Bank registrations wait while any worker on the same server streams a reply, and for 1.5 s after, so a refill
never lands inside a reply. The workers share a hold directory per server under
~/.local/state/foxline/tts-stream-hold/(VULPINE_TTS_STREAM_HOLD_DIRoverrides it).
Knobs
All are environment variables the gateway reads at start, set in <state>/gateway.env (docs/install.md), so a
change takes a gateway restart. A roll stages them from its own overlay.env (docs/release.md).
| variable | effect |
|---|---|
VULPINE_TTS_HTTP_URL |
set: qwentts-http mode against that server; unset or empty: the MLX worker |
VULPINE_TTS_HTTP_VOICE |
the name the sealed reference registers under (default operator) |
VULPINE_TTS_EMOBANK_DIR |
the mood bank directory (worker default ~/Library/Application Support/Foxline/voice/emobank) |
VULPINE_TTS_MOOD_VOICES |
mood=voice pairs (wry=<voice>-wry,dark=<voice>-dark); unset turns mood selection off (mood.rs) |
VULPINE_TTS_TRUNCATION_RERENDER |
1/true/on/yes: the worker buffers each segment before it plays. It re-renders once a clip shorter than 0.65 × its text’s expected duration (texts of 20+ characters, at 17.5 chars/s, the live voice’s measured rate: benchmarks/results/truncation-recalibration-20261002.md). A second short render plays as it is. Default off: the buffering adds the whole render time to every segment’s first audio. Off or on, a cut-off clip is traced as tts_render_truncated (retried, outcome: streamed / retry_full / retry_short / retry_failed) |
VULPINE_TTS_REF_AUDIO + VULPINE_TTS_REF_TEXT_FILE |
the deployment voice override: both or neither (one alone refuses the bind). Without them the gateway uses the persona package’s manifest voice |
Swap the operator voice
The gateway resolves the voice reference when a session binds and seals that copy into the session’s worker:
- Put the new clip and its exact transcript where the override points (
reference.wav/reference.txt). Follow the candidate rules below. Keep the old pair beside it as a dated backup. - The next session uses it. No build and no roll.
- For sessions already open, re-register by hand on the gateway host; it replaces the named entry from the next request:
curl -X POST localhost:8850/v1/audio/voices -d '{"name":"operator","ref_text":"…","wav_b64":"…"}'. This lasts only until a worker re-registers its own sealed copy (a new session, or recovery after a server restart). Step 1 is the swap; step 3 only bridges it.
Rollback is the same move with the backup pair.
Add or replace a mood voice
A mood voice is <name>.wav + <name>.txt (the transcript) in the emobank directory. The worker registers every pair
at startup, after the sealed reference, and again when a voice goes missing. It skips the name equal to
VULPINE_TTS_HTTP_VOICE, which the sealed reference owns. The brain opens a reply with [mood:<tag>], and
VULPINE_TTS_MOOD_VOICES maps the tag to a voice name. An unknown tag is stripped and the reply speaks in the
default voice. Ship a new clip under a new name and remap VULPINE_TTS_MOOD_VOICES (a gateway restart); never overwrite a
live name.
Every bank clip runs the intake gate first: python3 tools/emobank/gate.py <dir-or-wav> (8–15 s, floor ≤ −38 dBFS,
no clipping, speech/noise contrast ≥ 14 dB, pauses present; exit 1 on any failure). The worker re-runs the same
checks at registration. A failing clip still registers, so the palette never vanishes, and shows on the Admin panel
as a reference_gate event.
Candidate rules
- One speaker, one channel. Segment every candidate with an ASR pass first: a two-speaker clip clones both. Clean runs from one speaker, session and register hold identity; noisy cross-scene montages do not. Cut at phrase boundaries, never at a time cap, and fade the outer edges.
- Everything in the reference transfers: channel static, sibilance (de-ess the reference, not the output), the character’s physical state. Pick clean, quiet, healthy takes.
- ICL with a clean reference is the default. x-vector-only registration (no
ref_text) measured worse. - A palette candidate passes
tools/emobank/gate.py, has phrase-final fry ≤ 0.1, runs ≥ 10 s for a reliable cosine, and matches the live clip’s identity by ear. Mood renders are gated rank-only with a hard floor at 0.45. The 0.80 bar applies to plain-register renders only. Never raise the floor “to be safe”. - Gate before ears: no render reaches the operator before the ASR and speaker-similarity gates pass.
- Licensing: reference clips cut from game audio are for private local use on the operator’s own hardware only. Never commit the audio and never distribute it.
Re-check
What is live on the gateway host (read-only; run from another Mac):
ssh <gateway-host> 'grep -hE "^(VULPINE|FOXLINE)_TTS" ~/.local/state/foxline/overlay.env.mirror ~/.local/state/foxline/gateway.env 2>/dev/null; curl -s localhost:8850/health; echo; ls ~/Library/Application\ Support/Foxline/voice/emobank | head'
A session’s own trace names what it registered: http_voice_registered (the sealed reference, with its byte count),
http_emobank_registered (the bank: count, how many failed the gate, why it ran) and one http_emobank_voice_gate
per clip that failed the gate.