Skip to content
Vulpine
Esc
↑↓navigate↵open⌘Jpreview
On this page

The spoken voice

Swap Vulpine's spoken voice, add a mood voice, and roll back. just models installs the qwentts.cpp tts-server on :8850 with qwen-talker-0.6b-base-Q8_0 (tools/install/models.toml); the gateway registers its voice there as operator, ICL.

Living doc, checked against the code 2026-11-14

Which clip is live is host state; the re-check below reads it.

The path

The gateway runs runtime/qwen3_tts_worker/qwen3_tts_worker.py and hands it the sealed voice reference (WAV + transcript) as inherited descriptors (ws/adapters.rs, build_tts_adapter; ARCHITECTURE’s TTS section). The worker has two modes behind the same frame protocol:

  • qwentts-http (with the installed server). With VULPINE_TTS_HTTP_URL set, the gateway passes --http-url and --http-voice (VULPINE_TTS_HTTP_VOICE, default operator). The worker health-checks the server (GET /health), registers the sealed reference under that name (POST /v1/audio/voices), registers the mood bank in the background, and streams PCM from the server. It loads no model.
  • MLX (the fallback). Without VULPINE_TTS_HTTP_URL the worker loads Qwen3-TTS through MLX in-process and clones the sealed reference per request.

The server’s voice registry is memory-only; the worker recovers on its own:

  • On unknown voice for the default voice, it re-registers the sealed reference and retries once.
  • A mood voice that answers empty or unknown is re-registered, or the request falls back to the default voice. Then the rest of the bank refills. Mood never fails a turn.
  • Bank registrations wait while any worker on the same server streams a reply, and for 1.5 s after, so a refill never lands inside a reply. The workers share a hold directory per server under ~/.local/state/foxline/tts-stream-hold/ (VULPINE_TTS_STREAM_HOLD_DIR overrides it).

Knobs

All are environment variables the gateway reads at start, set in <state>/gateway.env (docs/install.md), so a change takes a gateway restart. A roll stages them from its own overlay.env (docs/release.md).

variable effect
VULPINE_TTS_HTTP_URL set: qwentts-http mode against that server; unset or empty: the MLX worker
VULPINE_TTS_HTTP_VOICE the name the sealed reference registers under (default operator)
VULPINE_TTS_EMOBANK_DIR the mood bank directory (worker default ~/Library/Application Support/Foxline/voice/emobank)
VULPINE_TTS_MOOD_VOICES mood=voice pairs (wry=<voice>-wry,dark=<voice>-dark); unset turns mood selection off (mood.rs)
VULPINE_TTS_TRUNCATION_RERENDER 1/true/on/yes: the worker buffers each segment before it plays. It re-renders once a clip shorter than 0.65 × its text’s expected duration (texts of 20+ characters, at 17.5 chars/s, the live voice’s measured rate: benchmarks/results/truncation-recalibration-20261002.md). A second short render plays as it is. Default off: the buffering adds the whole render time to every segment’s first audio. Off or on, a cut-off clip is traced as tts_render_truncated (retried, outcome: streamed / retry_full / retry_short / retry_failed)
VULPINE_TTS_REF_AUDIO + VULPINE_TTS_REF_TEXT_FILE the deployment voice override: both or neither (one alone refuses the bind). Without them the gateway uses the persona package’s manifest voice

Swap the operator voice

The gateway resolves the voice reference when a session binds and seals that copy into the session’s worker:

  1. Put the new clip and its exact transcript where the override points (reference.wav / reference.txt). Follow the candidate rules below. Keep the old pair beside it as a dated backup.
  2. The next session uses it. No build and no roll.
  3. For sessions already open, re-register by hand on the gateway host; it replaces the named entry from the next request: curl -X POST localhost:8850/v1/audio/voices -d '{"name":"operator","ref_text":"…","wav_b64":"…"}'. This lasts only until a worker re-registers its own sealed copy (a new session, or recovery after a server restart). Step 1 is the swap; step 3 only bridges it.

Rollback is the same move with the backup pair.

Add or replace a mood voice

A mood voice is <name>.wav + <name>.txt (the transcript) in the emobank directory. The worker registers every pair at startup, after the sealed reference, and again when a voice goes missing. It skips the name equal to VULPINE_TTS_HTTP_VOICE, which the sealed reference owns. The brain opens a reply with [mood:<tag>], and VULPINE_TTS_MOOD_VOICES maps the tag to a voice name. An unknown tag is stripped and the reply speaks in the default voice. Ship a new clip under a new name and remap VULPINE_TTS_MOOD_VOICES (a gateway restart); never overwrite a live name.

Every bank clip runs the intake gate first: python3 tools/emobank/gate.py <dir-or-wav> (8–15 s, floor ≤ −38 dBFS, no clipping, speech/noise contrast ≥ 14 dB, pauses present; exit 1 on any failure). The worker re-runs the same checks at registration. A failing clip still registers, so the palette never vanishes, and shows on the Admin panel as a reference_gate event.

Candidate rules

  • One speaker, one channel. Segment every candidate with an ASR pass first: a two-speaker clip clones both. Clean runs from one speaker, session and register hold identity; noisy cross-scene montages do not. Cut at phrase boundaries, never at a time cap, and fade the outer edges.
  • Everything in the reference transfers: channel static, sibilance (de-ess the reference, not the output), the character’s physical state. Pick clean, quiet, healthy takes.
  • ICL with a clean reference is the default. x-vector-only registration (no ref_text) measured worse.
  • A palette candidate passes tools/emobank/gate.py, has phrase-final fry ≤ 0.1, runs ≥ 10 s for a reliable cosine, and matches the live clip’s identity by ear. Mood renders are gated rank-only with a hard floor at 0.45. The 0.80 bar applies to plain-register renders only. Never raise the floor “to be safe”.
  • Gate before ears: no render reaches the operator before the ASR and speaker-similarity gates pass.
  • Licensing: reference clips cut from game audio are for private local use on the operator’s own hardware only. Never commit the audio and never distribute it.

Re-check

What is live on the gateway host (read-only; run from another Mac):

ssh <gateway-host> 'grep -hE "^(VULPINE|FOXLINE)_TTS" ~/.local/state/foxline/overlay.env.mirror ~/.local/state/foxline/gateway.env 2>/dev/null; curl -s localhost:8850/health; echo; ls ~/Library/Application\ Support/Foxline/voice/emobank | head'

A session’s own trace names what it registered: http_voice_registered (the sealed reference, with its byte count), http_emobank_registered (the bank: count, how many failed the gate, why it ran) and one http_emobank_voice_gate per clip that failed the gate.

Was this page helpful?