---
title: "The spoken voice"
description: "Swap Vulpine's spoken voice, add a mood voice, and roll back. just models installs the qwentts.cpp tts-server on :8850 with qwen-talker-0.6b-base-Q8_0 (tools/install/models.toml); the gateway registers its voice there as operator, ICL."
type: "living"
source: "docs/voice-operator-reference.md"
verified: "2026-11-14"
group: "The voice path"
---

> **Living doc**, checked against the code 2026-11-14

Which clip is live is host state; the re-check below reads it.

## The path

The gateway runs `runtime/qwen3_tts_worker/qwen3_tts_worker.py` and hands it the sealed voice reference (WAV +
transcript) as inherited descriptors (`ws/adapters.rs`, `build_tts_adapter`; ARCHITECTURE's TTS section). The worker
has two modes behind the same frame protocol:

- **qwentts-http (with the installed server).** With `VULPINE_TTS_HTTP_URL` set, the gateway passes `--http-url` and
  `--http-voice` (`VULPINE_TTS_HTTP_VOICE`, default `operator`). The worker health-checks the server (`GET /health`),
  registers the sealed reference under that name (`POST /v1/audio/voices`), registers the mood bank in the
  background, and streams PCM from the server. It loads no model.
- **MLX (the fallback).** Without `VULPINE_TTS_HTTP_URL` the worker loads Qwen3-TTS through MLX in-process and clones
  the sealed reference per request.

The server's voice registry is memory-only; the worker recovers on its own:

- On `unknown voice` for the default voice, it re-registers the sealed reference and retries once.
- A mood voice that answers empty or unknown is re-registered, or the request falls back to the default voice. Then
  the rest of the bank refills. Mood never fails a turn.
- Bank registrations wait while any worker on the same server streams a reply, and for 1.5 s after, so a refill
  never lands inside a reply. The workers share a hold directory per server under
  `~/.local/state/foxline/tts-stream-hold/` (`VULPINE_TTS_STREAM_HOLD_DIR` overrides it).

## Knobs

All are environment variables the gateway reads at start, set in `<state>/gateway.env` (`docs/install.md`), so a
change takes a gateway restart. A roll stages them from its own `overlay.env` (`docs/release.md`).


| variable | effect |
|---|---|
| `VULPINE_TTS_HTTP_URL` | set: qwentts-http mode against that server; unset or empty: the MLX worker |
| `VULPINE_TTS_HTTP_VOICE` | the name the sealed reference registers under (default `operator`) |
| `VULPINE_TTS_EMOBANK_DIR` | the mood bank directory (worker default `~/Library/Application Support/Foxline/voice/emobank`) |
| `VULPINE_TTS_MOOD_VOICES` | `mood=voice` pairs (`wry=<voice>-wry,dark=<voice>-dark`); unset turns mood selection off (`mood.rs`) |
| `VULPINE_TTS_TRUNCATION_RERENDER` | `1`/`true`/`on`/`yes`: the worker buffers each segment before it plays. It re-renders once a clip shorter than 0.65 × its text's expected duration (texts of 20+ characters, at 17.5 chars/s, the live voice's measured rate: `benchmarks/results/truncation-recalibration-20261002.md`). A second short render plays as it is. Default off: the buffering adds the whole render time to every segment's first audio. Off or on, a cut-off clip is traced as `tts_render_truncated` (`retried`, `outcome`: `streamed` / `retry_full` / `retry_short` / `retry_failed`) |
| `VULPINE_TTS_REF_AUDIO` + `VULPINE_TTS_REF_TEXT_FILE` | the deployment voice override: both or neither (one alone refuses the bind). Without them the gateway uses the persona package's manifest voice |

## Swap the operator voice

The gateway resolves the voice reference when a session binds and seals that copy into the session's worker:

1. Put the new clip and its exact transcript where the override points (`reference.wav` / `reference.txt`). Follow
   the candidate rules below. Keep the old pair beside it as a dated backup.
2. The next session uses it. No build and no roll.
3. For sessions already open, re-register by hand on the gateway host; it replaces the named entry from the next request:
   `curl -X POST localhost:8850/v1/audio/voices -d '{"name":"operator","ref_text":"…","wav_b64":"…"}'`.
   This lasts only until a worker re-registers its own sealed copy (a new session, or recovery after a server
   restart). Step 1 is the swap; step 3 only bridges it.

Rollback is the same move with the backup pair.

## Add or replace a mood voice

A mood voice is `<name>.wav` + `<name>.txt` (the transcript) in the emobank directory. The worker registers every pair
at startup, after the sealed reference, and again when a voice goes missing. It skips the name equal to
`VULPINE_TTS_HTTP_VOICE`, which the sealed reference owns. The brain opens a reply with `[mood:<tag>]`, and
`VULPINE_TTS_MOOD_VOICES` maps the tag to a voice name. An unknown tag is stripped and the reply speaks in the
default voice. Ship a new clip under a new name and remap `VULPINE_TTS_MOOD_VOICES` (a gateway restart); never overwrite a
live name.

Every bank clip runs the intake gate first: `python3 tools/emobank/gate.py <dir-or-wav>` (8–15 s, floor ≤ −38 dBFS,
no clipping, speech/noise contrast ≥ 14 dB, pauses present; exit 1 on any failure). The worker re-runs the same
checks at registration. A failing clip still registers, so the palette never vanishes, and shows on the Admin panel
as a `reference_gate` event.

## Candidate rules


- **One speaker, one channel.** Segment every candidate with an ASR pass first: a two-speaker clip clones both.
  Clean runs from one speaker, session and register hold identity; noisy cross-scene montages do not. Cut at phrase
  boundaries, never at a time cap, and fade the outer edges.
- **Everything in the reference transfers**: channel static, sibilance (de-ess the reference, not the output), the
  character's physical state. Pick clean, quiet, healthy takes.
- **ICL with a clean reference is the default.** x-vector-only registration (no `ref_text`) measured worse.
- **A palette candidate** passes `tools/emobank/gate.py`, has phrase-final fry ≤ 0.1, runs ≥ 10 s for a reliable
  cosine, and matches the live clip's identity by ear. Mood renders are gated rank-only with a hard floor at 0.45.
  The 0.80 bar applies to plain-register renders only. Never raise the floor "to be safe".
- **Gate before ears**: no render reaches the operator before the ASR and speaker-similarity gates pass.
- **Licensing**: reference clips cut from game audio are for private local use on the operator's own hardware only.
  Never commit the audio and never distribute it.

## Re-check

What is live on the gateway host (read-only; run from another Mac):

```sh
ssh <gateway-host> 'grep -hE "^(VULPINE|FOXLINE)_TTS" ~/.local/state/foxline/overlay.env.mirror ~/.local/state/foxline/gateway.env 2>/dev/null; curl -s localhost:8850/health; echo; ls ~/Library/Application\ Support/Foxline/voice/emobank | head'
```

A session's own trace names what it registered: `http_voice_registered` (the sealed reference, with its byte count),
`http_emobank_registered` (the bank: count, how many failed the gate, why it ran) and one `http_emobank_voice_gate`
per clip that failed the gate.
