The early start
Speculative brain dispatch: the gateway starts the brain on the stable partial transcript before the operator has finished talking, holding output and tools; it promotes the answer when the final matches, and forks the pi session and re-dispatches the committed text when it differs.
Living doc, checked against the code 2026-11-14
Speculation overlaps the end-of-utterance wait with the Brain’s first token. Once the armed interim transcript is
stable for speculation.trigger_silence_ms, the gateway sends it to Pi with output and tools held. When the STT final
commits, a matching final promotes the answer; a different final forks the Pi session at the anchored entry and
re-dispatches the committed text. It is off by default.
Mechanism
One speculative turn, from the stable interim to the commit’s three outcomes:
- Trigger: while an armed capture is open, a 100 ms session tick watches the interim transcript. Once it has
been stable for
speculation.trigger_silence_ms(silence alone decides), the gateway sends it to Pi as a speculative prompt. The dispatch is phase-silent: no thinking chip, all brain output held in a session buffer, TTS and side effects withheld. - Promote (final matches the speculated text): the speculative generation is the turn. The thinking phase emits and the buffer replays through the normal processing path, with no second prompt.
- Refork (final differs): the gateway aborts the generation, forks the Pi session with
forkat the speculative user-message entry (anchored fromget_entries→pi_session_entries) and re-dispatches the committed text as the real turn. The rewound prefix is byte-identical, so provider prompt caching absorbs most of the cost. - Cancel: a disarm, a brain error or the
max_reforks_per_turnbound retires speculation for the turn, and the committed path runs normally. A final that resolves to a tier-1/2 gateway verb also cancels it (speculation_cancelled {reason: gateway_verb}); the verb routes with itsverb_tierrow. - Hold: with
[invocation] enrolled_pcm_gateon, speculation does not dispatch while the request’s speaker vote is open (speculation_held {reason: enrolled_gate_pending}, once per request). An admitted commit dispatches on the committed path, so a refused request’s text never reaches the brain, and a refused verb writesverb_refused {verb}instead of itsverb_tierrow. A request the gate cannot judge (no enrollment from its wake) is held the same way and refused at its routing frame (enrolled_gate_request \{decision: refuse, reason: unjudged, fresh_ms\}). - Calls: a call loadout (
dialogue_mode = true: Vigil) never arms, so its requests get no speculative start by default. With the loadout’s[invocation] call_speculation = true(set by thevigilloadout), an open call window is a speculation window: the partials gate opens for the call (partials_gate_opened {reason: call}) and closes at its end, and the trigger runs as above. A final ends the utterance: its interim is dropped, and a running speculation gets the disarm grace to meet its commit. Words heard over a reply never speculate. A call’s request with no head start writesspeculation_skipped {reason, window: call}before its commit, with reasoncall_speculation_off,speculation_disabled,reply_in_flight,in_call,playback_latchorfinal_first.- While the partials gate is open, the segmenter runs its close tick at speech end, so the ended segment’s whole
words arrive as a
closedpartial and the trigger fires at once, about 250 ms ahead of the final. - The call trigger holds while the call take would refuse the line at its final:
in_call(a phone call is live) andplayback_latch(a reply ended under 3 s ago). A head start never hands the brain words the take then refuses.
- While the partials gate is open, the segmenter runs its close tick at speech end, so the ended segment’s whole
words arrive as a
- Tool gate: the
fox9-reforkPi extension (crates/voice_gateway/extensions/fox9-refork) parks every Pitool_callwhile the gateway’s gate file exists. The path comes fromVULPINE_SPECULATION_GATE(the extension still reads the oldFOX9_SPECULATION_GATE), one per gateway process, and the file follows the speculative hold. Promotion releases parked calls, and an uncleared gate refuses at 10 s. It fails closed, so a truncated instruction never runs tools.
Configuration
[speculation] is a section of the gateway config (docs/reference/config.md). The defaults:
[speculation]
enabled = false # true dispatches speculatively
trigger_silence_ms = 500 # longer: fewer reforks, a later head start
max_reforks_per_turn = 3
The loadout must carry builtin:fox9-refork in extensions; the operator loadout does. The Admin’s audio section
shows the three speculation.* fields.
Trace vocabulary
speculation_triggered, speculation_brain_dispatch,
speculation_entry_observed, speculation_promoted,
speculation_refork_started, speculation_refork_without_entry,
speculation_buffer_released, speculation_cancelled,
speculation_retired_on_brain_error, speculation_dispatch_failed,
speculation_skipped_brain_backoff, speculation_skipped,
speculation_fork_timeout, speculation_refork_skipped_unsaved_session,
speculation_cancel_restarted_brain, speculation_held.