Skip to content
Vulpine
Esc
↑↓navigate↵open⌘Jpreview
On this page

The early start

Speculative brain dispatch: the gateway starts the brain on the stable partial transcript before the operator has finished talking, holding output and tools; it promotes the answer when the final matches, and forks the pi session and re-dispatches the committed text when it differs.

Living doc, checked against the code 2026-11-14

Speculation overlaps the end-of-utterance wait with the Brain’s first token. Once the armed interim transcript is stable for speculation.trigger_silence_ms, the gateway sends it to Pi with output and tools held. When the STT final commits, a matching final promotes the answer; a different final forks the Pi session at the anchored entry and re-dispatches the committed text. It is off by default.

Mechanism

One speculative turn, from the stable interim to the commit’s three outcomes:

  • Trigger: while an armed capture is open, a 100 ms session tick watches the interim transcript. Once it has been stable for speculation.trigger_silence_ms (silence alone decides), the gateway sends it to Pi as a speculative prompt. The dispatch is phase-silent: no thinking chip, all brain output held in a session buffer, TTS and side effects withheld.
  • Promote (final matches the speculated text): the speculative generation is the turn. The thinking phase emits and the buffer replays through the normal processing path, with no second prompt.
  • Refork (final differs): the gateway aborts the generation, forks the Pi session with fork at the speculative user-message entry (anchored from get_entries → pi_session_entries) and re-dispatches the committed text as the real turn. The rewound prefix is byte-identical, so provider prompt caching absorbs most of the cost.
  • Cancel: a disarm, a brain error or the max_reforks_per_turn bound retires speculation for the turn, and the committed path runs normally. A final that resolves to a tier-1/2 gateway verb also cancels it (speculation_cancelled {reason: gateway_verb}); the verb routes with its verb_tier row.
  • Hold: with [invocation] enrolled_pcm_gate on, speculation does not dispatch while the request’s speaker vote is open (speculation_held {reason: enrolled_gate_pending}, once per request). An admitted commit dispatches on the committed path, so a refused request’s text never reaches the brain, and a refused verb writes verb_refused {verb} instead of its verb_tier row. A request the gate cannot judge (no enrollment from its wake) is held the same way and refused at its routing frame (enrolled_gate_request \{decision: refuse, reason: unjudged, fresh_ms\}).
  • Calls: a call loadout (dialogue_mode = true: Vigil) never arms, so its requests get no speculative start by default. With the loadout’s [invocation] call_speculation = true (set by the vigil loadout), an open call window is a speculation window: the partials gate opens for the call (partials_gate_opened {reason: call}) and closes at its end, and the trigger runs as above. A final ends the utterance: its interim is dropped, and a running speculation gets the disarm grace to meet its commit. Words heard over a reply never speculate. A call’s request with no head start writes speculation_skipped {reason, window: call} before its commit, with reason call_speculation_off, speculation_disabled, reply_in_flight, in_call, playback_latch or final_first.
    • While the partials gate is open, the segmenter runs its close tick at speech end, so the ended segment’s whole words arrive as a closed partial and the trigger fires at once, about 250 ms ahead of the final.
    • The call trigger holds while the call take would refuse the line at its final: in_call (a phone call is live) and playback_latch (a reply ended under 3 s ago). A head start never hands the brain words the take then refuses.
  • Tool gate: the fox9-refork Pi extension (crates/voice_gateway/extensions/fox9-refork) parks every Pi tool_call while the gateway’s gate file exists. The path comes from VULPINE_SPECULATION_GATE (the extension still reads the old FOX9_SPECULATION_GATE), one per gateway process, and the file follows the speculative hold. Promotion releases parked calls, and an uncleared gate refuses at 10 s. It fails closed, so a truncated instruction never runs tools.

Configuration

[speculation] is a section of the gateway config (docs/reference/config.md). The defaults:

[speculation]
enabled = false             # true dispatches speculatively
trigger_silence_ms = 500    # longer: fewer reforks, a later head start
max_reforks_per_turn = 3

The loadout must carry builtin:fox9-refork in extensions; the operator loadout does. The Admin’s audio section shows the three speculation.* fields.

Trace vocabulary

speculation_triggered, speculation_brain_dispatch, speculation_entry_observed, speculation_promoted, speculation_refork_started, speculation_refork_without_entry, speculation_buffer_released, speculation_cancelled, speculation_retired_on_brain_error, speculation_dispatch_failed, speculation_skipped_brain_backoff, speculation_skipped, speculation_fork_timeout, speculation_refork_skipped_unsaved_session, speculation_cancel_restarted_brain, speculation_held.

Was this page helpful?