Integrations
How a third party plugs into fox9 (ADR 0034): every integration is an adapter outside the gateway on one of two contracts, verbs out (a command and one JSON receipt) or events in (JSON lines), and keeps the upstream's names and credentials to itself.
Living doc, checked against the code 2026-10-05
The Gateway, the protocol crate and the Brain speak only Vulpine’s own vocabulary (ADR 0034). Everything third-party (Home
Assistant, a Sonos player, a receiver, a camera pipeline, a shell script) sits behind one of the contracts below as an
adapter, outside the Gateway process. Device knowledge is configuration, never code in crates/.
To use one segment of Vulpine with your own devices, implement one of these contracts, nothing else.
Every integration starts in the adapter tier. A first-party driver runs inside the gateway only after the promotion test below, and it implements the same contract.
Subjects and rooms are Vulpine’s names
Both contracts are keyed by names Vulpine owns:
- a room is a lowercase kebab name:
kitchen,living-room,office; - a subject is
<room>/<thing>:kitchen/player,office/light,camera-1/car,owner/presence.
What an upstream calls the same thing (media_player.dining_room_dining_room, a Ring device id, a Frigate zone) lives
only in the adapter’s map file. It never appears in a trace row, an Admin projection or a Brain tool result.
Contract 1 — verbs out: command + receipt
Reference implementation: crates/vulpine_room, with rooms in ~/.config/vulpine/rooms.json (or $VULPINE_ROOMS_FILE). Consumers: the
Gateway’s room-speaker route (speak { room }) and the Brain tool pi_room_speak.
The Gateway runs the adapter as a process with a fixed argv and reads one JSON receipt from stdout:
<adapter> speak --room <room> --text <text> --owner <owner-id> --settle-s <n>
Receipt (compact JSON when ROOMCTL_COMPACT=1):
{"ok": true, "room": "kitchen", "driver": "sonos_clip",
"ack": {"source": "measured", "detail": "clip_status", "ms": 4210},
"render_ms": 2063, "audible_ms_estimate": 4080, "playback_id": "1HvIXMixseF1"}
{"ok": false, "room": "attic", "error": "unknown room 'attic'", "rooms": ["kitchen", "office"]}
| field | meaning |
|---|---|
ok |
the verb completed |
room |
the resolved Vulpine room |
driver |
the driver name from the rooms file (sonos_clip, upnp, ha, alexa, cli, or yours) |
ack.source |
measured (the device said it finished) or estimated (Vulpine guessed from the clip length) |
ack.detail |
how the driver measured it; free text, the Gateway does not read it |
ack.ms |
milliseconds from the command to the ack |
render_ms, audible_ms_estimate, playback_id |
optional; shown in Admin → Room speakers |
error, rooms |
on failure: what went wrong, and the rooms that exist |
Exit codes: 0 played and acked, 1 played but never acked, 2 usage / unknown
room, 3 a render or driver failure.
Adding a device is one entry in the rooms file. Adding a transport is one Driver variant in
crates/vulpine_room with probe() and play(ctx). Or point the cli driver at any executable with \{url\} \{room\} \{text\} \{wav\} \{audible_ms\} \{volume\} placeholders: a shell script is a valid integration.
Contract 2 — events in: JSON lines, contract version 1
Reference implementation: homectl (vulpine-home, crates/vulpine_home), with subjects in
~/.config/vulpine/home.json (or $VULPINE_HOME_FILE). Consumer: the Gateway’s home bridge ([home_bridge]). The bridge keeps the debounce,
the store, the trace (<trace_dir>/home-bridge.jsonl) and Admin → Home.
The Gateway runs the adapter as a child process. A .py adapter runs under [home_bridge] python:
[<python>] <adapter> --map <map-file> run
It reads one JSON object per line from the adapter’s stdout, forever. The adapter owns its upstream connection and reconnects itself. The Gateway respawns the adapter if the process exits.
{"type":"hello","adapter":"homectl","source":"ha","contract":1,"upstream":"Home Assistant 2026.8.2","subjects":16,"reconnect":false}
{"type":"state","subject":"office/light","kind":"light","state":"on"}
{"type":"ready","subjects":16}
{"type":"event","subject":"kitchen/player","kind":"media_player","state":"playing","from":"idle","to":"playing","volume":0.3,"title":"News"}
{"type":"event","subject":"office/light","kind":"light","state":"on","on":true,"brightness":254}
{"type":"event","subject":"camera-1/car","kind":"occupancy","state":"on","occupant":"car","occupied":true}
{"type":"event","subject":"owner/presence","kind":"person","state":"home","home":true,"zone":"home"}
{"type":"event","subject":"living-room/mic","kind":"other","state":"on","from":"off","to":"on"}
{"type":"event","subject":"camera-1/events","kind":"camera","state":"person","source":"frigate","camera":"amcrest_1","label":"person","phase":"new","zones":["porch"],"score":0.87,"event_id":"1757…"}
{"type":"churn","subject":"office/receiver"}
{"type":"status","source":"frigate","state":"connected"}
{"type":"status","source":"frigate","state":"connected","health":{"detector":"ov","inference_ms":12.1,"detection_fps":7.9,"uptime_s":30510,"version":"0.17.2","latest_version":"0.18.0","cameras":[{"subject":"camera-1","fps":5.1,"detection_fps":7.8,"skipped_fps":0.0}]}}
{"type":"traffic","source":"ring","at_ms":1793966400000}
{"type":"bye","error":"upstream closed the socket"}
| line | when | fields |
|---|---|---|
hello |
the primary upstream connected | adapter, source (ha, yours), sources (every source the adapter multiplexes), contract (must be 1), upstream (free text, shown in Admin), subjects, reconnect |
state |
once per subject after hello |
the seed: subject, kind, raw state |
ready |
the seed is complete | subjects |
event |
a real state flip on a subject | subject, kind, raw state, optional room (the map’s), optional source (which of the adapter’s sources), plus the kind’s fields below |
churn |
an upstream change that was not a state flip | subject (counted, not traced) |
status |
a SECONDARY source came or went (frigate) |
source, state (connected | disconnected | refused), error; shown in Admin, never marks the bridge itself disconnected; optional health — the source’s self-reported health, once a minute for Frigate (frigate/stats on the broker): detector, inference_ms, detection_fps, uptime_s, version, latest_version, cameras[{subject, fps, detection_fps, skipped_fps}] in Vulpine subjects (unmapped cameras left out); the gateway keeps the last one per source, traces home_bridge_source_health, and projects it on GetHome → sources[].health (Admin: the Home platform line) |
traffic |
a SECONDARY (MQTT) source heard a LIVE message | source, at_ms; at most one a minute per source, and a retained message (the broker’s memory, sent on subscribe) never counts. The gateway’s silent-bridge watch reads it: the home_bridge:<source> Health row says ok or quiet for N h, and past [home_bridge] quiet_alert_hours (6; 0 is off) one notify line goes out, then an hourly reminder, then one line when the source speaks again. The Ring source listens on its whole <prefix>/# for it ("watch": false in its map block keeps the two snapshot topics alone) |
bye |
the upstream dropped | error; fatal: true tells the Gateway not to respawn immediately |
Kinds and their fields (the closed vocabulary):
kind |
fields |
|---|---|
media_player |
from, to ∈ off idle playing paused unavailable unknown; optional volume (0–1), title, own_speaker: true (also on the seed state line; the map row says "fox9_speaker": true, the room’s own Vulpine speaker, so the ambient room note never reads its playing as a TV) |
light |
on; optional brightness (0–255) |
occupancy |
occupant ∈ person car all motion other; occupied; optional states (a magic-areas room’s area states, lower-case: occupied, extended, dark, bright, sleep, …; also on the seed state line). A change of the set alone is an event. The announcement board reads sleep and holds lines for a sleeping room (traced as home_room_states {room, asleep}) |
person |
home; zone |
camera |
camera, label (person, car, …, or a review severity), phase ∈ new update end; optional place (the spoken name of where it looks, “backyard”), zones, score (0–1), event_id, sub_label, has_snapshot, announce: false (an interior camera: never a board candidate, still traced / projected / presence). An event, not a state: no debounce, never a frame. A new event with an announceable label becomes an announcement candidate (“A person at the backyard camera.”). A map subject of kind camera on a Ring event.* entity (or a binary_sensor.*_ding / *_motion) emits label ding or motion, with camera = the map’s name or the subject’s room part (front-door/doorbell, side-yard/motion). ding (a doorbell press) is announceable by default: “Someone’s at the front door.” (score 0.9). motion is not announceable |
other |
from, to (raw strings) |
The Gateway applies the occupancy debounce after the adapter: off is held for occupancy_hold_ms and dropped on a
flap. Adapters emit every real flip and never debounce themselves.
Adding a device is one entry in the map file. Adding an upstream is a second adapter, or a second source
inside homectl, emitting the same lines; the Gateway never learns which. homectl’s second source is Frigate over MQTT
(frigate/events, frigate/reviews, read with the fox9 broker user from ~/.config/vulpine/mqtt.env; cameras
mapped in the map’s frigate block). Camera events reach the same trace and the same Admin panel as everything
else. homectl run --replay <tap.jsonl> replays a recorded ha_events.py tap through the contract, to re-derive a
day’s flips from a recording.
Looking through a camera is a verb with a receipt like the room verb: runtime/vision/visionctl.py describe --place backyard, brain tool pi_who_is_there. It takes one fresh frame from the vision platform (Frigate
latest.jpg?height=360, logged in with the user in ~/.config/vulpine/frigate.env). It asks one question of an
OpenAI-compatible vision model on a LAN host (VISION_URL + VISION_API_KEY, for example mlx-vlm behind a bearer
token). It returns one spoken sentence: \{ok, camera, place, subject, text, model, tokens, ms: \{snapshot, describe, total\}\}.
- Cameras are named by Vulpine place, room or subject from
home.json’sfrigate.cameras. The vendor’s camera names and the model host never reach the brain. It never uses a cloud model. - A Ring camera (
home.json’sring.cameras) is answered from the latest motion or doorbell snapshot homectl kept from ring-mqtt, never a live stream. The receipt carrieslive: false,frame: \{source: ring_event, event, age_s, when\}and anotenaming the age, orerror: no_event_frame. describe --event <id> --clipasks no model. It fetches the event’sclip.mp4into an owner-only folder outside the checkout and samples--clip-framesJPEGs from it, receipt\{ok, event_id, dir, clip, frames, model: null, ms\}. A video reasoner reads the folder later.
Adding a camera never touches Vulpine’s code:
- Find it.
runtime/home_bridge/discover_lan.py --subnet 192.0.2.0/24names what is on the wire (Kasa :9999, Tapo/Kasa :20002, ONVIF WS-Discovery, a camera-port sweep, the RTSP realm). It prints the platform line each device needs. - Give it to Frigate in Frigate’s
config.yml: RTSP for ONVIF cameras; go2rtckasa://for Kasa KC cameras, which have no RTSP. Keep credentials only in Frigate’s.env. - Map it:
frigate.cameras.<frigate name> = {subject: camera-N, label, place}inhome.json.placeis the spoken name the announcement uses (“A person at the backyard camera.”).
Plugs, bulbs and every non-latency device go to Home Assistant (tplink config flow) and become ordinary subjects.
Contract 3 — verbs on subjects: act + receipt
Reference implementation: vulpine-home act, the same adapter and map as contract 2. Consumers: the Gateway’s
home-verb route (AdminRequest::HomeAct over the owner-local Admin socket), the CLI
runtime/home_bridge/homeact.py, and the Brain tool pi_home_act. Nothing reaches the adapter except through the
Gateway, so every action lands on one trace (<trace_dir>/home-actions.jsonl: home_action_done /
home_action_failed) and one Admin projection (GetHomeActions, Admin → Home “Home verbs” line + Recent verbs).
[<python>] <adapter> --map <map-file> act --subject <subject|words> --verb <verb> [--value <0-100>]
--allow-kinds light,scene,switch,media_player --confirm-subjects <a,b> --settle-s <n> [--owner <id>]
The vocabulary is closed, per kind:
kind |
verbs |
|---|---|
light |
on off toggle brightness (0–100) |
switch |
on off toggle |
scene |
on |
media_player |
play pause stop volume (0–100) mute unmute play_station (--query <the station's words>; off when the map sets "radio": false) |
lock |
status (read-only) lock unlock — never a default kind; unlock needs the redeemed confirm code as well as --confirmed |
What each verb means upstream (light.turn_on with brightness_pct, scene.turn_on, media_player.volume_set, …)
is the adapter’s business. --allow-kinds and --confirm-subjects are the Gateway’s policy ([home_actions]),
handed down on every call:
- Kinds outside
--allow-kindsare denied. - A subject in the confirm list is refused with
confirmation_required(tierconfirm) unless the Gateway vouches for the operator’s yes with--confirmed. - An
unlockacts only with--confirmed --confirm-code <the code the Gateway redeemed>, whatever the confirm list says. Learned trust and an undo plan can never open a door. Without the code the receipt isconfirmation_requiredwithrequires: confirm_code. - A lock’s own flips arrive as
otherevents.
Confirmation is bound to one exact action and to words the Gateway itself heard. It lives in the Gateway, never in the adapter or the Brain.
- A
confirmation_requiredreceipt comes back from the Admin socket with a one-timeconfirm_code(60 s), recorded against that subject, verb and value. A newer code for the same action withdraws the older one. - The Brain says what would happen and asks (“office light: off — yes?”).
- The Gateway judges the operator’s next committed requests (
judge_confirmation). A question is never a yes. A retraction anywhere cancels. In the first six words, the earliest of stop and affirm decides, and a stop wins a tie. “yes, but…”, “yeah, no” and “…instead” are unclear, never yes. - The Brain re-sends the same request with the code. The Gateway redeems it before spawning the adapter, then runs
the adapter with
--confirmed. The receipt carriestier: confirm, confirmed: true, confirm_heard: "<phrase>"(the closed-grammar phrase, never the operator’s text).
The Gateway refuses without touching the house: a different subject, verb or value (confirm_code_mismatch), an
expired code (confirm_code_expired), an already-used code (confirm_code_unknown), a code the operator has not
answered yet (confirm_not_heard) or answered with a no (confirm_cancelled). The Brain tool drops any code no
receipt gave it. There is no “remember”, no batch and no code for a kind.
Learned trust ([approvals]) re-runs a subject-and-verb shape with --confirmed alone once the operator has
confirmed it approvals_to_trust times running (receipt tier: trusted). It never applies to unlock, nor while a
memo holds it off.
subject may be the operator’s words (“office overhead”). It resolves as an exact subject, an upstream alias
(homectl aliases --write keeps them in home-aliases.json beside the map), room thing, or the one subject whose
name or label carries every word, narrowed by the verb when only one kind takes it. It never guesses between several;
the candidates come back instead.
Receipt (compact JSON when HOMECTL_COMPACT=1):
{"ok": true, "subject": "office/light", "kind": "light", "verb": "brightness", "value": 40, "tier": "auto",
"before": {"state": "off", "brightness": 0}, "after": {"state": "on", "brightness": 40},
"ack": {"source": "measured", "detail": "state_read_back", "ms": 310}, "undo": {"verb": "brightness", "value": 0},
"changed": 1, "owner": "admin"}
{"ok": false, "subject": "office", "verb": "on", "reason": "unknown_subject", "error": "unknown subject 'office'",
"candidates": ["office/light", "office/overhead", "office/scene-concentrate"]}
{"ok": false, "subject": "living-room/player", "kind": "media_player", "verb": "play", "tier": "auto",
"before": {"state": "idle", "volume": 30}, "after": {"state": "idle", "volume": 30},
"ack": {"source": "measured", "detail": "no_change", "ms": 3004}, "reason": "no_effect", "error": "…"}
| field | meaning |
|---|---|
before, after |
the subject’s state in Vulpine’s units (state, brightness %, volume %, muted) read from the upstream before the call and after it settled |
ack.source |
measured (the state was read back and matched what the verb promised, or was already there) or estimated (nothing to read back — a scene) |
ack.detail |
state_read_back, already_there, scene_activated, changed_states, no_change, read_only (a lock status), in_transit (a lock still locking or unlocking when the wait ends) |
already_satisfied |
the subject was already where the verb asked before the call, so no call went out (changed: 0, reason already_satisfied) |
resolver |
{tier, matched} — which resolver pass named the subject (exact/alias/room_thing tier 1, all_words/verb_narrowed/group tier 2, kind_narrowed tier 3); null on a refusal |
undo |
the verb (and value) that puts the subject back, when there is one; null for a scene or a stop |
tier |
auto (acted), confirm (refused pending confirmation), deny (kind not allowed); the Gateway stamps trusted on a receipt learned trust ran |
reason |
typed: already_satisfied (on an ok receipt), unknown_subject, unknown_verb, bad_value, kind_not_actionable, kind_not_allowed, confirmation_required, subject_missing_upstream, no_effect, upstream_refused, upstream_unreachable |
Exit codes: 0 ok, 1 the upstream refused or the verb had no effect, 2 usage (unknown subject / verb / value,
policy refusal), 3 upstream unreachable. A verb whose promised state never arrives is ok: false, no_effect, with
both states in the receipt (“play” on an idle player with nothing queued).
Adding a controllable thing is one entry in the map file with kind light / switch / scene /
media_player. Scenes never seed or flip; they take verbs only. The Admin projection shows the policy and the last
32 receipts.
What is first-party, and how something gets there
A first-party integration runs inside the Gateway, in Rust, and exists only for a real-time path (turn-taking, barge, Vulpine’s own voice in a room). It implements the same contract as a driver, selected in the rooms file, so nothing upstream of the contract changes when an integration is promoted. Promotion needs a measured miss against a stated budget, and proof that the miss is in the hop a native client removes rather than in the device’s own floor.
A test fails the build if any Rust file under crates/ names Home Assistant’s entity domains or WebSocket verbs
(media_player., binary_sensor., subscribe_events, get_states).
Fleet adapter
The fleet adapter is not a device contract but has the same shape. vulpine-fleet (crates/vulpine_fleet,
just install-vulpine-fleet; its usage text and the examples below call it fleetctl) runs outside the gateway. It
speaks ssh so the gateway never does, and hands the gateway facts under Vulpine’s own labels. fleet_probe.py runs on each host:
stdlib, read-only, no arguments, no stdin. It is the forced command behind the restricted probe key.
fleetctl --manifest fleet.json run prints one line per interval:
{"type":"fleet_hello","adapter":"fleetctl/1","interval_s":30,"hosts":["gateway-host","laptop",…]}
{"type":"fleet_snapshot","at_ms":…,"interval_s":30,"hosts":[{
"label","role","reachable","error","elapsed_ms",
"hostname","os","kernel","arch","cpus","uptime_s","load":[1m,5m,15m],
"mem_total_bytes","mem_available_bytes","disk_total_bytes","disk_free_bytes",
"gpu_utilization_pct","gpu_model",
"logins":[{"user","line","since"}],
"ssh_sessions":[{"peer_label","peer_kind"}],
"listeners","services",
"herdr_present","herdr_version","herdr_server",
"agents":[{"pane","tab","workspace","agent","status","cwd_base","title","focused","session_source","foreground",
"ci"?:{"state","at_ms","label"?}}],
"quota":[…],"agent_reports":[…],
"tailnet","probe_ms"}],"sessions":{…}}
{"type":"fleet_error","error":"<class name>"}
erroris a short code:permission_denied,timeout,unresolved,refused,unreachable,host_key,ssh_failed,spawn_failed,no_probe_output.peer_labelis a manifest label,link-local,local,other-lanorother. Raw addresses exist only underfleetctl probe --raw.cwd_baseis a basename andtitlethe stripped terminal title. Pane text never crosses.focusedis true on the one pane herdr names as focused (focused_pane_id, or the agent’s own flag on a herdr without it).ciis the fleet CI row: the latest GitHub Actions run of the agent’s checkout branch.stateis one offailed/passed/running/pending/cancelled,at_msthe run’s stamp andlabelthe workflow’s name. The probe asks the host’s owngh run list(read-only) once per distinct repo and branch, and caches every answer for 60 s in~/.local/state/foxline/fleet-ci.json. A host withoutghor aghlogin sends noci. The branch, repo and run id never leave the probe. Hosts probed through the restricted key run their installed probe copy, so reinstall the probe there to get their CI rows.
The gateway ([fleet], fleet.rs) keeps the latest row per host and diffs consecutive rows into reachable /
unreachable / herdr_server_up / herdr_server_down / agent_added / agent_removed / agent_status. It
projects GetFleet. A down host keeps its last-known agents, so an outage is one unreachable and one reachable
row.
Agent notices. The same diff feeds the announcement board’s herdr source. An agent seen working that is now
idle / done is offered as “<Agent> in <project> on the <host> is done.” (score 0.6). One now blocked /
waiting / error is offered as “… needs you.” (0.9). Anything -> working, a first sighting and the probe’s own
add/remove flaps say nothing. There is one notice per host:pane:agent per [fleet] agent_notice_cooldown_ms
(default 10 min). [fleet] announce_agents = false turns the source off. The board speaks it like a camera line: the
room presence picks the speaker, else the session voice does.
Agent presence. The same rows feed a cross-harness AgentPresence projection on GetFleet (agent_presence,
absent until a sighting). It is built from the adapter’s rows, never from pane text (fleet_presence.rs). The brain
reads it through pi_fleet_status.
- One row per
host:pane:agent, withharness(claude/codex/pi/herdr, from the agent name) andstate(awaiting_input/busy/idle/error/compacting/done). The state folds the hook spool’s states and herdr’s:blockedreadsawaiting_input,workingreadsbusy. since_msis when the agent entered its state.seen_watermark_msmoves only on the owner-localMarkAgentPresenceSeen {ids}(empty = all), or when a probe sees the agent’s herdr pane focused; never on a probe or a read.seen_reasonsays which (operator/focus).- An agent that went idle after working and has not been seen since reads
done, withunseen: true: “finished while you weren’t looking”. attention_events(newest first) record enteringawaiting_inputand each change of the fleet CI row (ci_failed/ci_passed/ci_running/ci_pending/ci_cancelled). A finishing one after the watermark makes the row unseen.
Meet adapter
The meeting inlet has the fleet adapter’s shape: meetctl runs outside the gateway and hands it typed events under
Vulpine’s names on an owner-local unix socket. Nothing is on the WebSocket protocol. The inlet is off until
[meet] enabled = true.
| piece | what it is | where |
|---|---|---|
meetctl |
the local two-leg Meet adapter: the operator’s mic (near) and the browser’s output (far), decoded, labeled and sent as typed events | tools/meet/meetctl.swift |
meeting.v1 |
the events it emits (meeting_started, meeting_segment, meeting_ended, meeting_forget, meeting_opt_out), their acks and refusal reasons |
docs/contracts/meeting.v1.md |
| ADR 0039 | the design: two adapters, one transcript of record, consent before capture | the ADR set |