Skip to content
Vulpine
Esc
↑↓navigate↵open⌘Jpreview
On this page

Integrations

How a third party plugs into fox9 (ADR 0034): every integration is an adapter outside the gateway on one of two contracts, verbs out (a command and one JSON receipt) or events in (JSON lines), and keeps the upstream's names and credentials to itself.

Living doc, checked against the code 2026-10-05

The Gateway, the protocol crate and the Brain speak only Vulpine’s own vocabulary (ADR 0034). Everything third-party (Home Assistant, a Sonos player, a receiver, a camera pipeline, a shell script) sits behind one of the contracts below as an adapter, outside the Gateway process. Device knowledge is configuration, never code in crates/.

To use one segment of Vulpine with your own devices, implement one of these contracts, nothing else.

Every integration starts in the adapter tier. A first-party driver runs inside the gateway only after the promotion test below, and it implements the same contract.

Subjects and rooms are Vulpine’s names

Both contracts are keyed by names Vulpine owns:

  • a room is a lowercase kebab name: kitchen, living-room, office;
  • a subject is <room>/<thing>: kitchen/player, office/light, camera-1/car, owner/presence.

What an upstream calls the same thing (media_player.dining_room_dining_room, a Ring device id, a Frigate zone) lives only in the adapter’s map file. It never appears in a trace row, an Admin projection or a Brain tool result.


Contract 1 — verbs out: command + receipt

Reference implementation: crates/vulpine_room, with rooms in ~/.config/vulpine/rooms.json (or $VULPINE_ROOMS_FILE). Consumers: the Gateway’s room-speaker route (speak { room }) and the Brain tool pi_room_speak.

The Gateway runs the adapter as a process with a fixed argv and reads one JSON receipt from stdout:

<adapter> speak --room <room> --text <text> --owner <owner-id> --settle-s <n>

Receipt (compact JSON when ROOMCTL_COMPACT=1):

{"ok": true, "room": "kitchen", "driver": "sonos_clip",
 "ack": {"source": "measured", "detail": "clip_status", "ms": 4210},
 "render_ms": 2063, "audible_ms_estimate": 4080, "playback_id": "1HvIXMixseF1"}
{"ok": false, "room": "attic", "error": "unknown room 'attic'", "rooms": ["kitchen", "office"]}
field meaning
ok the verb completed
room the resolved Vulpine room
driver the driver name from the rooms file (sonos_clip, upnp, ha, alexa, cli, or yours)
ack.source measured (the device said it finished) or estimated (Vulpine guessed from the clip length)
ack.detail how the driver measured it; free text, the Gateway does not read it
ack.ms milliseconds from the command to the ack
render_ms, audible_ms_estimate, playback_id optional; shown in Admin → Room speakers
error, rooms on failure: what went wrong, and the rooms that exist

Exit codes: 0 played and acked, 1 played but never acked, 2 usage / unknown room, 3 a render or driver failure.

Adding a device is one entry in the rooms file. Adding a transport is one Driver variant in crates/vulpine_room with probe() and play(ctx). Or point the cli driver at any executable with \{url\} \{room\} \{text\} \{wav\} \{audible_ms\} \{volume\} placeholders: a shell script is a valid integration.


Contract 2 — events in: JSON lines, contract version 1

Reference implementation: homectl (vulpine-home, crates/vulpine_home), with subjects in ~/.config/vulpine/home.json (or $VULPINE_HOME_FILE). Consumer: the Gateway’s home bridge ([home_bridge]). The bridge keeps the debounce, the store, the trace (<trace_dir>/home-bridge.jsonl) and Admin → Home.

The Gateway runs the adapter as a child process. A .py adapter runs under [home_bridge] python:

[<python>] <adapter> --map <map-file> run

It reads one JSON object per line from the adapter’s stdout, forever. The adapter owns its upstream connection and reconnects itself. The Gateway respawns the adapter if the process exits.

{"type":"hello","adapter":"homectl","source":"ha","contract":1,"upstream":"Home Assistant 2026.8.2","subjects":16,"reconnect":false}
{"type":"state","subject":"office/light","kind":"light","state":"on"}
{"type":"ready","subjects":16}
{"type":"event","subject":"kitchen/player","kind":"media_player","state":"playing","from":"idle","to":"playing","volume":0.3,"title":"News"}
{"type":"event","subject":"office/light","kind":"light","state":"on","on":true,"brightness":254}
{"type":"event","subject":"camera-1/car","kind":"occupancy","state":"on","occupant":"car","occupied":true}
{"type":"event","subject":"owner/presence","kind":"person","state":"home","home":true,"zone":"home"}
{"type":"event","subject":"living-room/mic","kind":"other","state":"on","from":"off","to":"on"}
{"type":"event","subject":"camera-1/events","kind":"camera","state":"person","source":"frigate","camera":"amcrest_1","label":"person","phase":"new","zones":["porch"],"score":0.87,"event_id":"1757…"}
{"type":"churn","subject":"office/receiver"}
{"type":"status","source":"frigate","state":"connected"}
{"type":"status","source":"frigate","state":"connected","health":{"detector":"ov","inference_ms":12.1,"detection_fps":7.9,"uptime_s":30510,"version":"0.17.2","latest_version":"0.18.0","cameras":[{"subject":"camera-1","fps":5.1,"detection_fps":7.8,"skipped_fps":0.0}]}}
{"type":"traffic","source":"ring","at_ms":1793966400000}
{"type":"bye","error":"upstream closed the socket"}
line when fields
hello the primary upstream connected adapter, source (ha, yours), sources (every source the adapter multiplexes), contract (must be 1), upstream (free text, shown in Admin), subjects, reconnect
state once per subject after hello the seed: subject, kind, raw state
ready the seed is complete subjects
event a real state flip on a subject subject, kind, raw state, optional room (the map’s), optional source (which of the adapter’s sources), plus the kind’s fields below
churn an upstream change that was not a state flip subject (counted, not traced)
status a SECONDARY source came or went (frigate) source, state (connected | disconnected | refused), error; shown in Admin, never marks the bridge itself disconnected; optional health — the source’s self-reported health, once a minute for Frigate (frigate/stats on the broker): detector, inference_ms, detection_fps, uptime_s, version, latest_version, cameras[{subject, fps, detection_fps, skipped_fps}] in Vulpine subjects (unmapped cameras left out); the gateway keeps the last one per source, traces home_bridge_source_health, and projects it on GetHome → sources[].health (Admin: the Home platform line)
traffic a SECONDARY (MQTT) source heard a LIVE message source, at_ms; at most one a minute per source, and a retained message (the broker’s memory, sent on subscribe) never counts. The gateway’s silent-bridge watch reads it: the home_bridge:<source> Health row says ok or quiet for N h, and past [home_bridge] quiet_alert_hours (6; 0 is off) one notify line goes out, then an hourly reminder, then one line when the source speaks again. The Ring source listens on its whole <prefix>/# for it ("watch": false in its map block keeps the two snapshot topics alone)
bye the upstream dropped error; fatal: true tells the Gateway not to respawn immediately

Kinds and their fields (the closed vocabulary):

kind fields
media_player from, to ∈ off idle playing paused unavailable unknown; optional volume (0–1), title, own_speaker: true (also on the seed state line; the map row says "fox9_speaker": true, the room’s own Vulpine speaker, so the ambient room note never reads its playing as a TV)
light on; optional brightness (0–255)
occupancy occupant ∈ person car all motion other; occupied; optional states (a magic-areas room’s area states, lower-case: occupied, extended, dark, bright, sleep, …; also on the seed state line). A change of the set alone is an event. The announcement board reads sleep and holds lines for a sleeping room (traced as home_room_states {room, asleep})
person home; zone
camera camera, label (person, car, …, or a review severity), phase ∈ new update end; optional place (the spoken name of where it looks, “backyard”), zones, score (0–1), event_id, sub_label, has_snapshot, announce: false (an interior camera: never a board candidate, still traced / projected / presence). An event, not a state: no debounce, never a frame. A new event with an announceable label becomes an announcement candidate (“A person at the backyard camera.”). A map subject of kind camera on a Ring event.* entity (or a binary_sensor.*_ding / *_motion) emits label ding or motion, with camera = the map’s name or the subject’s room part (front-door/doorbell, side-yard/motion). ding (a doorbell press) is announceable by default: “Someone’s at the front door.” (score 0.9). motion is not announceable
other from, to (raw strings)

The Gateway applies the occupancy debounce after the adapter: off is held for occupancy_hold_ms and dropped on a flap. Adapters emit every real flip and never debounce themselves.

Adding a device is one entry in the map file. Adding an upstream is a second adapter, or a second source inside homectl, emitting the same lines; the Gateway never learns which. homectl’s second source is Frigate over MQTT (frigate/events, frigate/reviews, read with the fox9 broker user from ~/.config/vulpine/mqtt.env; cameras mapped in the map’s frigate block). Camera events reach the same trace and the same Admin panel as everything else. homectl run --replay <tap.jsonl> replays a recorded ha_events.py tap through the contract, to re-derive a day’s flips from a recording.

Looking through a camera is a verb with a receipt like the room verb: runtime/vision/visionctl.py describe --place backyard, brain tool pi_who_is_there. It takes one fresh frame from the vision platform (Frigate latest.jpg?height=360, logged in with the user in ~/.config/vulpine/frigate.env). It asks one question of an OpenAI-compatible vision model on a LAN host (VISION_URL + VISION_API_KEY, for example mlx-vlm behind a bearer token). It returns one spoken sentence: \{ok, camera, place, subject, text, model, tokens, ms: \{snapshot, describe, total\}\}.

  • Cameras are named by Vulpine place, room or subject from home.json’s frigate.cameras. The vendor’s camera names and the model host never reach the brain. It never uses a cloud model.
  • A Ring camera (home.json’s ring.cameras) is answered from the latest motion or doorbell snapshot homectl kept from ring-mqtt, never a live stream. The receipt carries live: false, frame: \{source: ring_event, event, age_s, when\} and a note naming the age, or error: no_event_frame.
  • describe --event <id> --clip asks no model. It fetches the event’s clip.mp4 into an owner-only folder outside the checkout and samples --clip-frames JPEGs from it, receipt \{ok, event_id, dir, clip, frames, model: null, ms\}. A video reasoner reads the folder later.

Adding a camera never touches Vulpine’s code:

  1. Find it. runtime/home_bridge/discover_lan.py --subnet 192.0.2.0/24 names what is on the wire (Kasa :9999, Tapo/Kasa :20002, ONVIF WS-Discovery, a camera-port sweep, the RTSP realm). It prints the platform line each device needs.
  2. Give it to Frigate in Frigate’s config.yml: RTSP for ONVIF cameras; go2rtc kasa:// for Kasa KC cameras, which have no RTSP. Keep credentials only in Frigate’s .env.
  3. Map it: frigate.cameras.<frigate name> = {subject: camera-N, label, place} in home.json. place is the spoken name the announcement uses (“A person at the backyard camera.”).

Plugs, bulbs and every non-latency device go to Home Assistant (tplink config flow) and become ordinary subjects.


Contract 3 — verbs on subjects: act + receipt

Reference implementation: vulpine-home act, the same adapter and map as contract 2. Consumers: the Gateway’s home-verb route (AdminRequest::HomeAct over the owner-local Admin socket), the CLI runtime/home_bridge/homeact.py, and the Brain tool pi_home_act. Nothing reaches the adapter except through the Gateway, so every action lands on one trace (<trace_dir>/home-actions.jsonl: home_action_done / home_action_failed) and one Admin projection (GetHomeActions, Admin → Home “Home verbs” line + Recent verbs).

[<python>] <adapter> --map <map-file> act --subject <subject|words> --verb <verb> [--value <0-100>]
    --allow-kinds light,scene,switch,media_player --confirm-subjects <a,b> --settle-s <n> [--owner <id>]

The vocabulary is closed, per kind:

kind verbs
light on off toggle brightness (0–100)
switch on off toggle
scene on
media_player play pause stop volume (0–100) mute unmute play_station (--query <the station's words>; off when the map sets "radio": false)
lock status (read-only) lock unlock — never a default kind; unlock needs the redeemed confirm code as well as --confirmed

What each verb means upstream (light.turn_on with brightness_pct, scene.turn_on, media_player.volume_set, …) is the adapter’s business. --allow-kinds and --confirm-subjects are the Gateway’s policy ([home_actions]), handed down on every call:

  • Kinds outside --allow-kinds are denied.
  • A subject in the confirm list is refused with confirmation_required (tier confirm) unless the Gateway vouches for the operator’s yes with --confirmed.
  • An unlock acts only with --confirmed --confirm-code <the code the Gateway redeemed>, whatever the confirm list says. Learned trust and an undo plan can never open a door. Without the code the receipt is confirmation_required with requires: confirm_code.
  • A lock’s own flips arrive as other events.

Confirmation is bound to one exact action and to words the Gateway itself heard. It lives in the Gateway, never in the adapter or the Brain.

  1. A confirmation_required receipt comes back from the Admin socket with a one-time confirm_code (60 s), recorded against that subject, verb and value. A newer code for the same action withdraws the older one.
  2. The Brain says what would happen and asks (“office light: off — yes?”).
  3. The Gateway judges the operator’s next committed requests (judge_confirmation). A question is never a yes. A retraction anywhere cancels. In the first six words, the earliest of stop and affirm decides, and a stop wins a tie. “yes, but…”, “yeah, no” and “…instead” are unclear, never yes.
  4. The Brain re-sends the same request with the code. The Gateway redeems it before spawning the adapter, then runs the adapter with --confirmed. The receipt carries tier: confirm, confirmed: true, confirm_heard: "<phrase>" (the closed-grammar phrase, never the operator’s text).

The Gateway refuses without touching the house: a different subject, verb or value (confirm_code_mismatch), an expired code (confirm_code_expired), an already-used code (confirm_code_unknown), a code the operator has not answered yet (confirm_not_heard) or answered with a no (confirm_cancelled). The Brain tool drops any code no receipt gave it. There is no “remember”, no batch and no code for a kind.

Learned trust ([approvals]) re-runs a subject-and-verb shape with --confirmed alone once the operator has confirmed it approvals_to_trust times running (receipt tier: trusted). It never applies to unlock, nor while a memo holds it off.

subject may be the operator’s words (“office overhead”). It resolves as an exact subject, an upstream alias (homectl aliases --write keeps them in home-aliases.json beside the map), room thing, or the one subject whose name or label carries every word, narrowed by the verb when only one kind takes it. It never guesses between several; the candidates come back instead.

Receipt (compact JSON when HOMECTL_COMPACT=1):

{"ok": true, "subject": "office/light", "kind": "light", "verb": "brightness", "value": 40, "tier": "auto",
 "before": {"state": "off", "brightness": 0}, "after": {"state": "on", "brightness": 40},
 "ack": {"source": "measured", "detail": "state_read_back", "ms": 310}, "undo": {"verb": "brightness", "value": 0},
 "changed": 1, "owner": "admin"}
{"ok": false, "subject": "office", "verb": "on", "reason": "unknown_subject", "error": "unknown subject 'office'",
 "candidates": ["office/light", "office/overhead", "office/scene-concentrate"]}
{"ok": false, "subject": "living-room/player", "kind": "media_player", "verb": "play", "tier": "auto",
 "before": {"state": "idle", "volume": 30}, "after": {"state": "idle", "volume": 30},
 "ack": {"source": "measured", "detail": "no_change", "ms": 3004}, "reason": "no_effect", "error": "…"}
field meaning
before, after the subject’s state in Vulpine’s units (state, brightness %, volume %, muted) read from the upstream before the call and after it settled
ack.source measured (the state was read back and matched what the verb promised, or was already there) or estimated (nothing to read back — a scene)
ack.detail state_read_back, already_there, scene_activated, changed_states, no_change, read_only (a lock status), in_transit (a lock still locking or unlocking when the wait ends)
already_satisfied the subject was already where the verb asked before the call, so no call went out (changed: 0, reason already_satisfied)
resolver {tier, matched} — which resolver pass named the subject (exact/alias/room_thing tier 1, all_words/verb_narrowed/group tier 2, kind_narrowed tier 3); null on a refusal
undo the verb (and value) that puts the subject back, when there is one; null for a scene or a stop
tier auto (acted), confirm (refused pending confirmation), deny (kind not allowed); the Gateway stamps trusted on a receipt learned trust ran
reason typed: already_satisfied (on an ok receipt), unknown_subject, unknown_verb, bad_value, kind_not_actionable, kind_not_allowed, confirmation_required, subject_missing_upstream, no_effect, upstream_refused, upstream_unreachable

Exit codes: 0 ok, 1 the upstream refused or the verb had no effect, 2 usage (unknown subject / verb / value, policy refusal), 3 upstream unreachable. A verb whose promised state never arrives is ok: false, no_effect, with both states in the receipt (“play” on an idle player with nothing queued).

Adding a controllable thing is one entry in the map file with kind light / switch / scene / media_player. Scenes never seed or flip; they take verbs only. The Admin projection shows the policy and the last 32 receipts.


What is first-party, and how something gets there

A first-party integration runs inside the Gateway, in Rust, and exists only for a real-time path (turn-taking, barge, Vulpine’s own voice in a room). It implements the same contract as a driver, selected in the rooms file, so nothing upstream of the contract changes when an integration is promoted. Promotion needs a measured miss against a stated budget, and proof that the miss is in the hop a native client removes rather than in the device’s own floor.

A test fails the build if any Rust file under crates/ names Home Assistant’s entity domains or WebSocket verbs (media_player., binary_sensor., subscribe_events, get_states).

Fleet adapter

The fleet adapter is not a device contract but has the same shape. vulpine-fleet (crates/vulpine_fleet, just install-vulpine-fleet; its usage text and the examples below call it fleetctl) runs outside the gateway. It speaks ssh so the gateway never does, and hands the gateway facts under Vulpine’s own labels. fleet_probe.py runs on each host: stdlib, read-only, no arguments, no stdin. It is the forced command behind the restricted probe key. fleetctl --manifest fleet.json run prints one line per interval:

{"type":"fleet_hello","adapter":"fleetctl/1","interval_s":30,"hosts":["gateway-host","laptop",…]}
{"type":"fleet_snapshot","at_ms":…,"interval_s":30,"hosts":[{
   "label","role","reachable","error","elapsed_ms",
   "hostname","os","kernel","arch","cpus","uptime_s","load":[1m,5m,15m],
   "mem_total_bytes","mem_available_bytes","disk_total_bytes","disk_free_bytes",
   "gpu_utilization_pct","gpu_model",
   "logins":[{"user","line","since"}],
   "ssh_sessions":[{"peer_label","peer_kind"}],
   "listeners","services",
   "herdr_present","herdr_version","herdr_server",
   "agents":[{"pane","tab","workspace","agent","status","cwd_base","title","focused","session_source","foreground",
              "ci"?:{"state","at_ms","label"?}}],
   "quota":[…],"agent_reports":[…],
   "tailnet","probe_ms"}],"sessions":{…}}
{"type":"fleet_error","error":"<class name>"}
  • error is a short code: permission_denied, timeout, unresolved, refused, unreachable, host_key, ssh_failed, spawn_failed, no_probe_output.
  • peer_label is a manifest label, link-local, local, other-lan or other. Raw addresses exist only under fleetctl probe --raw.
  • cwd_base is a basename and title the stripped terminal title. Pane text never crosses.
  • focused is true on the one pane herdr names as focused (focused_pane_id, or the agent’s own flag on a herdr without it).
  • ci is the fleet CI row: the latest GitHub Actions run of the agent’s checkout branch. state is one of failed / passed / running / pending / cancelled, at_ms the run’s stamp and label the workflow’s name. The probe asks the host’s own gh run list (read-only) once per distinct repo and branch, and caches every answer for 60 s in ~/.local/state/foxline/fleet-ci.json. A host without gh or a gh login sends no ci. The branch, repo and run id never leave the probe. Hosts probed through the restricted key run their installed probe copy, so reinstall the probe there to get their CI rows.

The gateway ([fleet], fleet.rs) keeps the latest row per host and diffs consecutive rows into reachable / unreachable / herdr_server_up / herdr_server_down / agent_added / agent_removed / agent_status. It projects GetFleet. A down host keeps its last-known agents, so an outage is one unreachable and one reachable row.

Agent notices. The same diff feeds the announcement board’s herdr source. An agent seen working that is now idle / done is offered as “<Agent> in <project> on the <host> is done.” (score 0.6). One now blocked / waiting / error is offered as “… needs you.” (0.9). Anything -> working, a first sighting and the probe’s own add/remove flaps say nothing. There is one notice per host:pane:agent per [fleet] agent_notice_cooldown_ms (default 10 min). [fleet] announce_agents = false turns the source off. The board speaks it like a camera line: the room presence picks the speaker, else the session voice does.

Agent presence. The same rows feed a cross-harness AgentPresence projection on GetFleet (agent_presence, absent until a sighting). It is built from the adapter’s rows, never from pane text (fleet_presence.rs). The brain reads it through pi_fleet_status.

  • One row per host:pane:agent, with harness (claude / codex / pi / herdr, from the agent name) and state (awaiting_input / busy / idle / error / compacting / done). The state folds the hook spool’s states and herdr’s: blocked reads awaiting_input, working reads busy.
  • since_ms is when the agent entered its state.
  • seen_watermark_ms moves only on the owner-local MarkAgentPresenceSeen {ids} (empty = all), or when a probe sees the agent’s herdr pane focused; never on a probe or a read. seen_reason says which (operator / focus).
  • An agent that went idle after working and has not been seen since reads done, with unseen: true: “finished while you weren’t looking”.
  • attention_events (newest first) record entering awaiting_input and each change of the fleet CI row (ci_failed / ci_passed / ci_running / ci_pending / ci_cancelled). A finishing one after the watermark makes the row unseen.

Meet adapter

The meeting inlet has the fleet adapter’s shape: meetctl runs outside the gateway and hands it typed events under Vulpine’s names on an owner-local unix socket. Nothing is on the WebSocket protocol. The inlet is off until [meet] enabled = true.

piece what it is where
meetctl the local two-leg Meet adapter: the operator’s mic (near) and the browser’s output (far), decoded, labeled and sent as typed events tools/meet/meetctl.swift
meeting.v1 the events it emits (meeting_started, meeting_segment, meeting_ended, meeting_forget, meeting_opt_out), their acks and refusal reasons docs/contracts/meeting.v1.md
ADR 0039 the design: two adapters, one transcript of record, consent before capture the ADR set

Was this page helpful?