Files in the panel
A file the brain shows (file.v1, pi_show_file): the request (path, title, page), the gateway's checks in order (the allowed folders [files] roots after symlinks, no dot-dot, no hidden or secret part, a regular file within [files] max_bytes) and their typed, sayable refusals, the type sniffed from the bytes, the FileSpec body (name, mime, size, sha256, page), who gets the file and who its card, the file_fetch / file_chunk / file_fetch_refused frames and who may fetch, and what the Admin row logs.
Living doc, checked against the code 2026-11-14
file.v1 — a file the gateway shows from its own disk: named on the render, fetched by its hash
- Status: landed on protocol 5, additive. Decided in ADR 0049.
- Scope: the visual response channel’s file tier, beside
panel-card.v1andvisual.v1. The brain shows a file that already exists (a picture, a PDF, a document, a log, a video, a glTF model) instead of data it wrote.
What the brain sends
pi_show_file {path, title?, page?}:
path: the file’s full path on the gateway’s machine (~/allowed).title: optional plain words for the pane’s header, at most 80.page: a PDF’s page to open at, from 1.
The tool sends the gateway’s show_render mutation line with kind file.v1 and that request as its body. The bytes
never pass through the brain.
What the gateway does with the request
The mutation listener resolves it on a blocking thread before the line reaches the session (crates/voice_gateway/src/file_show.rs), in this order:
[files] enabledis on, elsefile_off.- The path is absolute (or
~/, the gateway user’s home), at most 4096 bytes, with no NUL and no.., elsebad_path. - The real path (every symlink followed) is under a real allowed folder (
[files] roots), elsenot_allowed. A path outside every folder is refused whether it exists or not. Inside a folder, a missing file isnot_found. - No part of the real path below its folder, nor of the asked path below a folder it names, is hidden (a dot name:
.ssh,.env,.git) or a secret’s name (id_rsa,id_dsa,id_ecdsa,id_ed25519, each also with.pub,authorized_keys,known_hosts,credentials,credentials.json,secrets.json,secrets.toml,secrets.yaml,token.json,netrc, a name or stem that sayspassword,passwords,walletorseed-phrase, or an extensionpem,key,p12,pfx,keychain,keychain-db,kdbx,ovpn,ppk), elsesecret_path. - It is a regular file, else
not_a_file(a folder, a device, a pipe, a socket). It is at most[files] max_bytes, elsetoo_large. It is readable, elseunreadable. - Its type is sniffed from its first 8 KiB (below).
pageapplies to a PDF only, from 1 to 100000, elsebad_page.
A refusal is answered at once, in the session’s render_shown shape, and the line never reaches the session:
{"status": "render_shown", "kind": "file.v1", "outcome": "refused", "reason": "secret_path", "waited_ms": 1,
"error": "that file is hidden or holds keys or secrets, so I will not show it"}
error is plain words the brain may say: never a path, a hash or markup. not_allowed names the allowed folders by
name, Desktop, Documents, Downloads, Pictures and Movies. unreadable says the gateway may need the Mac’s permission
for that folder.
A file kept in iCloud. With Optimize Mac Storage, an iCloud folder holds dataless placeholders (SF_DATALESS).
Their bytes download on the first read only when the reader’s materialization policy is on (man setiopolicy_np); a
launchd agent’s default is off, and the read then fails with EDEADLK. The gateway turns the policy on for each
shown or fetched file’s read (crates/voice_gateway/src/file_icloud.rs). A dataless file that still fails is
not_downloaded (“that file is kept in iCloud and is not downloaded on this Mac, and it could not be downloaded just
now; open it once on the Mac and ask again”), counted on the gateway.files row apart from unreadable.
macOS’s folder permission. macOS guards Desktop, Documents and Downloads per process (Privacy & Security > Files
and Folders); Pictures and Movies are not guarded. The grant belongs to the gateway binary (foxline-voice-gateway).
It is keyed to the binary’s path and anchored on its signing identifier and certificate, so an update keeps it.
The gateway reads each guarded folder’s state without asking (TCC’s preflight) and never opens one it may not, so
macOS never prompts during a turn. A find walks the other folders. With no match there, or for a path inside the
folder, the refusal is folder_not_granted (“I can’t read your Documents folder yet: … Allow it in the Glyph’s
welcome window, on the Documents Folder row”). A grant made while the gateway runs is restart_to_apply: macOS hands
a grant over only at a restart.
Vulpine.app’s welcome window on the gateway’s Mac walks the grant, through the gateway’s
<state>/file-access/file-access.json. Allow… makes the gateway open the folder once, so macOS asks then. Open
System Settings opens Files and Folders. Restart gateway kickstarts its launchd job. A tool refused for a folder opens
the window at its row. The Admin’s gateway.file_access Health row shows each folder’s state.
A root on another volume is guarded by that volume’s grant: Removable Volumes for a local drive under /Volumes,
Network Volumes for a share. The gateway reads which applies from the mount table at start, touching nothing on the
volume. The root is held, asked and reported like a home folder, with the words “I can’t read your external drives
yet: … on the Removable Volumes row”. A refused read there names that row instead of unreadable.
A file that passes replaces the request with its body. The gateway keeps it by its hash for a later fetch and sends it through the session’s render router.
[files] (gateway config)
| key | default | meaning |
|---|---|---|
enabled |
true |
show files at all; false refuses each with file_off |
roots |
["~/Desktop", "~/Documents", "~/Downloads", "~/Pictures", "~/Movies"] |
the allowed folders; one that does not exist is left out |
max_bytes |
33554432 (32 MiB) |
the largest file shown; held to at most 256 MiB (FILE_MAX_BYTES_CEILING) |
find |
true |
find a file by description (below); false refuses each with find_off |
find_min_margin |
0.5 |
the decision model’s pick of a close find is taken at or above this margin (0..=1) |
Finding a file by description
The find runs on the gateway’s machine only. The brain never lists the operator’s folders and never sees a file
name, folder, size or date it did not name itself. pi_show_file takes {describe, folder?, title?} in place of
{path, title?, page?}: never both, never a page. describe is the operator’s own words (1..=300 characters, one
line). folder is one of desktop, documents, downloads, pictures, movies. The gateway finds the file on its
own disk (crates/voice_gateway/src/file_find.rs):
- The name index. A bounded walk of the allowed folders (or the one
foldernames; a folder not allowed isnot_allowed): at most 4 s, 100,000 files and 12 folders deep, kept 120 s. It never follows a symlink, never enters a hidden folder, a secret-named one or a package (an app, a Photos or Music library), and never keeps a file the show checks would refuse (a hidden or secret name, overmax_bytes). The index is the gateway’s own, not Spotlight’s. Its name matching is a port of fsearch’s (v0.2.1, MIT): a whole word ranks above a word’s start, and that above a run inside a word. One typo is forgiven in a word of 5 letters or more, two in one of 8 or more, and a digit is never edited. A recent change lifts a file by a two-week half-life. A word of 5 letters or more that no typo reaches still matches, below a typo, when it sounds like the name’s word: the same consonant key (schassh,phasf,ckand a hardcask, a softcass, vowels after the first letter dropped, doubles once) of at least 4 letters. So a name the recognizer spelled by ear (“Shooler” for “Schuler”) still matches. The words go through a stop list. Kind words (“the PDF”, “a screenshot”) become a kind hint, time words (“latest”) weigh recency up, and folder words favor that folder. A file scoring under 40 is no candidate. - The pick. One clear match (a score of 90 or more and 40 above the next) is shown. Close ones are asked of the
served decision model: decision.v1, one
choicequestionfile_pickoverfile_1..file_8whose descriptions are the candidates’ names, folders, kinds and ages, plusnone, with the user’s words asstate.utterance. The model is asked only when every entry of[decision_shadow] entriesis on this machine (a loopback URL), never with a model load, and within 2.5 s. Its pick is taken when the guarded answer’s margin reachesfind_min_margin. Otherwise the window gets a chooser. - The chooser. The body is the best match’s
FileSpecwithtitle“Which file did you mean?” andchoices(below): up to three files, each read, hashed and kept for a fetch like a shown file, and granted with the render. The conversation window draws the choices and fetches none until one is tapped. The tap fetches that choice by its hash (file_fetch). A card frontend gets the list of names and “Choose one in the conversation window.” - Each file shown or offered goes through the show checks again (
file_show::resolve): the same refusals, read and hash as a path.
What the brain hears is the render’s answer (outcome, reason, projected_to) plus found (shown or
asked_user), by (name, model or user), file_kind (the shown file’s kind in words, “PDF”) and choices
(how many), or a refusal: no_match (“I found no file like that in Documents, Pictures and Desktop”), find_off,
folder_not_granted or restart_to_apply (a guarded folder the find could not walk, when nothing else matched),
find_timeout (“the search took too long; name the folder it is in, or give its full path”: the find ran past the
listener’s 15 s wait), or any show refusal. It is never a name, a folder, a size, a date or a hash. The tool’s result
has no name in this case, and its line says only the kind (“The PDF is on the screen.”) or the chooser (“I put 3
files that may match on the screen; choose one there.”). The receipt keeps found, file_kind and choices, never
the words.
Which tool asks. The operator prompt (catalog/agents/operator/SYSTEM.md, AGENTS.md) sends any file the
operator asks to see, open or read to pi_show_file with describe: their words as heard, never a spelling corrected
or guessed. pi_file_searcher searches the agent’s own workspace only; its out-of-workspace refusal names
pi_show_file describe.
The sniffed types
| bytes | mime |
|---|---|
89 50 4E 47 0D 0A 1A 0A |
image/png |
FF D8 FF |
image/jpeg |
GIF87a, GIF89a |
image/gif |
RIFF … WEBP / WAVE |
image/webp / audio/wav |
II*\0, MM\0* |
image/tiff |
BM with a Windows or OS/2 header size |
image/bmp |
ftyp at 4: brands heic heix hevc heim heis mif1 msf1 / avif avis / qt / M4A / others |
image/heic / image/avif / video/quicktime / audio/mp4 / video/mp4 |
%PDF- |
application/pdf |
glTF |
model/gltf-binary |
1A 45 DF A3, OggS, fLaC, ID3 or an MPEG frame, PK\x03\x04 |
video/webm, audio/ogg, audio/flac, audio/mpeg, application/zip |
UTF-8 text (no NUL, at most 1 % control characters) holding <svg |
image/svg+xml |
| other UTF-8 text, by its extension | text/markdown, text/csv, application/json, model/gltf+json, text/x-rust, text/x-python, text/javascript, text/x-typescript, text/x-swift, text/x-go, text/x-c, text/x-c++, text/x-java, text/x-ruby, text/x-shellscript, text/x-toml, text/x-yaml, text/html, text/css, text/x-sql, text/x-lua, text/x-shader, text/x-nix, else text/plain |
| anything else | application/octet-stream |
The extension only names a text’s language: a .png that holds words is text/plain, and a PDF named .bin is
application/pdf.
FileSpec, the body of file.v1
| field | type | bound |
|---|---|---|
title |
string | card.v1’s title rule (1..=80, plain, no hex token): the brain’s title, else the file’s name, else its kind (“A PDF”) |
summary |
string | card.v1’s summary rule: the kind and size in words, and the page (PDF, 2.1 MB, page 3) |
name |
string | 1..=255 characters, no /, no markup or control character: the file’s name, never its folder |
mime |
string | lowercase type/subtype, at most 100 characters (the table above) |
size |
number | bytes, at most 256 MiB |
sha256 |
string | 64 lowercase hex digits: the name a window fetches it by |
page |
number, optional | a PDF’s page to open at, 1..=100000 |
choices |
array, optional | a find’s chooser, 1..=3 {name, folder, mime, size, sha256} (name as above; folder at most 120 characters below the allowed folder, Documents/Taxes, no ..); absent (never empty) on the wire for a plain file |
Closed (deny_unknown_fields). The protocol’s validate_render checks every bound before relay.
Who gets what
| frontend | declares | gets |
|---|---|---|
| the conversation window (apps/convo-gpui, an observer) | file.v1 in its hello |
the render whole; it fetches the bytes and draws them |
| Vulpine on iPhone (apps/phone) | file.v1 in its hello |
the render whole; phone-core fetches an image of at most 8 MiB by its hash (crates/phone_core/src/link/fetch.rs) and the card draws it (apps/phone/src/convo/pictures.rs); any other file shows its name |
| Vulpine.app and Vulpine in a browser (the Glyph and Vigil looks; they draw cards) | card.v1, not file.v1 |
the file’s card projection (file_card: its title, its summary as the one block, the note “A file; it shows in full in the conversation window.”); the brain’s answer says projected_to: card.v1 |
| a frontend that draws neither | nothing | nothing (dropped, undeclared) |
The session’s observers are offered the render whole, even when the session’s own frontend got its card. An observer
that did not declare file.v1 gets the card (ws::observe::observer_frame). The panel kit draws a file.v1 frame as
its card and never declares the kind (KIT_UNDECLARED_FRAMES).
Fetching the bytes
| frame | direction | fields |
|---|---|---|
file_fetch |
client → gateway | sha256 |
file_chunk |
gateway → client | sha256, offset, total, data (base64 of at most 512 KiB, FILE_CHUNK_BYTES) |
file_fetch_refused |
gateway → client | sha256, reason |
- Who may fetch: a connection whose hello declared
file.v1, for a hash shown to its own session or to the session it observes (granted at theshow_renderdispatch). Elsenot_declaredornot_shown. A hash never shown isnot_showntoo. The gateway keeps the last 64 shown files for two hours. - Checked again: the gateway reads the file again under the policy in force (folders, secrets, kind, cap) and hashes
it:
file_changedwhen the hash differs,file_gonewhen it left or was removed,file_offwhen the switch went off. - In order: the chunks of one file go in order on the connection. The file is whole when
offset + bytes == total(an empty file is one chunk with no data). The receiver checks the whole file’s SHA-256 against the name it asked for and refuses a mismatch (bad_hash) or a piece out of order (bad_chunk). - The window’s side (apps/convo-gpui/src/files.rs): it asks once per hash when it first draws the panel
(
LinkCommand::FetchFile). It keeps whole files by hash, at most 256 MiB, dropping the least recently drawn first. It forgets a fetch in flight when its link drops, so the next draw asks again. The link (crates/convo_feed) only carries the frames. An observer may sendfile_fetch, because it acts on nothing.
The answer
A file that passed goes through the session’s render router like any render (drawn, refused, dropped, or sent
unconfirmed; projected_to when a frontend got the card). The tool’s answer adds the file’s name (never its folder)
and the line the brain may say (floor-plan.pdf is on the screen., The card for floor-plan.pdf is on the panel; the file itself shows in the conversation window., or the refusal’s words). The receipt keeps the outcome only: never the
path, the name or the title.
Logged
- The Admin’s
gateway.filesHealth row: the switch, the folders and the cap; today’s files shown, fetched by a window (with the bytes) and refused by reason (not_allowed 2, secret_path 1). The host’s local day. - The session trace:
file_fetched {bytes, ms}andfile_fetch_refused {reason}. - Never a path, a file name or a hash in either.
- The same row adds the finds by description today (shown by name, by the decision model, asked on screen with the
choices tapped by place, no match), why the decision model gave no pick (
model_not_local,no_model,low_margin,abstained,none), and the name index’s size, age and unreadable folders.
Drawing it (the conversation window)
The pane (apps/convo-gpui/src/file_pane.rs; apps/convo-gpui/README.md “Files”) draws a file.v1 panel through a
viewer registry keyed by media type (apps/convo-gpui/src/file_view/), asked newest registration first, the fallback
last:
| viewer | types | how |
|---|---|---|
| image | PNG, JPEG, GIF, WebP, BMP, TIFF; HEIC and AVIF through AppKit | at most 4096 px a side, EXIF orientation; Fit, 1:1 and 2x, pan by scrolling; an animated GIF plays 8 s, then holds (never under Reduce Motion) |
| svg | image/svg+xml |
GPUI’s resvg renderer at the window’s scale; over 8 MiB refused |
application/pdf |
CGPDFDocument, a page at a time at points x scale, opening on page (clamped); Previous, “Page n of m”, Next, the zoom buttons; a locked PDF says so |
|
| markdown | text/markdown |
the reply’s markdown blocks |
| code | text/x-*, JSON, HTML, CSS, JavaScript |
monospace with line numbers; syntax colors (keywords, strings, comments, numbers, type names) from the theme’s tokens for every code type above, by a small tokenizer in the window (apps/convo-gpui/src/file_view/syntax.rs), with a Syntax colors toggle (on) |
| text | plain text, CSV | monospace |
| video | video/* (MP4 and QuickTime play; WebM and other types AVFoundation cannot read say so) |
the clip’s first frame, then Play / Pause / Play again and an ASCII look toggle (off at first); AVAssetReader decodes the video track only (the sound is never read, “sound off”) into NV12 frames that GPUI’s surface() draws; a track whose matrix turns its picture (a phone’s portrait clip) is read through AVFoundation’s video composition instead, so it draws upright at its turned size; a frame is asked only while it plays, so it rests at 0 frames when paused, at its end and under Reduce Motion (its first frame only) |
| model | model/gltf-binary, and model/gltf+json whose buffers and textures are data URIs (one that names files beside it is drawn by the code viewer under a line saying why) |
glTF’s reader first (bounds, clips, morph targets; a texture past 4096 px refused before it decodes), then kiss3d 0.47 on the window’s wgpu device in a surface3d layer clipped to its card; the first clip plays once (no clip: the morph targets sweep), Play and Reset view, a drag turns it, the wheel zooms; 0 frames after the clip and the drag, nothing plays under Reduce Motion; two live at most (an older pane: Show in 3D); the model3d feature, on |
| fallback | anything else (audio, zip, a Word, Excel, Keynote or Pages file) | Quick Look’s thumbnail when it makes one (QLThumbnailGenerator, asked off the main thread: a document’s first page, a song’s artwork), else the file type’s icon (NSWorkspace iconForFile:); both read the bytes kept owner-only under the window’s cache folder (the newest 16); its name, kind and size. A Word document (.docx, .doc) or an OpenDocument text (.odt), told by its name under application/zip or application/octet-stream (the gateway’s sniff) or its own type, is also laid out off the main thread by AppKit’s reader and TextKit into a PDF of at most 200 pages (apps/convo-gpui/src/file_view/office.rs); once it is, the pane draws its pages as the pdf viewer does (Previous, “Page n of m”, Next, opening on page); until then, and when it cannot be read, the thumbnail |
The text viewers draw at most 400 lines within 128 KiB, then “N more lines”. The log’s own fenced code blocks are
colored by the same tokenizer when the fence names a language (```rust;
apps/convo-gpui/src/code_colors.rs), at most 16 KiB a block. A file over 2 MiB is prepared off the main thread
(“Preparing the file…”). A new viewer is one Viewer type and one registry.register line in Registry::standard;
the pane does not change.