---
title: "Project memory retrieval"
description: "How the brain looks up project memory: bounded, citation-bearing project evidence added before a turn from an index built by tools/retrieval/build_retrieval_index.py, marked untrusted, failing closed when the index is missing or malformed, without giving voice input tools, shell access or write authority (ADR 0007)."
type: "living"
source: "docs/retrieval.md"
verified: "2026-11-14"
group: "The voice path"
---

> **Living doc**, checked against the code 2026-11-14

Vulpine can add citation-bearing project evidence before a Pi turn without giving voice input any tool, shell,
recursive filesystem or write authority (ADR 0007). The gateway reads an index that
`tools/retrieval/build_retrieval_index.py` builds offline: a BM25 JSON index, or the SQLite FTS5 store. The evidence
is marked untrusted, and a missing or malformed index fails closed.

## Runtime boundary

1. `tools/retrieval/build_retrieval_index.py` reads only records admitted by `catalog/retrieval/sources.json`.
2. An operator supplies the named roots. A declared path must stay beneath its root after symlink resolution.
3. The only command adapters are fixed Argo project metadata, fixed Memento item listing and fixed Memento search.
   Manifest data cannot select an executable or an argument vector.
4. The builder writes an atomic, mode-`0600` JSON index. Unchanged records keep their `updated_at` values.
5. The gateway opens only the configured index, or the SQLite FTS5 store when `store_path` is set.
   - The JSON engine ranks with in-process BM25 (k1 1.5, b 0.75) plus authority boosts.
   - The store takes up to 200 FTS5 `bm25()` candidates and re-scores them with the same authority boosts. It ranks in
     the FTS index first (the best 400 rowids) and reads only those rows. When a score tie crosses that window's edge,
     or restricted rows leave it short, the full join answers instead, so the candidates never change.
   - The reader maps the store (`mmap_size` 256 MiB) and keeps a 64 MiB page cache. On the gateway host a read after a
     quiet gap takes 17-30 ms (2026-10-29).
   - Both engines then filter visibility, stale records and superseded coordinates the same way.
6. When `VULPINE_EVIDENCE_EMBEDDING_DIR` (or its old name, `FOXLINE_EVIDENCE_EMBEDDING_DIR`) names a Model2Vec model, the evidence floor drops a head record that is weak
   on both BM25 and cosine. At most `top_k` complete records are admitted under `max_context_bytes`. Evidence keeps
   its source coordinate, update time, authority and content hash.
7. Missing or malformed indexes fail closed. For project-state questions, Pi is told that source evidence is
   unavailable, so it is not invited to guess.

The gateway places evidence in nonce-delimited JSON Lines, each with its coordinate, visibility, authority, update
time, content hash and a stable citation token. A Pi system rule says the evidence is untrusted quoted data and
requires citation tokens for project-fact claims.

Generated indexes and benchmark traces stay local under `benchmarks/traces/` or the gateway's private state directory; they
are never committed. The brain tool `brain_work_status` reads live Memento items through `mm serve`, and its answer
overrides retrieved evidence for current item status.

## Canonical sources

| Source | Bounded interface | Authority | Freshness |
|---|---|---|---|
| Vulpine's glossary, changelog, an ADR, a handoff, a plan | Exact admitted Markdown files | glossary, project history, architecture decision, receipt, current plan | 7–90 days by source |
| Vulpine's tracker | Canonical repo-local `.memento/items` inventory through fixed `mm item list`; admitted Markdown bodies remain path-bounded | issue tracker | 24 hours |
| The nix host configuration's deployment digest | Exact admitted external Markdown file | declarative deployment receipt | 7 days |
| Argo | Fixed `argo project info` JSON call (its `repos` member) | Argo registry | 24 hours |
| Memento | Fixed `mm search query foxline` JSON call under a named root | derived search receipt | 24 hours |

Memento semantic-only matches are not promoted as exact project evidence. The index records an explicit
no-exact-evidence receipt when the lexical score is zero.


## `[retrieval]` config

| Key | Default | Meaning |
|---|---|---|
| `enabled` | `false` | Add evidence to brain turns. |
| `index_path` | `retrieval-index.json` | The JSON index. It serves every turn when `store_path` is unset, and it is the per-turn fallback when the store fails. |
| `store_path` | unset | The ADR 0031 SQLite FTS5 store. The gateway opens it once at startup and serves turns from it. |
| `top_k` | `4` | Most evidence records per turn (1 to 100). |
| `max_context_bytes` | `8192` | Byte budget for the evidence block (256 to 1,048,576). |
| `allowed_visibilities` | `["private"]` | Record visibilities a turn may see. |
| `operator_memory_dir` | unset | Operator-memory directory. Only the Admin health projection reads it. |
| `timeline_path` | unset | The gateway's own writable recall-by-clock timeline, a separate file from `store_path`, created owner-only (0600 in a 0700 directory) on first use. Setting it turns on the "keep that" verb and lets `pi_recall_window` answer, for example `~/.local/state/foxline/timeline.sqlite`. |

**Recall by clock.** With `timeline_path` set, the session holds its room lines in memory only, in a consent ring of
at most 15 minutes: the operator's committed requests, Vulpine's spoken segments, and an unaddressed final as a length
(`(N chars, not addressed)`, never its words). A spoken "keep that" writes the held lines from the last 15 minutes to
the timeline in one transaction and says how many it kept (trace `keep_that`); nothing is written without it. The verb
is a closed phrase set: keep / remember / save + that / this, "keep that one", "… what we just said", and "keep / save
the last few minutes". The brain tool `pi_recall_window` (a clock time or a range, `session` `any` or `current`) reads
a kept window back over the gateway mutation socket (trace `recall_window`). Unset `timeline_path` to turn it off.

**Meeting minutes.** The brain tool `pi_meeting_minutes` (`call_id`, or `last: true`) reads one stored call's
segments, written to the same timeline by the meeting inlet ([`meeting.v1`](/docs/contracts/meeting.v1)). The gateway
runs one tool-free prompt on the brain lane and returns typed minutes over the mutation socket: summary, decisions,
open questions, doubts and speakers (the contract's "Minutes" section has the rules). The model has
`[brain] minutes_deadline_secs` (default 45) before the call answers `timeout`, and a forgotten or opted-out call is
refused (trace `meeting_minutes`). The typed tool door serves both tools too: `vulpine-glyphctl meet minutes` and
`vulpine-glyphctl recall`.

Each `retrieval_end` trace names the `engine` (`json` or `sqlite`) and the `store_generation`. On the store it also
carries:

- `candidates_ms`: the candidate read alone. `duration_ms` minus it is the re-score, the evidence floor and the
  assembly.
- `rank_fallback`: the full join answered.
- the candidate read's split: `rank_ms` (the FTS index's rank) and `rows_ms` (the ranked rows' reads, their sort and
  any fallback).

The Admin's `gateway.retrieval` row shows the last turn's `last_ms`, `rank_ms` and `rows_ms`. If the store did not
open, or its query fails, the JSON index serves that turn and the trace records a `retrieval_store_fallback` event
with `reason` `not_open` or `query_failed`. Unset `store_path` to go back to the JSON index.

## Build and query

```bash
python3 tools/retrieval/build_retrieval_index.py \
  --manifest catalog/retrieval/sources.json \
  --output "$HOME/Library/Application Support/Foxline/state/retrieval/index.json" \
  --root "foxline=$PWD" \
  --root "macmini_nix=<path to the nix host configuration>" \
  --root "memento=<path to the memento checkout>" \
  --sqlite "$HOME/Library/Application Support/Foxline/state/retrieval/memory.sqlite"

cargo run -p vulpine-voice-gateway --bin vulpine-retrieval-query -- \
  --index "$HOME/Library/Application Support/Foxline/state/retrieval/index.json" \
  --query "Where did we leave Fox9?"
```

`--sqlite` is optional. It writes or updates the store in place and adds a new generation only when a record changed.
`runtime/operator_memory/retrieval_store.py` does the same from an existing index. `vulpine-retrieval-query` reads the
JSON index only.

