---
title: "Avatar rigs"
description: "The rig side of the avatar wire: how every rig (VRM, Live2D, the 2D fox) draws typed picks, with the VRM 1.0 override table, the gaze split over spine, chest, neck, head and eyes, the VRMA rest-pose retarget, the face director (1.1 s dwell, 0.85 apex cap, per-family onset and release, blinks) and the gated five-question decision.v1 set."
type: "living"
source: "docs/contracts/avatar-puppet.v1.md"
verified: "2026-11-14"
group: "Contracts"
---

> **Living doc**, checked against the code 2026-11-14

avatar-puppet.v1 — what a rig draws from typed picks: overrides, gaze, clips, the director and the gate

- Status: **live in the native kit**. No wire frame. The VRM scene, the Live2D scene and the puppet2d fox all consume
  it (`apps/native-kit/src/surface3d/avatar.rs`, feature `surface3d`). The kit holds the wire and the
  `ExpressionDriver` seam and links no model client; the real client (`Decidectl`) is the app's, in
  `apps/native-kit-edge`.
- Where it is seen: the fox window's Avatar look (a Live2D portrait) and its VRM look (a `.vrm` read in place) take
  the same commands from one driver (`apps/fox-gpui/src/avatar.rs`), so one stream of relayed avatar actions directs
  either.
- Scope: the rig side of [`avatar-action.v1`](/docs/contracts/avatar-action.v1), which carries closed verbs and names from the
  gateway to a frontend. This one says how a rig turns typed picks (an expression's weight, a face, a focus target, a
  clip) into what it draws, the same way on every rig, so a model's picks never flicker or fight. The model's wire is
  [`decision.v1`](/docs/contracts/decision.v1). The brain never sets a weight.
- Doctrine: a decision-like model gives typed expression, gaze and gesture picks at sub-second latency. The avatar
  need not be humanoid. The rules are plain data, so a rig that is not a VRM keeps the semantics and maps them onto its
  own parameters by a recipe.

## Expressions and picks

Eleven expressions, named by their VRM 1.0 presets: the emotions `happy`, `angry`, `sad`, `relaxed`, `surprised`;
`blink`; the lip vowels `aa`, `ih`, `ou`, `ee`, `oh`. A pick is one expression's weight, `0..=1`, held for a time or
until the next pick (`Command::Pick`). decision.v1 `noul` answers become picks through `picks_from_nouls` and the
`expression` choice through `picks_from_answer`, both behind the reply guard. A rig maps each expression onto what
it has: a VRM onto its preset or the ARKit channels its preparation composed (`vrm/presets.rs`), a Live2D model and
the puppet2d fox by a recipe (a share of each parameter's range per expression).

**Layers meet by max, never by sum.** A frame's raw weight for an expression is the largest of its eased pick, the
director's layer (its face, the blink schedule) and, for a vowel, the lip track and the played mouth (`Ease::drawn`).

**The played mouth.** `Command::Mouth(lips)` is the mouth from the audio as it is played: the five lip weights a host
read off its player's level, one reading at a time. The ease follows the newest reading quickly open
(`MOUTH_OPEN_TAU_S`, 35 ms) and more slowly shut (`MOUTH_CLOSE_TAU_S`, 90 ms). It shuts the mouth on its own once no
reading came for `MOUTH_STALE_S` (0.4 s). An open played mouth engages the director as a spoken line does (the blinks
run). A lip track (`Command::Speak`, the words' syllable estimate) plays beside it by max. A host that has the played
level sends the track's shape inside its readings instead (Vulpine.app's Avatar portrait, `fox-gpui`'s
`avatar_mouth.rs`).

## Overrides

Every rig gets one table (`face.rs`). It follows VRM 1.0's rule (`overrideBlink`, `overrideLookAt`,
`overrideMouth`): each expression adds a rate to each category it overrides (`block`: 1 once its weight is above 0;
`blend`: its weight), and each category is drawn at `1 - min(rate, 1)` of its weight. The categories are the blink,
the eyes' look-at and the five vowels. The emotions are never overridden.

| expression | blink | look-at | mouth |
|---|---|---|---|
| happy, angry, sad, relaxed | blend | none | none |
| surprised | block | none | none |
| blink, aa, ih, ou, ee, oh | none | none | none |

So a smile and a blink on one frame do not stack past shut, wide eyes do not blink, and a smiling fox still talks. A
VRM's own declared overrides apply inside its preset on top (`compose_with` takes another table).

## Gaze

One aim, degrees in the character's frame (yaw positive toward the character's left, the viewer's right; pitch
positive up), shared out over the body and the eyes (`gaze.rs`):

| bone | share | yaw limit | pitch limit |
|---|---|---|---|
| spine | 0.15 | 15 | 0 |
| chest | 0.20 | 20 | 0 |
| neck | 0.25 | 25 | 20 |
| head | 0.40 | 40 | 30 |

The body follows the aim at 10 a second (exponential) and each bone its share at 25 a second. The eyes aim at the raw
target at once and take what the body has not turned yet, within 12° yaw and 10° pitch: the eyes lead, the head
catches up, the eyes re-center. A look-at override scales the eyes' share. A pick is avatar-action.v1's `focus`
target: `user` (ahead), `away` (30° aside), `up` / `down` (18°), `point` x, y in `-1..=1` (45° and 30° at the edges).

Where it lands:

- A VRM turns the spine, chest, neck and head on its animator's base pose (normalized space) and the eyes on the
  `eyeLook*` channels.
- A Live2D model takes the body on `ParamBodyAngleX`, the head on `ParamAngleX` / `ParamAngleY` and the eyes on
  `ParamEyeBallX` / `ParamEyeBallY`.
- The puppet2d fox has no eyes to lead with, so it turns `angle_x` by the whole aim.

## Clips

One clip library plays on every rig by the VRM animation spec's rest-pose formula (`retarget.rs`): a source bone's
local rotation is normalized against the source's local and global rests, then denormalized onto the destination's:
`dst = dst_local * dst_global^-1 * (src_global * src_local^-1 * src * src_global^-1) * dst_global`. A clip authored on
one VRM means the same motion on another VRM or the 3D fox rig. A clip is picked by the gate's `what` question (below)
or avatar-action.v1's `play_animation`.

The VRM scene plays clips from `.vrma` files. The loader's reader (`apps/gpui-scene3d-gltf-vrm/src/vrma.rs`) applies
the same formula onto VRM's normalized space and gives the hips' motion as an offset the scene scales by its model's
hips height. The scene's clip player (`apps/native-kit/src/surface3d/vrm/clips.rs`) loops a clip on its clock and
crossfades every start, switch and stop over 0.4 s. A clip replaces the rotations of the bones it names. The gaze,
the breath and the springs are laid on after.

## The director

A face pick (`Command::Face`: a name, a family, a mix of emotion presets, an intensity) goes through the director
(`director.rs`):

- **Dwell 1.1 s**: no other face replaces the one on show sooner. A later offer waits and starts when the dwell ends
  (a still later one replaces it). An `alert` face (surprised, shocked, afraid, curious) cuts in at once.
- **Apex cap 0.85**: no emotion preset is driven past it.
- **Per-family onset and release** (seconds): alert 0.12 / 0.30, warm 0.30 / 0.42, down 0.45 / 0.60, hot and calm
  0.25 / 0.42. A face rises over its onset (the old face blends out under it), holds its apex 0.6 to 1.2 s, settles to
  a sustain of 0.55 to 0.85 over 0.5 s, and lets go 4.5 s after its last renewal. A renewal (the same face again)
  extends the hold, never past 13.5 s from the face's start. The variation is seeded, so a scripted stream replays.
- **Neutral** lets the face on show go and is over once its onset has blended it out.
- **Blinks**: every 2.4 to 6 s (closing 60 ms, shut 30 ms, opening 90 ms), one in seven a double, while a face is
  held or a line is spoken, or while the scene keeps its avatar alive at rest (`Ease::alive`, a scene's idle life).
  Otherwise at rest the eyes stay open and the view draws nothing. The override table then thins a blink under an
  emotion.

The decision model's `expression` choice reaches the director through `face_from_answer` (the reply guard, then
p >= 0.6). A caller that wants weights uses the raw picks (`picks_from_answer`).

## Picks over decision.v1

Five questions about a reply (`instinct.rs`; `benchmarks/verbs/avatar-instinct.v1.json` holds a sample request for
the fox's clips):

| id | type | options |
|---|---|---|
| `performs` | noul | is the avatar committing to an action in its own current words |
| `what` | choice | the rig's declared clips, each an action or a gesture, and `none` |
| `reaction` | choice | agree, disagree, empathize, celebrate, recoil, ponder, bashful, scoff, none |
| `face` | choice | neutral and 21 faces over the five emotion presets (`FACES`) |
| `energy` | choice | gentle, moderate, high (gentle shows a face at 0.75) |

The gate (`Gate::gate`):

| what | acts when |
|---|---|
| any answer | it passes decision.v1's own checks (typed as asked, a choice among its options, probabilities summing to 1, a noul in `0..=1`); a missing or refused answer abstains for its question alone, never a no |
| a face | p >= 0.6 |
| a reaction | the reply's words are final, p >= 0.75, and 5 s since the last reaction |
| a clip | p >= 0.6 on final words, 0.8 (gesture) or 0.85 (action) on a partial; an action also needs `performs` >= 0.5 (final) or 0.6 (partial); one clip a turn on partials |
| a veto | `performs` <= 0.2 with `what` at `none` >= 0.7 vetoes every clip |

A choice's p is its `confidence`, else its own probability, else 0 (no probability, no act).

## Errors

Nothing here is on a wire, so nothing is refused to a peer. A reply the guard refuses moves nothing (`PickError`). A
gated answer under its threshold is counted (`GateCounts`: `faces_under`, `reactions_under`, `reactions_cooling`,
`vetoes`, `abstains`) and does not act.

## Compatibility

Additive. `surface3d::vrm` re-exports the picks, the ease, the lips, the expression question and the
`ExpressionDriver` seam; `Decidectl` is `apps/native-kit-edge`'s.
`Command` ends with `Face`, `Gaze` and then `Mouth`. A rig that ignores the director and the gaze draws its picks
under the override table.


## See also

- [`avatar-action.v1`](/docs/contracts/avatar-action.v1) — the verbs and names a rig declares and the gateway relays.
- [`decision.v1`](/docs/contracts/decision.v1) — the typed questions' wire and its reply guard.
