Skip to content
Vulpine
Esc
↑↓navigate↵open⌘Jpreview
On this page

Avatar rigs

The rig side of the avatar wire: how every rig (VRM, Live2D, the 2D fox) draws typed picks, with the VRM 1.0 override table, the gaze split over spine, chest, neck, head and eyes, the VRMA rest-pose retarget, the face director (1.1 s dwell, 0.85 apex cap, per-family onset and release, blinks) and the gated five-question decision.v1 set.

Living doc, checked against the code 2026-11-14

avatar-puppet.v1 — what a rig draws from typed picks: overrides, gaze, clips, the director and the gate

  • Status: live in the native kit. No wire frame. The VRM scene, the Live2D scene and the puppet2d fox all consume it (apps/native-kit/src/surface3d/avatar.rs, feature surface3d). The kit holds the wire and the ExpressionDriver seam and links no model client; the real client (Decidectl) is the app’s, in apps/native-kit-edge.
  • Where it is seen: the fox window’s Avatar look (a Live2D portrait) and its VRM look (a .vrm read in place) take the same commands from one driver (apps/fox-gpui/src/avatar.rs), so one stream of relayed avatar actions directs either.
  • Scope: the rig side of avatar-action.v1, which carries closed verbs and names from the gateway to a frontend. This one says how a rig turns typed picks (an expression’s weight, a face, a focus target, a clip) into what it draws, the same way on every rig, so a model’s picks never flicker or fight. The model’s wire is decision.v1. The brain never sets a weight.
  • Doctrine: a decision-like model gives typed expression, gaze and gesture picks at sub-second latency. The avatar need not be humanoid. The rules are plain data, so a rig that is not a VRM keeps the semantics and maps them onto its own parameters by a recipe.

Expressions and picks

Eleven expressions, named by their VRM 1.0 presets: the emotions happy, angry, sad, relaxed, surprised; blink; the lip vowels aa, ih, ou, ee, oh. A pick is one expression’s weight, 0..=1, held for a time or until the next pick (Command::Pick). decision.v1 noul answers become picks through picks_from_nouls and the expression choice through picks_from_answer, both behind the reply guard. A rig maps each expression onto what it has: a VRM onto its preset or the ARKit channels its preparation composed (vrm/presets.rs), a Live2D model and the puppet2d fox by a recipe (a share of each parameter’s range per expression).

Layers meet by max, never by sum. A frame’s raw weight for an expression is the largest of its eased pick, the director’s layer (its face, the blink schedule) and, for a vowel, the lip track and the played mouth (Ease::drawn).

The played mouth. Command::Mouth(lips) is the mouth from the audio as it is played: the five lip weights a host read off its player’s level, one reading at a time. The ease follows the newest reading quickly open (MOUTH_OPEN_TAU_S, 35 ms) and more slowly shut (MOUTH_CLOSE_TAU_S, 90 ms). It shuts the mouth on its own once no reading came for MOUTH_STALE_S (0.4 s). An open played mouth engages the director as a spoken line does (the blinks run). A lip track (Command::Speak, the words’ syllable estimate) plays beside it by max. A host that has the played level sends the track’s shape inside its readings instead (Vulpine.app’s Avatar portrait, fox-gpui’s avatar_mouth.rs).

Overrides

Every rig gets one table (face.rs). It follows VRM 1.0’s rule (overrideBlink, overrideLookAt, overrideMouth): each expression adds a rate to each category it overrides (block: 1 once its weight is above 0; blend: its weight), and each category is drawn at 1 - min(rate, 1) of its weight. The categories are the blink, the eyes’ look-at and the five vowels. The emotions are never overridden.

expression blink look-at mouth
happy, angry, sad, relaxed blend none none
surprised block none none
blink, aa, ih, ou, ee, oh none none none

So a smile and a blink on one frame do not stack past shut, wide eyes do not blink, and a smiling fox still talks. A VRM’s own declared overrides apply inside its preset on top (compose_with takes another table).

Gaze

One aim, degrees in the character’s frame (yaw positive toward the character’s left, the viewer’s right; pitch positive up), shared out over the body and the eyes (gaze.rs):

bone share yaw limit pitch limit
spine 0.15 15 0
chest 0.20 20 0
neck 0.25 25 20
head 0.40 40 30

The body follows the aim at 10 a second (exponential) and each bone its share at 25 a second. The eyes aim at the raw target at once and take what the body has not turned yet, within 12° yaw and 10° pitch: the eyes lead, the head catches up, the eyes re-center. A look-at override scales the eyes’ share. A pick is avatar-action.v1’s focus target: user (ahead), away (30° aside), up / down (18°), point x, y in -1..=1 (45° and 30° at the edges).

Where it lands:

  • A VRM turns the spine, chest, neck and head on its animator’s base pose (normalized space) and the eyes on the eyeLook* channels.
  • A Live2D model takes the body on ParamBodyAngleX, the head on ParamAngleX / ParamAngleY and the eyes on ParamEyeBallX / ParamEyeBallY.
  • The puppet2d fox has no eyes to lead with, so it turns angle_x by the whole aim.

Clips

One clip library plays on every rig by the VRM animation spec’s rest-pose formula (retarget.rs): a source bone’s local rotation is normalized against the source’s local and global rests, then denormalized onto the destination’s: dst = dst_local * dst_global^-1 * (src_global * src_local^-1 * src * src_global^-1) * dst_global. A clip authored on one VRM means the same motion on another VRM or the 3D fox rig. A clip is picked by the gate’s what question (below) or avatar-action.v1’s play_animation.

The VRM scene plays clips from .vrma files. The loader’s reader (apps/gpui-scene3d-gltf-vrm/src/vrma.rs) applies the same formula onto VRM’s normalized space and gives the hips’ motion as an offset the scene scales by its model’s hips height. The scene’s clip player (apps/native-kit/src/surface3d/vrm/clips.rs) loops a clip on its clock and crossfades every start, switch and stop over 0.4 s. A clip replaces the rotations of the bones it names. The gaze, the breath and the springs are laid on after.

The director

A face pick (Command::Face: a name, a family, a mix of emotion presets, an intensity) goes through the director (director.rs):

  • Dwell 1.1 s: no other face replaces the one on show sooner. A later offer waits and starts when the dwell ends (a still later one replaces it). An alert face (surprised, shocked, afraid, curious) cuts in at once.
  • Apex cap 0.85: no emotion preset is driven past it.
  • Per-family onset and release (seconds): alert 0.12 / 0.30, warm 0.30 / 0.42, down 0.45 / 0.60, hot and calm 0.25 / 0.42. A face rises over its onset (the old face blends out under it), holds its apex 0.6 to 1.2 s, settles to a sustain of 0.55 to 0.85 over 0.5 s, and lets go 4.5 s after its last renewal. A renewal (the same face again) extends the hold, never past 13.5 s from the face’s start. The variation is seeded, so a scripted stream replays.
  • Neutral lets the face on show go and is over once its onset has blended it out.
  • Blinks: every 2.4 to 6 s (closing 60 ms, shut 30 ms, opening 90 ms), one in seven a double, while a face is held or a line is spoken, or while the scene keeps its avatar alive at rest (Ease::alive, a scene’s idle life). Otherwise at rest the eyes stay open and the view draws nothing. The override table then thins a blink under an emotion.

The decision model’s expression choice reaches the director through face_from_answer (the reply guard, then p >= 0.6). A caller that wants weights uses the raw picks (picks_from_answer).

Picks over decision.v1

Five questions about a reply (instinct.rs; benchmarks/verbs/avatar-instinct.v1.json holds a sample request for the fox’s clips):

id type options
performs noul is the avatar committing to an action in its own current words
what choice the rig’s declared clips, each an action or a gesture, and none
reaction choice agree, disagree, empathize, celebrate, recoil, ponder, bashful, scoff, none
face choice neutral and 21 faces over the five emotion presets (FACES)
energy choice gentle, moderate, high (gentle shows a face at 0.75)

The gate (Gate::gate):

what acts when
any answer it passes decision.v1’s own checks (typed as asked, a choice among its options, probabilities summing to 1, a noul in 0..=1); a missing or refused answer abstains for its question alone, never a no
a face p >= 0.6
a reaction the reply’s words are final, p >= 0.75, and 5 s since the last reaction
a clip p >= 0.6 on final words, 0.8 (gesture) or 0.85 (action) on a partial; an action also needs performs >= 0.5 (final) or 0.6 (partial); one clip a turn on partials
a veto performs <= 0.2 with what at none >= 0.7 vetoes every clip

A choice’s p is its confidence, else its own probability, else 0 (no probability, no act).

Errors

Nothing here is on a wire, so nothing is refused to a peer. A reply the guard refuses moves nothing (PickError). A gated answer under its threshold is counted (GateCounts: faces_under, reactions_under, reactions_cooling, vetoes, abstains) and does not act.

Compatibility

Additive. surface3d::vrm re-exports the picks, the ease, the lips, the expression question and the ExpressionDriver seam; Decidectl is apps/native-kit-edge’s. Command ends with Face, Gaze and then Mouth. A rig that ignores the director and the gaze draws its picks under the override table.

See also

  • avatar-action.v1 — the verbs and names a rig declares and the gateway relays.
  • decision.v1 — the typed questions’ wire and its reply guard.

Was this page helpful?