Browse Source

Centralize humanoid animation tuning knobs into AnimationConfig

idlePassiveTimeoutMs moves out of a one-off deriveAvatarState options
type into $lib/humanoid/config.ts's AnimationConfig, with
DEFAULT_ANIMATION_CONFIG and resolveAnimationConfig(overrides) merging
partial overrides onto it. Documents the convention in HUMANOID.md so
future knobs ported from the Unreal Engine avatar app's animation
systems land in this same interface instead of scattered options.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
main
Jonathan Hinkel 2 months ago
parent
commit
14f9e53e0c
  1. 18
      docs/HUMANOID.md
  2. 27
      src/lib/client/humanoid/avatar-state.ts
  3. 20
      src/lib/humanoid/config.spec.ts
  4. 34
      src/lib/humanoid/config.ts

18
docs/HUMANOID.md

@ -16,9 +16,23 @@ type AvatarState = 'idleActive' | 'idlePassive' | 'listening' | 'thinking' | 'ta
Note `listening_stopped` moves to `thinking`, not back to idle — ending listening almost always means a turn just completed and a response is expected next, so the avatar shouldn't visibly relax for the split second before the model's response starts arriving. `talking_stopped` and `reset` both land on `idleActive`, never straight to `idlePassive` — passive idle is only ever reached via `idle_timeout`. Note `listening_stopped` moves to `thinking`, not back to idle — ending listening almost always means a turn just completed and a response is expected next, so the avatar shouldn't visibly relax for the split second before the model's response starts arriving. `talking_stopped` and `reset` both land on `idleActive`, never straight to `idlePassive` — passive idle is only ever reached via `idle_timeout`.
## `$lib/humanoid/config.ts` — shared tuning knobs
```ts
interface AnimationConfig {
idlePassiveTimeoutMs: number; // default 30_000, see DEFAULT_ANIMATION_CONFIG
}
```
Every value this package's own state derivation exposes for tuning lives in this one `AnimationConfig` interface, not as a bespoke option on whichever function first needed it — `idlePassiveTimeoutMs` (consumed by `deriveAvatarState`, below) is the first entry, and it's where the next one goes too as more of the Unreal Engine avatar app's animation systems get ported over (further idle-escalation thresholds, turn-taking heuristics, etc.).
`resolveAnimationConfig(overrides?)` merges a project's partial overrides onto `DEFAULT_ANIMATION_CONFIG` — a project only names the knobs it wants to change. Every `$lib/humanoid`/`$lib/client/humanoid` function that reads a tunable value takes `Partial<AnimationConfig>` (default `{}`) and resolves it internally, so passing the same override object to several such functions keeps them in sync without each needing the whole config spelled out.
This interface is scoped to `humanoid`'s own contract layer. A future animation library built as a *consumer* of this package (eye look-at, blink, lip-sync, body/gesture) is a separate package and gets its own config surface — follow this same three-piece shape there (`XConfig` interface, `DEFAULT_X_CONFIG`, `resolveXConfig`) rather than growing this interface across a package boundary. This mirrors the base project's existing convention of config living beside the subsystem it configures rather than in one shared blob (see [CONFIGURATION.md](CONFIGURATION.md)'s note on `AnalyticsConfig`/`SessionDefaults`).
## `$lib/client/humanoid/avatar-state.ts` — deriving it live ## `$lib/client/humanoid/avatar-state.ts` — deriving it live
`deriveAvatarState(source, options?)` takes a `RealtimeClient` or `ResilientRealtimeClient` (accepted structurally, via an `AvatarStateSource` interface — this package doesn't care which transport variant is in use) and folds its `ServerEvent`s through `nextAvatarState`: `deriveAvatarState(source, animationConfig?)` takes a `RealtimeClient` or `ResilientRealtimeClient` (accepted structurally, via an `AvatarStateSource` interface — this package doesn't care which transport variant is in use) and folds its `ServerEvent`s through `nextAvatarState`:
| `ServerEvent` | Signal | | `ServerEvent` | Signal |
| ------------------------------------- | ----------------- | | ------------------------------------- | ----------------- |
@ -33,7 +47,7 @@ Two signals can't come from the event stream alone, so the returned store expose
- **`setListening(active)`** — call from whatever mic-capture code a project uses. That pipeline doesn't exist in `core` yet (see "Known limitations" below), and the client is the only thing that currently knows when the user starts/stops talking: the protocol doesn't surface OpenAI's `input_audio_buffer.speech_*` VAD events — see the "not part of our normalized vocabulary" comment in `server/core/providers/openai.ts`'s `fromOpenAiEvent`. - **`setListening(active)`** — call from whatever mic-capture code a project uses. That pipeline doesn't exist in `core` yet (see "Known limitations" below), and the client is the only thing that currently knows when the user starts/stops talking: the protocol doesn't surface OpenAI's `input_audio_buffer.speech_*` VAD events — see the "not part of our normalized vocabulary" comment in `server/core/providers/openai.ts`'s `fromOpenAiEvent`.
- **`notifyResponseRequested()`** — call right when sending a typed `input_text.send` (the voice path already gets this for free from `setListening(false)`). - **`notifyResponseRequested()`** — call right when sending a typed `input_text.send` (the voice path already gets this for free from `setListening(false)`).
This wrapper also owns the `idleActive` → `idlePassive` clock: the store starts (and every `reset`/`talking_stopped` returns to) `idleActive`, and a timer — `options.idlePassiveTimeoutMs`, default `DEFAULT_IDLE_PASSIVE_TIMEOUT_MS` (30s) — fires `idle_timeout` if nothing else happens first. Any other signal re-arms it on the next `idleActive` entry; `destroy()` cancels it. This wrapper also owns the `idleActive` → `idlePassive` clock: the store starts (and every `reset`/`talking_stopped` returns to) `idleActive`, and a timer — `animationConfig.idlePassiveTimeoutMs`, see `config.ts` above — fires `idle_timeout` if nothing else happens first. Any other signal re-arms it on the next `idleActive` entry; `destroy()` cancels it.
The returned `AvatarStateStore.subscribe()` follows the Svelte store contract (calls `run` immediately with the current value, then on every change), so it's usable directly as `$store` in a `.svelte` file without a hard Svelte dependency in this package itself. See `/demo/humanoid` (`src/routes/demo/humanoid/+page.svelte`) for the reference wiring, including the manual "listening" toggle standing in for real mic-capture UI. The returned `AvatarStateStore.subscribe()` follows the Svelte store contract (calls `run` immediately with the current value, then on every change), so it's usable directly as `$store` in a `.svelte` file without a hard Svelte dependency in this package itself. See `/demo/humanoid` (`src/routes/demo/humanoid/+page.svelte`) for the reference wiring, including the manual "listening" toggle standing in for real mic-capture UI.

27
src/lib/client/humanoid/avatar-state.ts

@ -1,4 +1,5 @@
import type { ServerEvent } from '../../core/protocol.js'; import type { ServerEvent } from '../../core/protocol.js';
import { resolveAnimationConfig, type AnimationConfig } from '../../humanoid/config.js';
import { nextAvatarState, type AvatarSignal, type AvatarState } from '../../humanoid/state.js'; import { nextAvatarState, type AvatarSignal, type AvatarState } from '../../humanoid/state.js';
/** /**
@ -33,20 +34,6 @@ export interface AvatarStateStore {
destroy(): void; destroy(): void;
} }
/** Default `idlePassiveTimeoutMs` — see `DeriveAvatarStateOptions`. */
export const DEFAULT_IDLE_PASSIVE_TIMEOUT_MS = 30_000;
export interface DeriveAvatarStateOptions {
/**
* How long the avatar sits in `idleActive` with no other signal before
* this wrapper fires the `idle_timeout` signal that settles it into
* `idlePassive`. The pure reducer in `$lib/humanoid/state.ts` has no
* timers of its own, so this wrapper is what actually owns the clock —
* tune to whatever cadence a project's idle animation set expects.
*/
idlePassiveTimeoutMs?: number;
}
/** /**
* Derives a reactive `AvatarState` from a live `RealtimeClient` or * Derives a reactive `AvatarState` from a live `RealtimeClient` or
* `ResilientRealtimeClient`. Translates `ServerEvent`s (plus the imperative * `ResilientRealtimeClient`. Translates `ServerEvent`s (plus the imperative
@ -61,15 +48,17 @@ export interface DeriveAvatarStateOptions {
* *
* Also owns the `idleActive` -> `idlePassive` clock: the store starts (and * Also owns the `idleActive` -> `idlePassive` clock: the store starts (and
* every `reset`/`talking_stopped` returns to) `idleActive`, and this * every `reset`/`talking_stopped` returns to) `idleActive`, and this
* function arms a timer — `options.idlePassiveTimeoutMs`, default * function arms a timer — `animationConfig.idlePassiveTimeoutMs`, see
* `DEFAULT_IDLE_PASSIVE_TIMEOUT_MS` — that fires `idle_timeout` if nothing * `$lib/humanoid/config.ts` — that fires `idle_timeout` if nothing else
* else happens first. * happens first. `animationConfig` only needs to name the knobs a project
* wants to override; anything omitted falls back to
* `DEFAULT_ANIMATION_CONFIG`.
*/ */
export function deriveAvatarState( export function deriveAvatarState(
source: AvatarStateSource, source: AvatarStateSource,
options: DeriveAvatarStateOptions = {} animationConfig: Partial<AnimationConfig> = {}
): AvatarStateStore { ): AvatarStateStore {
const idlePassiveTimeoutMs = options.idlePassiveTimeoutMs ?? DEFAULT_IDLE_PASSIVE_TIMEOUT_MS; const { idlePassiveTimeoutMs } = resolveAnimationConfig(animationConfig);
let state: AvatarState = 'idleActive'; let state: AvatarState = 'idleActive';
const subscribers = new Set<(state: AvatarState) => void>(); const subscribers = new Set<(state: AvatarState) => void>();
let idleTimer: ReturnType<typeof setTimeout> | null = null; let idleTimer: ReturnType<typeof setTimeout> | null = null;

20
src/lib/humanoid/config.spec.ts

@ -0,0 +1,20 @@
import { describe, expect, it } from 'vitest';
import { DEFAULT_ANIMATION_CONFIG, resolveAnimationConfig } from './config.js';
describe('resolveAnimationConfig', () => {
it('returns the defaults when called with no overrides', () => {
expect(resolveAnimationConfig()).toEqual(DEFAULT_ANIMATION_CONFIG);
});
it('merges a partial override onto the defaults', () => {
expect(resolveAnimationConfig({ idlePassiveTimeoutMs: 5_000 })).toEqual({
...DEFAULT_ANIMATION_CONFIG,
idlePassiveTimeoutMs: 5_000
});
});
it('does not mutate DEFAULT_ANIMATION_CONFIG', () => {
resolveAnimationConfig({ idlePassiveTimeoutMs: 5_000 });
expect(DEFAULT_ANIMATION_CONFIG.idlePassiveTimeoutMs).toBe(30_000);
});
});

34
src/lib/humanoid/config.ts

@ -0,0 +1,34 @@
/**
* Single home for every tunable value the `humanoid` package's own state
* derivation exposes — currently just the idleActive->idlePassive timeout,
* but the intent is that any future knob this package's own reducers/
* derivation logic need (further idle-escalation thresholds, turn-taking
* heuristics, etc. — the kind of thing an Unreal Engine avatar app's
* animation systems tend to expose as constants) lands here too, instead of
* as a one-off option on whichever function first needed it. See
* docs/HUMANOID.md's "Configuration" section.
*
* A future animation library built as a *consumer* of this package (eye
* look-at, blink, lip-sync, body/gesture — see HUMANOID.md's "What's not
* built here") is a separate package with its own config surface, not an
* extension of this interface — follow the same three-piece shape
* (`XConfig` interface, `DEFAULT_X_CONFIG`, `resolveXConfig`) there rather
* than growing this one across package boundaries.
*/
export interface AnimationConfig {
/**
* How long the avatar sits in `idleActive` with no other signal before
* `deriveAvatarState` (`$lib/client/humanoid/avatar-state.ts`) fires the
* `idle_timeout` signal that settles it into `idlePassive`.
*/
idlePassiveTimeoutMs: number;
}
export const DEFAULT_ANIMATION_CONFIG: AnimationConfig = {
idlePassiveTimeoutMs: 30_000
};
/** Merges a project's partial overrides onto `DEFAULT_ANIMATION_CONFIG`. */
export function resolveAnimationConfig(overrides: Partial<AnimationConfig> = {}): AnimationConfig {
return { ...DEFAULT_ANIMATION_CONFIG, ...overrides };
}
Loading…
Cancel
Save