diff --git a/docs/MONOREPO_MIGRATION_BRIEF.md b/docs/MONOREPO_MIGRATION_BRIEF.md new file mode 100644 index 0000000..bb2ebd9 --- /dev/null +++ b/docs/MONOREPO_MIGRATION_BRIEF.md @@ -0,0 +1,551 @@ +# Monorepo migration implementation brief + +> **Status:** Proposed target architecture, not yet implemented. This document is a handoff for a +> fresh implementation session. Until that migration is complete, `docs/ARCHITECTURE.md` describes +> the repository's actual structure. + +## Objective + +Convert `webavatar-core` from a SvelteKit template whose reusable modules live inside the app into an +npm-workspaces monorepo with independently versioned, privately distributed packages: + +- `@/webavatar-core` — reusable shared, browser, server, and Vite integration. +- `@/webavatar-humanoid` — optional humanoid state and animation functionality. +- `apps/template` — the SvelteKit application template, demo surfaces, project-owned configuration, + and composition roots. + +The packages must be normal npm-format packages, but they must not be published to the public +npmjs.com registry. Configure them for a private registry (GitHub Packages is the assumed default; +substitute the actual internal registry if different). + +This migration is about packaging, ownership, and build boundaries. Preserve existing runtime +behavior and wire protocols unless a change is specifically required to establish a package API. + +## Why this change + +The current directory convention reduces conflicts between `core`, `project`, and `humanoid`, but +client projects still have to merge the entire template branch to receive a reusable subsystem +update. Versioned packages will let a project upgrade core and humanoid independently, receive an +explicit compatibility error when versions do not match, and avoid merging unrelated routes, +deployment files, demos, or project content. + +The monorepo retains the important development property that core, humanoid, and the reference app +can be changed and tested together. + +## Non-goals + +- Do not publish anything publicly. +- Do not redesign the realtime protocol, tool execution, analytics, logging behavior, or animation + algorithms as part of the move. +- Do not make Three.js, the demo avatar, or any renderer part of the humanoid package. +- Do not make project persona, project tools, UI, or deployment policy part of core. +- Do not add a global service locator or global `configureHumanoid()` API. +- Do not preserve old `$lib/core` or `$lib/humanoid` import paths with permanent compatibility + aliases. Update consumers to the package API. +- Do not extract every small concern into its own package during the first migration. + +## Preconditions for the implementation session + +1. Read `AGENTS.md` and the relevant subsystem documentation completely. +2. Inspect `git status` before editing. At the time this brief was written, the worktree contained + unrelated changes in `docs/ANALYTICS.md`, `src/lib/server/core/session.ts`, and + `src/lib/server/core/session.spec.ts`. Preserve all user changes and do not fold them into the + migration accidentally. +3. Record a green baseline with the existing `check`, unit-test, lint, and build commands. If the + baseline is not green, distinguish pre-existing failures from migration regressions. +4. Decide the private package scope, replacing every `@` placeholder below. Do not invent a + scope if the repository or organization configuration does not reveal it. + +## Target repository structure + +```text +webavatar/ +├── package.json +├── package-lock.json +├── tsconfig.base.json +├── eslint.config.js +├── prettier.config.js +├── docs/ +├── packages/ +│ ├── core/ +│ │ ├── package.json +│ │ ├── tsconfig.json +│ │ ├── README.md +│ │ └── src/ +│ │ ├── index.ts +│ │ ├── protocol.ts +│ │ ├── logger.ts +│ │ ├── tools.ts +│ │ ├── client/ +│ │ │ └── index.ts +│ │ ├── server/ +│ │ │ └── index.ts +│ │ └── vite/ +│ │ └── index.ts +│ └── humanoid/ +│ ├── package.json +│ ├── tsconfig.json +│ ├── README.md +│ └── src/ +│ ├── index.ts +│ ├── client/ +│ │ └── index.ts +│ └── eye/ +│ ├── index.ts +│ └── client.ts +└── apps/ + └── template/ + ├── package.json + ├── vite.config.ts + ├── server.js + ├── tsconfig.json + ├── src/ + │ ├── lib/project/ + │ ├── lib/server/project/ + │ ├── lib/client/project/ + │ └── routes/ + └── static/ +``` + +Exact internal filenames may stay close to the current layout. The important change is that package +boundaries become real `package.json` and `exports` boundaries, not merely directory conventions. + +## Ownership and dependency graph + +The allowed graph is: + +```text +apps/template ──────────────▶ @/webavatar-core + │ ▲ + └────────▶ @/webavatar-humanoid + +@/webavatar-humanoid ───────────────┘ + contracts only +``` + +Rules: + +1. Core imports neither humanoid nor app/project code. +2. Humanoid may import only public, portable core contracts. It must not import core server or + browser implementations. +3. The template app may import public exports from both packages. +4. Cross-package imports use package names. Relative imports never cross a package boundary. +5. Neither package may use SvelteKit aliases such as `$lib`, `$app`, or `$env`. +6. Project code remains in `apps/template/src/lib/**/project` and may depend on either package. +7. Renderer and avatar-model knowledge remains in the app. + +Add lint rules that enforce these directions. Documentation alone is insufficient. + +## Package 1: `@/webavatar-core` + +### Responsibilities + +Move the reusable code currently under these paths into this package: + +- `src/lib/core/**` +- `src/lib/client/core/**`, except the Svelte debug-panel component described below +- `src/lib/server/core/**` + +The route endpoint that mints a session token stays in the app because routing is an application +concern; it imports token functionality from `@/webavatar-core/server`. + +Move the `SessionPersona` interface out of project code and into a portable core contract. The +current type-only imports from `server/core` to `project/persona.ts` violate the intended dependency +direction even though TypeScript erases them at runtime. The project should export a value that +`satisfies SessionPersona`. + +### Public entry points + +Create curated index files and export only supported APIs. A starting manifest shape is: + +```json +{ + "name": "@/webavatar-core", + "version": "0.1.0", + "type": "module", + "files": ["dist"], + "exports": { + ".": { + "types": "./dist/index.d.ts", + "import": "./dist/index.js" + }, + "./client": { + "types": "./dist/client/index.d.ts", + "import": "./dist/client/index.js" + }, + "./server": { + "types": "./dist/server/index.d.ts", + "import": "./dist/server/index.js" + }, + "./vite": { + "types": "./dist/vite/index.d.ts", + "import": "./dist/vite/index.js" + } + }, + "publishConfig": { + "registry": "https://npm.pkg.github.com" + } +} +``` + +The root entry point is portable and contains contracts such as: + +- Protocol events and session types +- Tool definitions +- `Logger`, `LogEntry`, `LogLevel`, and `NOOP_LOGGER` +- `SessionPersona` + +The `./client` entry point contains browser transports, resilience, client logging, tool dispatch, +and server-log forwarding. The `./server` entry point contains Node-only implementations. The +`./vite` entry point contains the Vite dev-server integration so importing the root does not load +Vite or server code. + +Do not export every internal file as a wildcard. The `exports` map is the compatibility boundary +that future semantic versions will protect. + +### Server-only safety + +Moving server code into `node_modules` means its path no longer automatically lives under +SvelteKit's `$lib/server` directory. In the template app, add a local wrapper: + +```ts +// apps/template/src/lib/server/webavatar-core.ts +export * from '@/webavatar-core/server'; +``` + +SvelteKit routes and server modules should normally import through this wrapper. Add lint rules that +forbid `@/webavatar-core/server` from client and universal app files. Root-only entrypoints +such as `server.js` may import the package server entry point directly. + +### Debug panel decision + +Keep the logger contract and client log store in core. For the first migration, move +`DebugPanel.svelte` and its component test into the template app as UI that consumes the core client +logger. Do not make the foundational core build responsible for packaging Svelte components during +the same migration. If multiple client projects later need the panel, extract it in a separate +`@/webavatar-devtools-svelte` package as a follow-up. + +## Package 2: `@/webavatar-humanoid` + +### Responsibilities + +Move these paths into the package: + +- `src/lib/humanoid/**` +- `src/lib/client/humanoid/**` + +Keep these in the template app: + +- `src/routes/demo/humanoid/**` +- `static/models/avatar.glb` +- Three.js and its type dependency +- Rig/morph-target binding and all rendering code + +The package must remain renderer-agnostic and must not acquire a Svelte dependency. Its hand-rolled +stores may continue to satisfy the Svelte store contract structurally. + +### Core compatibility + +Humanoid currently needs core's normalized `ServerEvent` contract. Declare core as a peer dependency +so the app owns the single compatible core installation: + +```json +{ + "name": "@/webavatar-humanoid", + "version": "0.1.0", + "type": "module", + "files": ["dist"], + "peerDependencies": { + "@/webavatar-core": "^0.1.0" + }, + "devDependencies": { + "@/webavatar-core": "^0.1.0" + }, + "exports": { + ".": { + "types": "./dist/index.d.ts", + "import": "./dist/index.js" + }, + "./client": { + "types": "./dist/client/index.d.ts", + "import": "./dist/client/index.js" + }, + "./eye": { + "types": "./dist/eye/index.d.ts", + "import": "./dist/eye/index.js" + }, + "./eye/client": { + "types": "./dist/eye/client.d.ts", + "import": "./dist/eye/client.js" + } + }, + "publishConfig": { + "registry": "https://npm.pkg.github.com" + } +} +``` + +Use the broadest core peer range actually tested. A core release does not require a humanoid release +when the existing peer range and APIs remain compatible. + +### Interaction with core services + +Humanoid may depend on core contracts, not ambient runtime implementations. For logging: + +```ts +import { NOOP_LOGGER, type Logger } from '@/webavatar-core'; + +export interface EyeGazeControllerOptions { + config?: Partial; + logger?: Logger; +} + +export function createEyeGazeController(options: EyeGazeControllerOptions = {}) { + const logger = options.logger ?? NOOP_LOGGER; + // Stateful/effectful wrapper may log under `humanoid.eye`. +} +``` + +The app supplies the implementation: + +```ts +import { clientLogger } from '@/webavatar-core/client'; +import { createEyeGazeController } from '@/webavatar-humanoid/eye/client'; + +const gaze = createEyeGazeController({ + config: projectEyeConfig, + logger: clientLogger +}); +``` + +Apply the same dependency-injection pattern to future clocks, randomness sources, metrics, or other +runtime facilities. Keep reducers, curves, animation composition, and per-frame math pure; log at +stateful boundaries rather than on every animation frame. + +If dependency threading becomes noisy later, add an instance-based `createHumanoidRuntime({ logger, +...config })` factory. Do not add mutable package-global configuration because it would make SSR, +tests, multiple avatars, and concurrent configurations interfere with each other. + +## Template app: `apps/template` + +The template owns: + +- SvelteKit routes and application HTML +- Project persona and tools +- `server.js` and `vite.config.ts` composition roots +- Environment examples and deployment configuration +- Demo routes and demo assets +- The Three.js humanoid renderer +- The debug-panel Svelte component during the first migration + +Replace existing internal imports as follows: + +| Current import family | Target import | +| ------------------------- | ----------------------------------------------------- | +| `$lib/core/**` | `@/webavatar-core` | +| `$lib/client/core/**` | `@/webavatar-core/client` | +| `$lib/server/core/**` | `$lib/server/webavatar-core` in app source | +| `$lib/humanoid/**` | `@/webavatar-humanoid` or `/eye` | +| `$lib/client/humanoid/**` | `@/webavatar-humanoid/client` or `/eye/client` | +| `$lib/**/project/**` | Remains app-local project code | + +Prefer importing from curated entry points instead of recreating one export per old source file. If +two old modules expose names that collide, use a deliberate subpath or rename the public API rather +than leaking the old filesystem layout wholesale. + +The custom production server should import built package output through package names. Once that +works, remove the app's special `tsconfig.server.json`/`build-server` compilation path if it is no +longer needed. Confirm that adapter-node deployment still includes the runtime package dependencies. + +## Workspace and build design + +The root package is never publishable: + +```json +{ + "name": "webavatar-monorepo", + "private": true, + "workspaces": ["packages/*", "apps/*"] +} +``` + +Use ordinary semver ranges between workspaces so npm links a matching local workspace during +development and published consumers see valid dependency metadata. Keep one root lockfile. + +Both reusable packages should ship compiled ESM JavaScript plus declarations under `dist/`: + +```json +{ + "scripts": { + "build": "tsc -p tsconfig.json", + "check": "tsc -p tsconfig.json --noEmit", + "test": "vitest run" + } +} +``` + +Use a root `tsconfig.base.json` for common strictness while letting each package select the runtime +libraries it needs. Do not let browser code gain Node globals merely because server code exists in +another package entry point. + +Root scripts should build in dependency order: + +1. Core +2. Humanoid +3. Template app + +The root `dev` workflow must build packages once, then watch core and humanoid alongside the SvelteKit +dev server. A small process runner such as `concurrently` is acceptable. Package exports should point +to `dist`, not to TypeScript source that only Vite understands, because the custom production Node +server must consume the same artifact that projects install. + +Keep `three` in the template's development dependencies. Keep `ws` and `better-sqlite3` with the core +server package if they remain required by its exported runtime. Treat Vite as a compatible peer/dev +dependency for the `./vite` integration rather than allowing an unrelated Vite copy to be pulled +silently into the app. + +## Private publication and versioning + +Assuming GitHub Packages, add this committed scope mapping without a token: + +```ini +@:registry=https://npm.pkg.github.com +``` + +Authentication belongs in developer and CI secrets. Never commit a package token. + +Version core and humanoid independently. Start both at `0.1.0` unless existing internal releases +require another baseline. During `0.x` development, document that minor releases may be breaking; +after `1.0.0`, use normal semantic-versioning expectations. + +Suggested tags: + +```text +core-v0.1.0 +humanoid-v0.1.0 +``` + +For each release: + +1. Update the package version and changelog. +2. Update dependent peer ranges only if compatibility changed. +3. Build, test, and inspect `npm pack --dry-run` for that workspace. +4. Install the produced tarball into a consumer fixture outside the workspace dependency graph. +5. Publish only the affected workspace to the private registry. +6. Create the corresponding immutable Git tag and release notes. + +Manual versioning is sufficient for the first migration. Introduce Changesets or another release +orchestrator only after the package boundaries and first private release work end to end. + +## Migration sequence + +Implement in small, reviewable stages while keeping behavior stable: + +### 1. Establish the workspace shell + +- Add the private root workspace manifest and shared TypeScript configuration. +- Create package/app directories and manifests. +- Keep root linting, formatting, and documentation coverage. + +### 2. Extract portable core contracts + +- Move protocol, logger, tools, and persona contracts first. +- Add the core root index and export map. +- Update imports and tests. +- Eliminate every core-to-project import, including type-only imports. + +### 3. Extract core client, server, and Vite entry points + +- Move implementation files without behavioral rewrites. +- Split public entry points by runtime. +- Move the debug-panel Svelte UI into the app. +- Make the package build, test, and pack independently. + +### 4. Extract humanoid + +- Move pure and client humanoid code. +- Replace the protocol import with the core package's public contract. +- Add core as a peer/dev dependency. +- Add logger injection only at effectful wrappers; use `NOOP_LOGGER` by default. +- Keep Three.js, GLB assets, and renderer components in the app. + +### 5. Move and rewire the template app + +- Move the SvelteKit shell under `apps/template`. +- Convert app imports to package entry points. +- Update `server.js`, `vite.config.ts`, API routes, project tools, demos, and tests. +- Remove obsolete standalone compilation plumbing only after package imports work in production. + +### 6. Enforce boundaries + +- Add restricted-import lint rules for core, humanoid, server-only, and app/project directions. +- Search for leftover `$lib/core`, `$lib/humanoid`, cross-package relative imports, and package imports + from app-owned paths. + +### 7. Add distribution verification + +- Pack both packages. +- Inspect tarball contents. +- Install tarballs into an isolated consumer. +- Test private-registry publication with prerelease versions before declaring the migration done. + +### 8. Update documentation + +- Rewrite `AGENTS.md` orientation for the workspace layout. +- Update `docs/ARCHITECTURE.md` to describe actual package boundaries and link this document as the + completed migration rationale or replace it with a permanent packaging document. +- Update subsystem docs and all source links/import examples. +- Add package-local READMEs documenting supported exports and compatibility. +- Remove the proposal warning at the top of this document only when the implementation is complete. + +## Verification requirements + +The migration is complete only when all of the following pass from a clean install: + +1. Root formatting and lint checks. +2. Core type-check, unit tests, and build. +3. Humanoid type-check, unit tests, and build. +4. Template `svelte-check`, unit tests, and production build. +5. `npm pack --dry-run` for both packages contains only intended runtime files, declarations, + package metadata, and documentation. +6. A consumer outside the workspace can install the packed core and humanoid tarballs and import + every documented entry point. +7. Browser smoke test of `/demo/humanoid`, including avatar-state and eye-gaze behavior. +8. Mock-provider smoke test through the template app. +9. Actual `/demo/realtime` round trip against OpenAI, including a tool call, because the custom + dev/production wiring and package boundary changed even if the wire protocol did not. +10. Production `server.js` boots from built package artifacts rather than workspace source files. + +Remember the existing server-core HMR limitation while the old layout exists. After extraction, +document the new package-watch behavior accurately rather than carrying that warning forward without +re-testing it. + +## Acceptance criteria + +- Core and humanoid are independently versioned installable artifacts. +- The root and template app are private and cannot be published accidentally. +- No package imports app/project code. +- Core does not import humanoid. +- Humanoid reaches core facilities through public contracts and explicit dependency injection. +- The app is the only composition root that chooses concrete client/server implementations. +- Server-only imports remain mechanically protected. +- Pure humanoid animation functions remain pure. +- The package tarballs do not contain demos, GLB assets, project persona/tools, database files, or + unrelated repository content. +- The template behaves the same in development and production as it did before the migration. +- Documentation describes the final filesystem, imports, release process, and compatibility policy. + +## Decisions that must survive implementation + +1. **Private packages, not copied source:** downstream projects install versions; they do not merge + package source from the template branch. +2. **Independent versions:** core and humanoid can release separately. +3. **One-way dependency:** humanoid may depend on portable core contracts; core never knows humanoid + exists. +4. **Dependency injection:** logger and future runtime services are passed into humanoid wrappers, + not imported as ambient core singletons. +5. **App-owned composition:** project persona, tools, routes, rendering, environment policy, and + deployment wiring stay in the template/project. +6. **Curated exports:** the public API is smaller and more stable than the internal filesystem. +7. **Artifact parity:** local development, packed consumers, and the production server use the same + compiled package outputs.