Add monorepo migration implementation brief

2 months ago · 162dc34e16
1 changed files with 551 additions and 0 deletions
--- a/docs/MONOREPO_MIGRATION_BRIEF.md
+++ b/docs/MONOREPO_MIGRATION_BRIEF.md
@ -0,0 +1,551 @@
 # Monorepo migration implementation brief
 > **Status:** Proposed target architecture, not yet implemented. This document is a handoff for a
 > fresh implementation session. Until that migration is complete, `docs/ARCHITECTURE.md` describes
 > the repository's actual structure.
 ## Objective
 Convert `webavatar-core` from a SvelteKit template whose reusable modules live inside the app into an
 npm-workspaces monorepo with independently versioned, privately distributed packages:
 - `@<scope>/webavatar-core` — reusable shared, browser, server, and Vite integration.
 - `@<scope>/webavatar-humanoid` — optional humanoid state and animation functionality.
 - `apps/template` — the SvelteKit application template, demo surfaces, project-owned configuration,
  and composition roots.
 The packages must be normal npm-format packages, but they must not be published to the public
 npmjs.com registry. Configure them for a private registry (GitHub Packages is the assumed default;
 substitute the actual internal registry if different).
 This migration is about packaging, ownership, and build boundaries. Preserve existing runtime
 behavior and wire protocols unless a change is specifically required to establish a package API.
 ## Why this change
 The current directory convention reduces conflicts between `core`, `project`, and `humanoid`, but
 client projects still have to merge the entire template branch to receive a reusable subsystem
 update. Versioned packages will let a project upgrade core and humanoid independently, receive an
 explicit compatibility error when versions do not match, and avoid merging unrelated routes,
 deployment files, demos, or project content.
 The monorepo retains the important development property that core, humanoid, and the reference app
 can be changed and tested together.
 ## Non-goals
 - Do not publish anything publicly.
 - Do not redesign the realtime protocol, tool execution, analytics, logging behavior, or animation
  algorithms as part of the move.
 - Do not make Three.js, the demo avatar, or any renderer part of the humanoid package.
 - Do not make project persona, project tools, UI, or deployment policy part of core.
 - Do not add a global service locator or global `configureHumanoid()` API.
 - Do not preserve old `$lib/core` or `$lib/humanoid` import paths with permanent compatibility
  aliases. Update consumers to the package API.
 - Do not extract every small concern into its own package during the first migration.
 ## Preconditions for the implementation session
 1. Read `AGENTS.md` and the relevant subsystem documentation completely.
 2. Inspect `git status` before editing. At the time this brief was written, the worktree contained
   unrelated changes in `docs/ANALYTICS.md`, `src/lib/server/core/session.ts`, and
   `src/lib/server/core/session.spec.ts`. Preserve all user changes and do not fold them into the
   migration accidentally.
 3. Record a green baseline with the existing `check`, unit-test, lint, and build commands. If the
   baseline is not green, distinguish pre-existing failures from migration regressions.
 4. Decide the private package scope, replacing every `@<scope>` placeholder below. Do not invent a
   scope if the repository or organization configuration does not reveal it.
 ## Target repository structure
 ```text
 webavatar/
 ├── package.json
 ├── package-lock.json
 ├── tsconfig.base.json
 ├── eslint.config.js
 ├── prettier.config.js
 ├── docs/
 ├── packages/
 │   ├── core/
 │   │   ├── package.json
 │   │   ├── tsconfig.json
 │   │   ├── README.md
 │   │   └── src/
 │   │       ├── index.ts
 │   │       ├── protocol.ts
 │   │       ├── logger.ts
 │   │       ├── tools.ts
 │   │       ├── client/
 │   │       │   └── index.ts
 │   │       ├── server/
 │   │       │   └── index.ts
 │   │       └── vite/
 │   │           └── index.ts
 │   └── humanoid/
 │       ├── package.json
 │       ├── tsconfig.json
 │       ├── README.md
 │       └── src/
 │           ├── index.ts
 │           ├── client/
 │           │   └── index.ts
 │           └── eye/
 │               ├── index.ts
 │               └── client.ts
 └── apps/
    └── template/
        ├── package.json
        ├── vite.config.ts
        ├── server.js
        ├── tsconfig.json
        ├── src/
        │   ├── lib/project/
        │   ├── lib/server/project/
        │   ├── lib/client/project/
        │   └── routes/
        └── static/
 ```
 Exact internal filenames may stay close to the current layout. The important change is that package
 boundaries become real `package.json` and `exports` boundaries, not merely directory conventions.
 ## Ownership and dependency graph
 The allowed graph is:
 ```text
 apps/template ──────────────▶ @<scope>/webavatar-core
      │                                  ▲
      └────────▶ @<scope>/webavatar-humanoid
@<scope>/webavatar-humanoid ───────────────┘
                 contracts only
 ```
 Rules:
 1. Core imports neither humanoid nor app/project code.
 2. Humanoid may import only public, portable core contracts. It must not import core server or
   browser implementations.
 3. The template app may import public exports from both packages.
 4. Cross-package imports use package names. Relative imports never cross a package boundary.
 5. Neither package may use SvelteKit aliases such as `$lib`, `$app`, or `$env`.
 6. Project code remains in `apps/template/src/lib/**/project` and may depend on either package.
 7. Renderer and avatar-model knowledge remains in the app.
 Add lint rules that enforce these directions. Documentation alone is insufficient.
 ## Package 1: `@<scope>/webavatar-core`
 ### Responsibilities
 Move the reusable code currently under these paths into this package:
 - `src/lib/core/**`
 - `src/lib/client/core/**`, except the Svelte debug-panel component described below
 - `src/lib/server/core/**`
 The route endpoint that mints a session token stays in the app because routing is an application
 concern; it imports token functionality from `@<scope>/webavatar-core/server`.
 Move the `SessionPersona` interface out of project code and into a portable core contract. The
 current type-only imports from `server/core` to `project/persona.ts` violate the intended dependency
 direction even though TypeScript erases them at runtime. The project should export a value that
 `satisfies SessionPersona`.
 ### Public entry points
 Create curated index files and export only supported APIs. A starting manifest shape is:
 ```json
 {
 	"name": "@<scope>/webavatar-core",
 	"version": "0.1.0",
 	"type": "module",
 	"files": ["dist"],
 	"exports": {
 		".": {
 			"types": "./dist/index.d.ts",
 			"import": "./dist/index.js"
 		},
 		"./client": {
 			"types": "./dist/client/index.d.ts",
 			"import": "./dist/client/index.js"
 		},
 		"./server": {
 			"types": "./dist/server/index.d.ts",
 			"import": "./dist/server/index.js"
 		},
 		"./vite": {
 			"types": "./dist/vite/index.d.ts",
 			"import": "./dist/vite/index.js"
 		}
 	},
 	"publishConfig": {
 		"registry": "https://npm.pkg.github.com"
 	}
 }
 ```
 The root entry point is portable and contains contracts such as:
 - Protocol events and session types
 - Tool definitions
 - `Logger`, `LogEntry`, `LogLevel`, and `NOOP_LOGGER`
 - `SessionPersona`
 The `./client` entry point contains browser transports, resilience, client logging, tool dispatch,
 and server-log forwarding. The `./server` entry point contains Node-only implementations. The
 `./vite` entry point contains the Vite dev-server integration so importing the root does not load
 Vite or server code.
 Do not export every internal file as a wildcard. The `exports` map is the compatibility boundary
 that future semantic versions will protect.
 ### Server-only safety
 Moving server code into `node_modules` means its path no longer automatically lives under
 SvelteKit's `$lib/server` directory. In the template app, add a local wrapper:
 ```ts
 // apps/template/src/lib/server/webavatar-core.ts
 export * from '@<scope>/webavatar-core/server';
 ```
 SvelteKit routes and server modules should normally import through this wrapper. Add lint rules that
 forbid `@<scope>/webavatar-core/server` from client and universal app files. Root-only entrypoints
 such as `server.js` may import the package server entry point directly.
 ### Debug panel decision
 Keep the logger contract and client log store in core. For the first migration, move
 `DebugPanel.svelte` and its component test into the template app as UI that consumes the core client
 logger. Do not make the foundational core build responsible for packaging Svelte components during
 the same migration. If multiple client projects later need the panel, extract it in a separate
 `@<scope>/webavatar-devtools-svelte` package as a follow-up.
 ## Package 2: `@<scope>/webavatar-humanoid`
 ### Responsibilities
 Move these paths into the package:
 - `src/lib/humanoid/**`
 - `src/lib/client/humanoid/**`
 Keep these in the template app:
 - `src/routes/demo/humanoid/**`
 - `static/models/avatar.glb`
 - Three.js and its type dependency
 - Rig/morph-target binding and all rendering code
 The package must remain renderer-agnostic and must not acquire a Svelte dependency. Its hand-rolled
 stores may continue to satisfy the Svelte store contract structurally.
 ### Core compatibility
 Humanoid currently needs core's normalized `ServerEvent` contract. Declare core as a peer dependency
 so the app owns the single compatible core installation:
 ```json
 {
 	"name": "@<scope>/webavatar-humanoid",
 	"version": "0.1.0",
 	"type": "module",
 	"files": ["dist"],
 	"peerDependencies": {
 		"@<scope>/webavatar-core": "^0.1.0"
 	},
 	"devDependencies": {
 		"@<scope>/webavatar-core": "^0.1.0"
 	},
 	"exports": {
 		".": {
 			"types": "./dist/index.d.ts",
 			"import": "./dist/index.js"
 		},
 		"./client": {
 			"types": "./dist/client/index.d.ts",
 			"import": "./dist/client/index.js"
 		},
 		"./eye": {
 			"types": "./dist/eye/index.d.ts",
 			"import": "./dist/eye/index.js"
 		},
 		"./eye/client": {
 			"types": "./dist/eye/client.d.ts",
 			"import": "./dist/eye/client.js"
 		}
 	},
 	"publishConfig": {
 		"registry": "https://npm.pkg.github.com"
 	}
 }
 ```
 Use the broadest core peer range actually tested. A core release does not require a humanoid release
 when the existing peer range and APIs remain compatible.
 ### Interaction with core services
 Humanoid may depend on core contracts, not ambient runtime implementations. For logging:
 ```ts
 import { NOOP_LOGGER, type Logger } from '@<scope>/webavatar-core';
 export interface EyeGazeControllerOptions {
 	config?: Partial<EyeConfig>;
 	logger?: Logger;
 }
 export function createEyeGazeController(options: EyeGazeControllerOptions = {}) {
 	const logger = options.logger ?? NOOP_LOGGER;
 	// Stateful/effectful wrapper may log under `humanoid.eye`.
 }
 ```
 The app supplies the implementation:
 ```ts
 import { clientLogger } from '@<scope>/webavatar-core/client';
 import { createEyeGazeController } from '@<scope>/webavatar-humanoid/eye/client';
 const gaze = createEyeGazeController({
 	config: projectEyeConfig,
 	logger: clientLogger
 });
 ```
 Apply the same dependency-injection pattern to future clocks, randomness sources, metrics, or other
 runtime facilities. Keep reducers, curves, animation composition, and per-frame math pure; log at
 stateful boundaries rather than on every animation frame.
 If dependency threading becomes noisy later, add an instance-based `createHumanoidRuntime({ logger,
 ...config })` factory. Do not add mutable package-global configuration because it would make SSR,
 tests, multiple avatars, and concurrent configurations interfere with each other.
 ## Template app: `apps/template`
 The template owns:
 - SvelteKit routes and application HTML
 - Project persona and tools
 - `server.js` and `vite.config.ts` composition roots
 - Environment examples and deployment configuration
 - Demo routes and demo assets
 - The Three.js humanoid renderer
 - The debug-panel Svelte component during the first migration
 Replace existing internal imports as follows:
 | Current import family     | Target import                                         |
 | ------------------------- | ----------------------------------------------------- |
 | `$lib/core/**`            | `@<scope>/webavatar-core`                             |
 | `$lib/client/core/**`     | `@<scope>/webavatar-core/client`                      |
 | `$lib/server/core/**`     | `$lib/server/webavatar-core` in app source            |
 | `$lib/humanoid/**`        | `@<scope>/webavatar-humanoid` or `/eye`               |
 | `$lib/client/humanoid/**` | `@<scope>/webavatar-humanoid/client` or `/eye/client` |
 | `$lib/**/project/**`      | Remains app-local project code                        |
 Prefer importing from curated entry points instead of recreating one export per old source file. If
 two old modules expose names that collide, use a deliberate subpath or rename the public API rather
 than leaking the old filesystem layout wholesale.
 The custom production server should import built package output through package names. Once that
 works, remove the app's special `tsconfig.server.json`/`build-server` compilation path if it is no
 longer needed. Confirm that adapter-node deployment still includes the runtime package dependencies.
 ## Workspace and build design
 The root package is never publishable:
 ```json
 {
 	"name": "webavatar-monorepo",
 	"private": true,
 	"workspaces": ["packages/*", "apps/*"]
 }
 ```
 Use ordinary semver ranges between workspaces so npm links a matching local workspace during
 development and published consumers see valid dependency metadata. Keep one root lockfile.
 Both reusable packages should ship compiled ESM JavaScript plus declarations under `dist/`:
 ```json
 {
 	"scripts": {
 		"build": "tsc -p tsconfig.json",
 		"check": "tsc -p tsconfig.json --noEmit",
 		"test": "vitest run"
 	}
 }
 ```
 Use a root `tsconfig.base.json` for common strictness while letting each package select the runtime
 libraries it needs. Do not let browser code gain Node globals merely because server code exists in
 another package entry point.
 Root scripts should build in dependency order:
 1. Core
 2. Humanoid
 3. Template app
 The root `dev` workflow must build packages once, then watch core and humanoid alongside the SvelteKit
 dev server. A small process runner such as `concurrently` is acceptable. Package exports should point
 to `dist`, not to TypeScript source that only Vite understands, because the custom production Node
 server must consume the same artifact that projects install.
 Keep `three` in the template's development dependencies. Keep `ws` and `better-sqlite3` with the core
 server package if they remain required by its exported runtime. Treat Vite as a compatible peer/dev
 dependency for the `./vite` integration rather than allowing an unrelated Vite copy to be pulled
 silently into the app.
 ## Private publication and versioning
 Assuming GitHub Packages, add this committed scope mapping without a token:
 ```ini
@<scope>:registry=https://npm.pkg.github.com
 ```
 Authentication belongs in developer and CI secrets. Never commit a package token.
 Version core and humanoid independently. Start both at `0.1.0` unless existing internal releases
 require another baseline. During `0.x` development, document that minor releases may be breaking;
 after `1.0.0`, use normal semantic-versioning expectations.
 Suggested tags:
 ```text
 core-v0.1.0
 humanoid-v0.1.0
 ```
 For each release:
 1. Update the package version and changelog.
 2. Update dependent peer ranges only if compatibility changed.
 3. Build, test, and inspect `npm pack --dry-run` for that workspace.
 4. Install the produced tarball into a consumer fixture outside the workspace dependency graph.
 5. Publish only the affected workspace to the private registry.
 6. Create the corresponding immutable Git tag and release notes.
 Manual versioning is sufficient for the first migration. Introduce Changesets or another release
 orchestrator only after the package boundaries and first private release work end to end.
 ## Migration sequence
 Implement in small, reviewable stages while keeping behavior stable:
 ### 1. Establish the workspace shell
 - Add the private root workspace manifest and shared TypeScript configuration.
 - Create package/app directories and manifests.
 - Keep root linting, formatting, and documentation coverage.
 ### 2. Extract portable core contracts
 - Move protocol, logger, tools, and persona contracts first.
 - Add the core root index and export map.
 - Update imports and tests.
 - Eliminate every core-to-project import, including type-only imports.
 ### 3. Extract core client, server, and Vite entry points
 - Move implementation files without behavioral rewrites.
 - Split public entry points by runtime.
 - Move the debug-panel Svelte UI into the app.
 - Make the package build, test, and pack independently.
 ### 4. Extract humanoid
 - Move pure and client humanoid code.
 - Replace the protocol import with the core package's public contract.
 - Add core as a peer/dev dependency.
 - Add logger injection only at effectful wrappers; use `NOOP_LOGGER` by default.
 - Keep Three.js, GLB assets, and renderer components in the app.
 ### 5. Move and rewire the template app
 - Move the SvelteKit shell under `apps/template`.
 - Convert app imports to package entry points.
 - Update `server.js`, `vite.config.ts`, API routes, project tools, demos, and tests.
 - Remove obsolete standalone compilation plumbing only after package imports work in production.
 ### 6. Enforce boundaries
 - Add restricted-import lint rules for core, humanoid, server-only, and app/project directions.
 - Search for leftover `$lib/core`, `$lib/humanoid`, cross-package relative imports, and package imports
  from app-owned paths.
 ### 7. Add distribution verification
 - Pack both packages.
 - Inspect tarball contents.
 - Install tarballs into an isolated consumer.
 - Test private-registry publication with prerelease versions before declaring the migration done.
 ### 8. Update documentation
 - Rewrite `AGENTS.md` orientation for the workspace layout.
 - Update `docs/ARCHITECTURE.md` to describe actual package boundaries and link this document as the
  completed migration rationale or replace it with a permanent packaging document.
 - Update subsystem docs and all source links/import examples.
 - Add package-local READMEs documenting supported exports and compatibility.
 - Remove the proposal warning at the top of this document only when the implementation is complete.
 ## Verification requirements
 The migration is complete only when all of the following pass from a clean install:
 1. Root formatting and lint checks.
 2. Core type-check, unit tests, and build.
 3. Humanoid type-check, unit tests, and build.
 4. Template `svelte-check`, unit tests, and production build.
 5. `npm pack --dry-run` for both packages contains only intended runtime files, declarations,
   package metadata, and documentation.
 6. A consumer outside the workspace can install the packed core and humanoid tarballs and import
   every documented entry point.
 7. Browser smoke test of `/demo/humanoid`, including avatar-state and eye-gaze behavior.
 8. Mock-provider smoke test through the template app.
 9. Actual `/demo/realtime` round trip against OpenAI, including a tool call, because the custom
   dev/production wiring and package boundary changed even if the wire protocol did not.
 10. Production `server.js` boots from built package artifacts rather than workspace source files.
 Remember the existing server-core HMR limitation while the old layout exists. After extraction,
 document the new package-watch behavior accurately rather than carrying that warning forward without
 re-testing it.
 ## Acceptance criteria
 - Core and humanoid are independently versioned installable artifacts.
 - The root and template app are private and cannot be published accidentally.
 - No package imports app/project code.
 - Core does not import humanoid.
 - Humanoid reaches core facilities through public contracts and explicit dependency injection.
 - The app is the only composition root that chooses concrete client/server implementations.
 - Server-only imports remain mechanically protected.
 - Pure humanoid animation functions remain pure.
 - The package tarballs do not contain demos, GLB assets, project persona/tools, database files, or
  unrelated repository content.
 - The template behaves the same in development and production as it did before the migration.
 - Documentation describes the final filesystem, imports, release process, and compatibility policy.
 ## Decisions that must survive implementation
 1. **Private packages, not copied source:** downstream projects install versions; they do not merge
   package source from the template branch.
 2. **Independent versions:** core and humanoid can release separately.
 3. **One-way dependency:** humanoid may depend on portable core contracts; core never knows humanoid
   exists.
 4. **Dependency injection:** logger and future runtime services are passed into humanoid wrappers,
   not imported as ambient core singletons.
 5. **App-owned composition:** project persona, tools, routes, rendering, environment policy, and
   deployment wiring stay in the template/project.
 6. **Curated exports:** the public API is smaller and more stable than the internal filesystem.
 7. **Artifact parity:** local development, packed consumers, and the production server use the same
   compiled package outputs.