leafyy
aa69ee62b5
## Description
The VRM and MMD renderers had the same two boundary problems:
- Both imported the Stage audio store from the `stage-ui` source tree.
- Both kept copies of the wLipSync profile and model-neutral vowel
policy.
This PR solves both problems as one architecture change.
```text
Stage owns AudioContext and the current audio source
-> ThreeScene / MMDScene receive both as inputs
-> model-driver-lipsync owns the profile and shared vowel policy
-> VRM maps AEIOU to aa / ee / ih / oh / ou
-> MMD maps AEIOU to vowelA / vowelE / vowelI / vowelO / vowelU
```
The shared driver owns viseme projection, winner and runner selection,
silence handling, and smoothing. It has no Vue, Three.js, VRM, or MMD
dependency. Each renderer keeps only its model-specific mapping.
The renderer packages no longer import the Stage audio store or depend
directly on `wlipsync`. Stage passes the owning `AudioContext` down with
the current audio source. Cleanup disconnects only the wLipSync node
owned by that renderer.
## Current solution
The package-boundary review identified two related findings:
- Finding 3: The VRM and MMD renderers imported `stage-ui` audio state
through source paths.
- Finding 4: The same renderers duplicated the wLipSync profile and
model-neutral vowel policy.
Both findings came from one missing boundary between Stage, the lip-sync
driver, and the renderer adapters. This PR fixes that boundary with one
ownership change.
| Layer | Responsibility |
| --- | --- |
| `stage-ui` | Owns the `AudioContext` and the current audio source. It
passes both values to each renderer. |
| `stage-ui-three` and `stage-ui-mmd` | Connect renderer components to
the shared driver. They do not import the Stage audio store. |
| `model-driver-lipsync/runtime/wlipsync` | Exposes `createWLipSyncNode`
as the public runtime entry. |
| `model-driver-lipsync/shared/wlipsync` | Owns the profile, public
types, and model-neutral vowel policy. |
Runtime flow:
1. `Stage.vue` passes `audioContext` and `currentAudioSource` to
`ThreeScene` or `MMDScene`.
2. The renderer composable watches the context and creates one wLipSync
node for that context.
3. The source watcher connects the current audio source to that node.
4. Each render frame sends the wLipSync values to
`createWLipSyncVowelDriver`.
5. The renderer adapter applies the returned AEIOU weights to its model
controls.
6. Cleanup disconnects the exact source and node pair that the renderer
owns.
The shared policy keeps the current renderer behavior:
- It maps the raw `S` viseme to `I`.
- It selects the two strongest vowels.
- It uses the existing volume and silence thresholds.
- It keeps the 160 ms silence window.
- It keeps the existing attack, release, cap, and output scale values.
The renderer-specific mappings stay in their renderer packages:
- VRM maps AEIOU to `aa`, `ee`, `ih`, `oh`, and `ou`.
- MMD maps AEIOU to `vowelA`, `vowelE`, `vowelI`, `vowelO`, and
`vowelU`.
This solution removes both duplicate profile files. It also removes the
direct `wlipsync` dependencies from both renderer packages.
## Linked Issues
None.
## Additional Context
There is no intended visual change. This refactor preserves the existing
vowel weights and renderer mappings.
Verification:
- `sem diff --staged --no-cosmetics -v --file-exts .ts .tsx`
- `sem diff --staged --no-cosmetics -v --file-exts .vue`
- `sem impact --entity-id <useVRMLipSync> --json --dependents`
- `sem impact --entity-id <useMMDLipSync> --json --dependents`
- `pnpm -F @proj-airi/model-driver-lipsync test` — 3 tests passed
- `pnpm -F @proj-airi/stage-ui-three exec vitest run` — 18 tests passed
- `pnpm -F @proj-airi/stage-ui-mmd exec vitest run` — 23 tests passed
- Type checks passed for `model-driver-lipsync`, `stage-ui-three`,
`stage-ui-mmd`, and `stage-ui`
- `pnpm install --frozen-lockfile --ignore-scripts`
- `pnpm lint` — 0 errors and 7 existing warnings outside this change
- `git diff --check`
The full workspace `pnpm typecheck` was also run. In the isolated
worktree it stopped in unchanged `electron-vueuse` and
`plugin-sdk-tamagotchi` because their generated dependency outputs were
not present. All affected package type checks above passed.
2.9 KiB
2.9 KiB
@proj-airi/stage-ui-mmd
MMD (MikuMikuDance) scene components, composables, and stores for Project AIRI.
What it does
Renders PMX/PMD models with VMD/VPD motion support inside AIRI's stage, reaching feature parity with the Live2D and VRM renderers:
- Model loading for
.pmx/.pmd, from a packaged ZIP (model + textures) or a bare model URL. - Physics for hair, skirts, and custom rigs via Bullet (Ammo.js) — rigid bodies and 6-DOF spring joints, toggleable at runtime.
- IK (CCD solver) and append-bone ("grant") propagation for standard and custom rigs.
- Importable, mappable motions: import VMD clips (persisted to IndexedDB, synced across windows), set one as the looping idle, preview any as a one-shot, remove them, and map AIRI emotions to gesture motions.
- Morph-based lip-sync driven by the shared wLipSync profile, mapping phonemes to the canonical あいうえお mouth morphs, with per-slot morph remapping for non-standard models.
- Emotion expressions via morph cross-fades.
- Looking-at modes (camera / cursor / disabled) with idle saccades, plus procedural blink.
- Scene settings: model transform, camera FOV, ambient + directional lights, albedo glow, render scale, physics gravity, and per-material opacity — all live, persisted, and synced across windows.
It builds on @moeru/three-mmd for
PMX/PMD loading, VMD animation building, IK, append-bone propagation, toon
materials, outlines, and the shared runtime update order. Physics comes from
@moeru/three-mmd-physics-ammo and is loaded lazily, so the Ammo WASM runtime
is initialized only after a live MMD model is mounted.
How to use
The package exposes the same scene contract as the other renderers, so it is
wired through @proj-airi/stage-ui's Stage.vue automatically when the
selected model resolves to the mmd renderer:
import { MMDScene, useMMD } from '@proj-airi/stage-ui-mmd'
<MMDScene
v-model:state="state"
:audio-context="audioContext"
:model-src="modelUrl"
:cursor-position="cursorPosition"
:current-audio-source="audioSource"
:enable-orbit-controls="true"
@error="onError"
/>
The component exposes canvasElement(), captureFrame(),
setEmotion(name, intensity), listMorphs(), and listMotions().
Runtime configuration (physics/IK/grant toggles, gaze tracking, scale,
morph overrides, emotion→motion mapping) lives in the useMMD Pinia store and
is edited through the model settings panel.
When to use it
- Displaying MMD/MikuMikuDance characters (
.pmx/.pmd) on the AIRI stage. - When you need physics-driven secondary motion, IK, and VMD playback.
When not to use it
- For VRM avatars use
@proj-airi/stage-ui-three; for Live2D use@proj-airi/stage-ui-live2d; for Spine use@proj-airi/stage-ui-spine. - In environments without WebGL/WASM support (MMD physics requires Ammo WASM).