Commit Graph
91 Commits
Author SHA1 Message Date
leafyy aa69ee62b5 refactor(lipsync): share renderer vowel policy (#2374)
## Description

The VRM and MMD renderers had the same two boundary problems:

- Both imported the Stage audio store from the `stage-ui` source tree.
- Both kept copies of the wLipSync profile and model-neutral vowel
policy.

This PR solves both problems as one architecture change.

```text
Stage owns AudioContext and the current audio source
  -> ThreeScene / MMDScene receive both as inputs
    -> model-driver-lipsync owns the profile and shared vowel policy
      -> VRM maps AEIOU to aa / ee / ih / oh / ou
      -> MMD maps AEIOU to vowelA / vowelE / vowelI / vowelO / vowelU
```

The shared driver owns viseme projection, winner and runner selection,
silence handling, and smoothing. It has no Vue, Three.js, VRM, or MMD
dependency. Each renderer keeps only its model-specific mapping.

The renderer packages no longer import the Stage audio store or depend
directly on `wlipsync`. Stage passes the owning `AudioContext` down with
the current audio source. Cleanup disconnects only the wLipSync node
owned by that renderer.

## Current solution

The package-boundary review identified two related findings:

- Finding 3: The VRM and MMD renderers imported `stage-ui` audio state
through source paths.
- Finding 4: The same renderers duplicated the wLipSync profile and
model-neutral vowel policy.

Both findings came from one missing boundary between Stage, the lip-sync
driver, and the renderer adapters. This PR fixes that boundary with one
ownership change.

| Layer | Responsibility |
| --- | --- |
| `stage-ui` | Owns the `AudioContext` and the current audio source. It
passes both values to each renderer. |
| `stage-ui-three` and `stage-ui-mmd` | Connect renderer components to
the shared driver. They do not import the Stage audio store. |
| `model-driver-lipsync/runtime/wlipsync` | Exposes `createWLipSyncNode`
as the public runtime entry. |
| `model-driver-lipsync/shared/wlipsync` | Owns the profile, public
types, and model-neutral vowel policy. |

Runtime flow:

1. `Stage.vue` passes `audioContext` and `currentAudioSource` to
`ThreeScene` or `MMDScene`.
2. The renderer composable watches the context and creates one wLipSync
node for that context.
3. The source watcher connects the current audio source to that node.
4. Each render frame sends the wLipSync values to
`createWLipSyncVowelDriver`.
5. The renderer adapter applies the returned AEIOU weights to its model
controls.
6. Cleanup disconnects the exact source and node pair that the renderer
owns.

The shared policy keeps the current renderer behavior:

- It maps the raw `S` viseme to `I`.
- It selects the two strongest vowels.
- It uses the existing volume and silence thresholds.
- It keeps the 160 ms silence window.
- It keeps the existing attack, release, cap, and output scale values.

The renderer-specific mappings stay in their renderer packages:

- VRM maps AEIOU to `aa`, `ee`, `ih`, `oh`, and `ou`.
- MMD maps AEIOU to `vowelA`, `vowelE`, `vowelI`, `vowelO`, and
`vowelU`.

This solution removes both duplicate profile files. It also removes the
direct `wlipsync` dependencies from both renderer packages.

## Linked Issues

None.

## Additional Context

There is no intended visual change. This refactor preserves the existing
vowel weights and renderer mappings.

Verification:

- `sem diff --staged --no-cosmetics -v --file-exts .ts .tsx`
- `sem diff --staged --no-cosmetics -v --file-exts .vue`
- `sem impact --entity-id <useVRMLipSync> --json --dependents`
- `sem impact --entity-id <useMMDLipSync> --json --dependents`
- `pnpm -F @proj-airi/model-driver-lipsync test` — 3 tests passed
- `pnpm -F @proj-airi/stage-ui-three exec vitest run` — 18 tests passed
- `pnpm -F @proj-airi/stage-ui-mmd exec vitest run` — 23 tests passed
- Type checks passed for `model-driver-lipsync`, `stage-ui-three`,
`stage-ui-mmd`, and `stage-ui`
- `pnpm install --frozen-lockfile --ignore-scripts`
- `pnpm lint` — 0 errors and 7 existing warnings outside this change
- `git diff --check`

The full workspace `pnpm typecheck` was also run. In the isolated
worktree it stopped in unchanged `electron-vueuse` and
`plugin-sdk-tamagotchi` because their generated dependency outputs were
not present. All affected package type checks above passed.
2026-08-31 14:58:43 +08:00
Neko Ayaka 2c1e223c8d release: v0.12.0-beta.5 2026-08-30 02:31:30 +08:00
Neko Ayaka fe38fb9ded release: v0.12.0-beta.4 2026-08-28 04:59:05 +08:00
Neko Ayaka 175ad08477 release: v0.12.0-beta.3 2026-08-27 20:48:27 +08:00
Neko Ayaka 146b3da65a Revert "style: lint"
This reverts commit 98f40d7d0b.
2026-08-26 20:13:10 +08:00
Neko Ayaka 98f40d7d0b style: lint 2026-08-26 19:49:58 +08:00
Neko Ayaka f53e82a9b3 release: v0.12.0-beta.2 2026-08-26 04:17:25 +08:00
Neko Ayaka f14a7ac9a1 release: v0.12.0-beta.1 2026-08-24 03:55:37 +08:00
Neko Ayaka dbf8124888 release: v0.11.3 2026-07-19 04:03:59 +08:00
Neko Ayaka b43b8944bf release: v0.11.2 2026-07-18 10:37:13 +08:00
Neko Ayaka 089ca99108 release: v0.11.1 2026-07-17 03:35:49 +08:00
Neko Ayaka 3779eb6103 release: v0.11.0 2026-07-08 01:20:12 +08:00
RainbowBird 7841df28f8 build(deps): migrate workspace dependencies to catalogs
Move direct external dependency versions into pnpm catalogs and update workspace packages to reference catalog entries.

Signed-off-by: RainbowBird <git@luoling.moe>

Commit-Message-Assisted-by: Codex
2026-06-03 17:12:13 +08:00
Neko Ayaka 6d61699d10 release: v0.10.2 2026-05-07 19:36:33 +08:00
Neko Ayaka 6dd370433a release: v0.10.1 2026-05-05 14:10:55 +08:00
Neko Ayaka ca722f69b7 release: v0.10.0 2026-05-03 23:17:03 +08:00
Neko Ayaka dd284f0f27 release: v0.9.0 2026-04-10 14:48:30 +08:00
Neko Ayaka a0caaf3214 release: v0.9.0-rc.1 2026-04-10 03:40:59 +08:00
Neko Ayaka dc7132bb91 release: v0.9.0-beta.7 2026-04-08 04:17:03 +08:00
Neko Ayaka 7f7f50b462 release: v0.9.0-beta.6 2026-04-07 04:58:46 +08:00
Neko Ayaka 389388820f release: v0.9.0-beta.5 2026-04-06 02:10:21 +08:00
Neko Ayaka 476ef2018f style: many typescript v6 pending errors 2026-04-05 02:07:09 +08:00
LemonNeko 60112eaa85 release: v0.9.0-beta.4 2026-04-01 19:19:22 +08:00
Makito e9be0cf98c release: v0.9.0-beta.3 and ios build 8 2026-04-01 15:35:07 +09:00
Neko Ayaka 497524a22f release: v0.9.0-beta.2 2026-03-31 17:33:53 +08:00
Neko Ayaka c751f48401 release: v0.9.0-beta.1 2026-03-31 14:35:19 +08:00
Neko Ayaka 21a092be71 release: v0.9.0-alpha.36 2026-03-30 22:49:38 +08:00
Makito 0e9122b5ac release: v0.9.0-alpha.35 2026-03-30 14:44:33 +09:00
Neko Ayaka fbf6a05f1e release: v0.9.0-alpha.34 2026-03-29 15:28:42 +08:00
Neko Ayaka d5f24107b7 release: v0.9.0-alpha.32 2026-03-29 02:44:08 +08:00
Neko Ayaka 06570b4dde release: v0.9.0-alpha.31 2026-03-28 12:57:31 +08:00
Makito c0e499b044 release: v0.9.0-alpha.30 2026-03-28 11:52:22 +09:00
Ayaka Neko 129fbf3fcb release: v0.9.0-alpha.29 2026-03-28 04:06:59 +08:00
Neko Ayaka 5535907aa8 release: v0.9.0-alpha.28 2026-03-27 18:01:13 +08:00
Neko Ayaka 6675b3234d release: v0.9.0-alpha.27 2026-03-27 15:55:49 +08:00
Neko Ayaka cd826dcbce release: v0.9.0-alpha.26 2026-03-27 15:33:24 +08:00
Neko Ayaka fc72b7a56c release: v0.9.0-alpha.25 2026-03-27 14:47:31 +08:00
Neko Ayaka e2080c7178 release: v0.9.0-alpha.24 2026-03-27 03:39:08 +08:00
Neko Ayaka 539ba83a61 release: v0.9.0-alpha.23 2026-03-26 23:39:14 +08:00
Neko Ayaka 45ce8ba602 release: v0.9.0-alpha.22 2026-03-25 01:19:04 +08:00
Neko Ayaka 0af4217666 release: v0.9.0-alpha.21 2026-03-24 09:21:27 +08:00
Neko Ayaka 11bb6b8e0a release: v0.9.0-alpha.20 2026-03-23 21:41:53 +08:00
Neko Ayaka ea7c832adc release: v0.9.0-alpha.19 2026-03-23 20:03:39 +08:00
Neko Ayaka a3b9a4b037 release: v0.9.0-alpha.18 2026-03-19 14:45:49 +08:00
Makito a49bd97ed3 release: v0.9.0-alpha.17 2026-03-18 14:45:49 +09:00
Neko Ayaka 72925c1372 release: v0.9.0-alpha.16 2026-03-15 04:04:07 +08:00
Garfield Lee 9fc9955615 release: v0.9.0-alpha.15 2026-03-14 18:23:50 +08:00
Neko Ayaka da62f1c16b release: v0.9.0-alpha.14 2026-03-12 01:48:21 +08:00
Neko Ayaka 53f79713d7 release: v0.9.0-alpha.13 2026-03-11 19:36:13 +08:00
Neko Ayaka 96d3149909 release: v0.9.0-alpha.12 2026-03-11 04:35:37 +08:00