Commit Graph
72 Commits
Author SHA1 Message Date
leafyy aa69ee62b5 refactor(lipsync): share renderer vowel policy (#2374)
## Description

The VRM and MMD renderers had the same two boundary problems:

- Both imported the Stage audio store from the `stage-ui` source tree.
- Both kept copies of the wLipSync profile and model-neutral vowel
policy.

This PR solves both problems as one architecture change.

```text
Stage owns AudioContext and the current audio source
  -> ThreeScene / MMDScene receive both as inputs
    -> model-driver-lipsync owns the profile and shared vowel policy
      -> VRM maps AEIOU to aa / ee / ih / oh / ou
      -> MMD maps AEIOU to vowelA / vowelE / vowelI / vowelO / vowelU
```

The shared driver owns viseme projection, winner and runner selection,
silence handling, and smoothing. It has no Vue, Three.js, VRM, or MMD
dependency. Each renderer keeps only its model-specific mapping.

The renderer packages no longer import the Stage audio store or depend
directly on `wlipsync`. Stage passes the owning `AudioContext` down with
the current audio source. Cleanup disconnects only the wLipSync node
owned by that renderer.

## Current solution

The package-boundary review identified two related findings:

- Finding 3: The VRM and MMD renderers imported `stage-ui` audio state
through source paths.
- Finding 4: The same renderers duplicated the wLipSync profile and
model-neutral vowel policy.

Both findings came from one missing boundary between Stage, the lip-sync
driver, and the renderer adapters. This PR fixes that boundary with one
ownership change.

| Layer | Responsibility |
| --- | --- |
| `stage-ui` | Owns the `AudioContext` and the current audio source. It
passes both values to each renderer. |
| `stage-ui-three` and `stage-ui-mmd` | Connect renderer components to
the shared driver. They do not import the Stage audio store. |
| `model-driver-lipsync/runtime/wlipsync` | Exposes `createWLipSyncNode`
as the public runtime entry. |
| `model-driver-lipsync/shared/wlipsync` | Owns the profile, public
types, and model-neutral vowel policy. |

Runtime flow:

1. `Stage.vue` passes `audioContext` and `currentAudioSource` to
`ThreeScene` or `MMDScene`.
2. The renderer composable watches the context and creates one wLipSync
node for that context.
3. The source watcher connects the current audio source to that node.
4. Each render frame sends the wLipSync values to
`createWLipSyncVowelDriver`.
5. The renderer adapter applies the returned AEIOU weights to its model
controls.
6. Cleanup disconnects the exact source and node pair that the renderer
owns.

The shared policy keeps the current renderer behavior:

- It maps the raw `S` viseme to `I`.
- It selects the two strongest vowels.
- It uses the existing volume and silence thresholds.
- It keeps the 160 ms silence window.
- It keeps the existing attack, release, cap, and output scale values.

The renderer-specific mappings stay in their renderer packages:

- VRM maps AEIOU to `aa`, `ee`, `ih`, `oh`, and `ou`.
- MMD maps AEIOU to `vowelA`, `vowelE`, `vowelI`, `vowelO`, and
`vowelU`.

This solution removes both duplicate profile files. It also removes the
direct `wlipsync` dependencies from both renderer packages.

## Linked Issues

None.

## Additional Context

There is no intended visual change. This refactor preserves the existing
vowel weights and renderer mappings.

Verification:

- `sem diff --staged --no-cosmetics -v --file-exts .ts .tsx`
- `sem diff --staged --no-cosmetics -v --file-exts .vue`
- `sem impact --entity-id <useVRMLipSync> --json --dependents`
- `sem impact --entity-id <useMMDLipSync> --json --dependents`
- `pnpm -F @proj-airi/model-driver-lipsync test` — 3 tests passed
- `pnpm -F @proj-airi/stage-ui-three exec vitest run` — 18 tests passed
- `pnpm -F @proj-airi/stage-ui-mmd exec vitest run` — 23 tests passed
- Type checks passed for `model-driver-lipsync`, `stage-ui-three`,
`stage-ui-mmd`, and `stage-ui`
- `pnpm install --frozen-lockfile --ignore-scripts`
- `pnpm lint` — 0 errors and 7 existing warnings outside this change
- `git diff --check`

The full workspace `pnpm typecheck` was also run. In the isolated
worktree it stopped in unchanged `electron-vueuse` and
`plugin-sdk-tamagotchi` because their generated dependency outputs were
not present. All affected package type checks above passed.
2026-08-31 14:58:43 +08:00
Neko Ayaka 146b3da65a Revert "style: lint"
This reverts commit 98f40d7d0b.
2026-08-26 20:13:10 +08:00
Neko Ayaka 98f40d7d0b style: lint 2026-08-26 19:49:58 +08:00
Neko Ayaka 6a24582696 chore(deps): bump dependencies 2026-08-26 19:01:04 +08:00
Neko Ayaka 07d5baf410 fix(stage-ui,stage-tamagotchi): should not flash messages, stream correctly, and sync tool call renderer 2026-08-10 14:32:42 +08:00
0xSelenicDove 78d6588cb0 feat(stage-*): add clickable VRM body-part reactions (#2068) 2026-07-20 16:13:03 +08:00
RainbowBird 007fae6810 chore: update package dependencies and remove specific catalog references
- Removed specific catalog references for '@unocss/core', 'es-toolkit', 'nanoid', and 'valibot' in pnpm-workspace.yaml and various service package.json files.
- Updated versions for '@vueuse/core', 'es-toolkit', 'nanoid', 'reka-ui', and 'valibot' to their latest compatible versions.
- Cleaned up unused catalog entries in pnpm-workspace.yaml.
2026-06-07 19:45:28 +08:00
RainbowBird 7841df28f8 build(deps): migrate workspace dependencies to catalogs
Move direct external dependency versions into pnpm catalogs and update workspace packages to reference catalog entries.

Signed-off-by: RainbowBird <git@luoling.moe>

Commit-Message-Assisted-by: Codex
2026-06-03 17:12:13 +08:00
Lilia_Chenandautofix-ci[bot] 484fb1f767 fix(stage-ui-three): OrbitControl initialisation (#1877)
## Summary

- Fix the first Live2D -> VRM switch camera regression where
OrbitControls could clamp the camera into the model.
- Remove the VRM render scale control from the model settings panel.
- Remove the now-unused `settings.vrm.render-scale` locale entries.
- Ignore Godot/.NET `bin/` and `obj/` build artifacts under
`engines/stage-tamagotchi-godot`.

## Root Cause

On the first VRM load, `OrbitControls` can mount before the VRM
bootstrap writes a real `modelSize`. The previous watcher copied
`modelSize.z = 0` into `minDistance` and `maxDistance`, so later camera
updates were clamped onto the target before the bootstrap camera pose
finished applying.

## Changes

- Skip OrbitControls distance-bound updates until `modelSize.z` is
finite and non-zero.
- Keep the render scale runtime state intact, but remove the VRM
settings UI entry so it is no longer hot-updated from the panel.
- Add Godot project ignores for local .NET build output.

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-05-27 08:35:43 +08:00
Neko Ayaka 92b20b0dd2 fix(stage-*): many live2d and vrm issues, default values were missing 2026-05-21 02:25:04 +08:00
DrHuangMHT 7f9eeb715d fix(stage-*): improve cursor tracking (#1842) 2026-05-20 22:30:38 +08:00
Makito 7c76ea680c chore(pnpm): fix pnpm-workspaces issues and update catalog for vitest 2026-05-20 11:38:20 +09:00
DrHuangMHT c2bf73c95d fix(stage-*): follow-up fix for #1696 (#1809) 2026-05-19 13:54:41 +08:00
DrHuangMHT 85627d34ad feat(stage-*): allow setting character offset in landscape (#1696) 2026-05-06 17:04:18 +08:00
Richard Pinedo 4e6da5ef04 feat(stage-*): port artistry & chatbox enhancements (#1636)
Co-authored-by-agent: Unknown <unknown@example.com>
2026-04-24 23:25:04 +08:00
Neko Ayaka 2ab21879dd chore(deps): bump dependencies 2026-04-18 19:18:17 +08:00
Neko Ayaka 95eac8e41c chore(deps): bump dependencies 2026-04-11 04:00:49 +08:00
Lilia-Chen 631a6ba65f fix: decouple vrm loading reason and ThreeScene phase transition reason 2026-04-02 00:11:38 +01:00
Lilia_Chen 522e48b063 fix(stage-ui-three): model offset and camera position setting is now persistent in stage-web app (#1543) 2026-04-02 07:04:03 +08:00
Lilia_Chenandautofix-ci[bot] d9ba2ab40e refactor(stage-ui-three): improved outline logic (#1536)
* refactor(stage-ui-three): improved outline logic

* [autofix.ci] apply automated fixes

* fix(stage-ui-three): restored runtime hooks being exposed

* some type fix

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-01 07:48:00 +08:00
Lilia-Chen 718e91a833 refactor(stage-ui-three): add hooks in VRM loading workflow 2026-03-31 00:24:13 +01:00
Lilia-Chen 1037614a89 fix(stage-ui-three): removed duplicated IBL inject 2026-03-26 14:23:55 +00:00
RainbowBird b0576d3b89 chore(deps): move eventa to catalog 2026-03-26 21:45:01 +08:00
Makito 279b4db1e7 fix(stage-ui-three): call nodeConstraintManager.update() in render loop 2026-03-25 23:59:11 +09:00
Lilia_Chen 088ce34a2a fix(stage-ui-three): cutout bug fix (#1471)
* fix(stage-ui-three): cutout bug fix

* fix(stage-ui-three): gemini review comment

* fix(stage-ui-three): codex review comment
2026-03-25 08:49:21 +08:00
Neko Ayaka 0ace1fb26c fix(stage-ui-three): type error 2026-03-23 14:08:42 +08:00
Neko Ayaka ce60323d8a style: lint 2026-03-23 02:16:10 +08:00
Neko Ayaka ac1de680d0 chore(deps): bump dependencies 2026-03-23 02:15:40 +08:00
Iro eb53744c08 fix(stage-tamagotchi): harden caption overlay follow+mouse handling (#1402) 2026-03-17 14:23:04 +08:00
dec2b04924 refactor: vrm performance tracing and better lifecycle management (#1194)
* refactor: vrm performance tracing and better lifecycle management

* Apply suggestion from @Lilia-Chen

* Apply suggestion from @Lilia-Chen

* Apply suggestion from @Lilia-Chen

* Apply suggestion from @Lilia-Chen

* Rename lag-visualizer.vue to performance-visualizer.vue

* Apply suggestion from @Lilia-Chen

* Apply suggestion from @Lilia-Chen

* Apply suggestion from @Lilia-Chen

* Apply suggestion from @sumimakito

Co-authored-by: Makito <i@maki.to>

* Apply suggestion from @sumimakito

Co-authored-by: Makito <i@maki.to>

* Apply suggestion from @sumimakito

Co-authored-by: Makito <i@maki.to>

* Apply suggestion from @Lilia-Chen

* Apply suggestion from @Lilia-Chen

* [autofix.ci] apply automated fixes

* Apply suggestion from @Lilia-Chen

* [autofix.ci] apply automated fixes

* Apply suggestion from @Lilia-Chen

* Apply suggestion from @Lilia-Chen

* Apply suggestion from @Lilia-Chen

* Apply suggestion from @Lilia-Chen

* Apply suggestion from @Lilia-Chen

* Apply suggestion from @Lilia-Chen

* Apply suggestion from @Lilia-Chen

* Apply suggestion from @Lilia-Chen

* Apply suggestion from @Lilia-Chen

* Apply suggestion from @Lilia-Chen

* Apply suggestion from @Lilia-Chen

* Apply suggestion from @Lilia-Chen

* Apply suggestion from @Lilia-Chen

* [autofix.ci] apply automated fixes

* Apply suggestion from @Lilia-Chen

* Apply suggestion from @Lilia-Chen

* [autofix.ci] apply automated fixes

* Apply suggestion from @Lilia-Chen

* Apply suggestion from @Lilia-Chen

* Apply suggestion from @Lilia-Chen

* fix: some tiny bugs and model URL lifecycle

* fix: TresCanvas size=0 bug

* Delete .claude/settings.json

* fix: some clean-up

* [autofix.ci] apply automated fixes

* fix: use screen to wrap up TresCanvas

* Delete .claude/settings.json

---------

Co-authored-by: Makito <i@maki.to>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Neko <neko@ayaka.moe>
2026-03-12 09:20:20 +08:00
NJX 1a5204878d fix(stage-ui-three): improve VRM expression rendering and add missing emotion mappings (#1147) 2026-03-06 13:57:24 +08:00
NJX 5d571c51e0 feat(stage-ui-three): add anti-aliasing support for 3D models (#1150) 2026-03-06 13:56:25 +08:00
Neko Ayaka 92d87833be chore(deps): bump dependencies 2026-02-26 18:33:52 +08:00
Neko Ayaka d051a07da9 chore(deps): bump dependencies 2026-01-21 05:42:01 +08:00
Lilia_Chenandautofix-ci[bot] 0093803920 feat(stage-ui-*): ACT special tokens for future motion gen (#974)
* feat(stage-ui*):ACT special tokens

* test(stage-ui): llm-marker-parser test updates for ACT tokens

* [autofix.ci] apply automated fixes

* style(stage-ui): apply gemini code review notes

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-01-20 09:32:47 +08:00
Lilia_Chenandautofix-ci[bot] 372f356964 fix(stage-ui-three): lipsync update() (#971)
* bug-fix: lipsync update

* [autofix.ci] apply automated fixes

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-01-18 20:55:43 +08:00
Neko Ayaka e38366d37b fix(deps): sync vue to catalog: 2026-01-17 15:57:54 +08:00
Neko Ayaka 222e776beb fix(stage-ui-three): memory leak & should update hook issue 2026-01-09 18:11:23 +08:00
Neko Ayaka bd9d5be254 feat(stage-ui,stage-ui-three): support fade on hover for three 2026-01-08 15:02:52 +08:00
Neko Ayaka 20b6978a9d refactor(stage-ui,stage-ui-live2d,stage-ui-three): split out stage-ui-live2d component 2026-01-08 00:31:07 +08:00
RinandNeko Ayaka 4f0477a2a0 fix(stage-ui,stage-ui-three): resolve tamagotchi VRM loading screen hang on startup (#904)
---------

Co-authored-by: Neko Ayaka <neko@ayaka.moe>
2026-01-07 22:23:01 +08:00
Neko Ayaka 0ef110aae5 feat(stage-ui-three): support loading three as preview cover of model 2026-01-04 04:50:44 +08:00
Lilia_ChenandMakito ecd5eee155 feat(model-driver-mediapipe): integrate MediaPipe as a motion capture tool (#828)
---------

Co-authored-by: Makito <i@maki.to>
2025-12-30 12:11:08 +08:00
Neko Ayaka a07854ca31 chore(deps): bump dependencies 2025-12-27 04:26:56 +08:00
Neko Ayaka 2ffe666368 chore(deps): bump dependencies 2025-12-21 18:41:11 +08:00
Neko Ayaka 7158c200bb chore(deps): bump dependencies 2025-12-21 18:35:34 +08:00
Neko Ayaka c9a9ec7bfa Revert "chore(deps): bump dependencies"
This reverts commit 349810ff27.
2025-12-21 17:16:47 +08:00
Neko Ayaka 349810ff27 chore(deps): bump dependencies 2025-12-21 15:58:52 +08:00
Neko Ayaka 1f24555797 chore(stage-ui-three): default to hemisphere 2025-12-17 10:53:06 +08:00
Neko Ayaka a3dba4f3cf chore(deps): bump dependencies & electron-builder 2025-12-13 16:23:45 +08:00