## Summary
This PR adds the first usable camera and view-state loop for the
experimental Godot stage.
The main design direction is that Godot owns the stage view state, while
Electron and the settings page act as lifecycle/transport/control
surfaces. The settings UI can now edit Godot camera state, and camera
changes from the Godot window can flow back to the UI.
## What Changed
- Added the shared Godot view-state contract for camera pose, avatar
bounds, snapshots, patches, and errors.
- Added Electron main bridge APIs for Godot status, scene input, view
snapshots, and view patches.
- Added Godot-side camera runtime:
- initial camera bootstrap from loaded avatar bounds
- avatar facing normalisation
- orbit, pan, wheel movement, and WASD-style camera movement
- idle persistence for camera position, yaw, pitch, and FOV
- Added settings-page Godot camera controls with coalesced updates,
restart-safe session handling, and stale request protection.
- Kept scene input simple: Electron does not cache the selected model
for replay. Each Godot start receives the current model selection from
the renderer.
## File Guide
Shared contracts:
- `packages/stage-shared/src/godot-stage/view-state.ts` defines the
Godot view-state contract and schema validation.
- `packages/stage-shared/src/godot-stage/index.ts` exports the Godot
shared contract surface.
- `packages/stage-shared/package.json` exposes the new shared entry.
Electron / renderer bridge:
- `apps/stage-tamagotchi/src/shared/eventa/index.ts` adds the Eventa
contracts for Godot status, scene input, view snapshot, patch, and error
events.
- `apps/stage-tamagotchi/src/main/services/airi/godot-stage/index.ts`
owns the Godot sidecar lifecycle, local WebSocket bridge, scene input
forwarding, and view-state forwarding.
Settings page:
- `apps/stage-tamagotchi/src/renderer/pages/settings/models/index.vue`
wires the settings page to Godot status, scene input, snapshots, and
patches.
-
`apps/stage-tamagotchi/src/renderer/pages/settings/models/godot-view-patch-queue.ts`
coalesces high-frequency camera edits before sending them to Godot.
-
`apps/stage-tamagotchi/src/renderer/pages/settings/models/godot-view-session.ts`
tracks renderer-local Godot session epochs so stale async results are
ignored.
Shared settings UI:
-
`packages/stage-ui/src/components/scenarios/settings/model-settings/godot.vue`
renders the Godot camera controls.
-
`packages/stage-ui/src/components/scenarios/settings/model-settings/panel.vue`
routes the model settings panel between Live2D, VRM, and Godot settings.
-
`packages/stage-ui/src/components/scenarios/settings/model-settings/runtime.ts`
resolves model-settings runtime display state and Godot camera control
range.
- `packages/stage-ui/src/components/data-pane/property-number.vue`
updates number control behavior so config changes such as min/max/range
are reflected correctly.
Godot runtime:
- `engines/stage-tamagotchi-godot/scripts/StageRoot.cs` wires scene
loading, view runtime, and transport together.
- `engines/stage-tamagotchi-godot/scripts/scene/StageSceneController.cs`
loads VRM avatars and normalizes avatar facing direction.
- `engines/stage-tamagotchi-godot/scripts/view/StageViewRuntime.cs` owns
Godot view-state updates, snapshots, patch application, and idle
persistence.
-
`engines/stage-tamagotchi-godot/scripts/view/StageCameraPoseController.cs`
computes avatar bounds, bootstraps the camera, applies camera pose, and
handles camera movement math.
-
`engines/stage-tamagotchi-godot/scripts/view/StageCameraInputController.cs`
maps mouse and keyboard input into camera operations.
- `engines/stage-tamagotchi-godot/scripts/view/StageViewStateRules.cs`
centralizes view-state defaults and validation rules.
- `engines/stage-tamagotchi-godot/scripts/view/StageViewStateStore.cs`
persists and loads Godot camera view state.
- `engines/stage-tamagotchi-godot/scripts/transport/StageViewJson.cs`
parses and serializes view-state transport payloads.
-
`engines/stage-tamagotchi-godot/scripts/transport/StageViewPayloads.cs`
defines the C# payload records used by the transport layer.
---------
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
1. Introduce the global shortcut service
1. Add more concrete failure reasons for shortcut registration attempts
1. Add a devtool page to test (un) registering and triggering shortcuts
---
<img width="1174" height="921" alt="Screenshot 2026-05-10 at 19 33 45"
src="https://github.com/user-attachments/assets/10712013-fd49-4285-bdc9-4e6955d9c3a7"
/>
---------
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
## Problem
The WebGPU inference pipeline had several structural issues:
1. **No unified protocol** — Kokoro TTS, Whisper ASR, and Background
Removal workers each used their own ad-hoc message formats. Adding a new
model meant reinventing worker communication from scratch.
2. **Infrastructure existed but was disconnected** —
`GPUResourceCoordinator`, `LoadQueue`, `InferenceWorkerManager`, and
`protocol.ts` were all implemented but had zero consumers. The adapters
duplicated the same lifecycle/timeout/mutex patterns independently.
3. **Performance gaps** — Kokoro only offered fp32 on WebGPU (no fp16),
Whisper warm-up compiled shaders for 187.5s of dummy audio, audio
transfer went through unnecessary WAV blob encode/decode, and
`listVoices` reloaded the model every time.
4. **Silent failures** — Whisper worker's `generate()` had no try-catch;
errors were swallowed and the main thread waited until timeout.
5. **No graceful degradation** — Whisper and Background Removal workers
hardcoded `device: 'webgpu'` with no WASM fallback.
6. **No observability** — Only Kokoro had performance tracing. No
adapter reported status to `useInferenceStatus`. No cache management UI
existed.
7. **Dead code accumulation** — Old `KokoroWorkerManager` (232 lines),
legacy Whisper message types, and scattered duplicate constants.
## Changes
### Phase 0 — Critical Performance & Bugs
- Add `fp16-webgpu` dtype for Kokoro TTS (~2x inference speed on
supported GPUs)
- Fix Whisper warm-up tensor from `[1, 128, 3000]` → `[1, 128, 1]`
(minimal shader compilation)
- Fix Whisper worker silent error bug (add try-catch to `generate()` and
`load()`)
### Phase 1 — Data Transfer & Caching
- Switch Kokoro audio to Float32Array transferable (skip WAV blob encode
in worker, lightweight WAV encode on main thread)
- Cache `listVoices` results (skip redundant model reload when adapter
state is `ready`)
- Normalize progress reporting to 0-100 across all adapters,
differentiate `warmup` phase
### Phase 2 — Protocol Unification & Infrastructure
- Migrate all 3 workers + 3 adapters to unified `protocol.ts` message
types (`load-model`, `run-inference`, `model-ready`, `inference-result`,
`progress`, `error`)
- Wire `GPUResourceCoordinator` into all adapters (VRAM allocation
tracking, LRU ordering, memory pressure events)
- Wire `LoadQueue` into all adapters (priority-based sequential model
loading: TTS=10 > ASR=5 > BG_REMOVAL=1)
- Add `coordinator.ts` global singleton for GPU coordinator + load queue
- Add WebGPU detection + WASM fallback in Whisper and Background Removal
workers
### Phase 3 — Error Recovery & Observability
- Add restart logic with exponential backoff to Whisper adapter
(matching Kokoro's existing pattern)
- Integrate `classifyError()` (OOM / DEVICE_LOST / TIMEOUT
classification) in Whisper adapter
- Extend `defaultPerfTracer` to Whisper `transcribe()` and Background
Removal `processImage()`
- Wire `useInferenceStatus` into all 3 adapters (downloading → ready →
terminated lifecycle)
### Phase 4 — Tests
- Add unit tests for `AsyncMutex` (4 tests), `LoadQueue` (4 tests),
`GPUResourceCoordinator` (7 tests) — all 15 passing
### Phase 5 — Cleanup & Features
- Delete old `KokoroWorkerManager` (232 lines, zero consumers)
- Delete orphaned `libs/workers/types.ts` (old Whisper message types)
- Clean up `workers/kokoro/types.ts` (remove legacy message types, keep
domain types)
- Create centralized `constants.ts` (MODEL_IDS, MODEL_NAMES, TIMEOUTS,
MAX_RESTARTS)
- Remove hardcoded WebGPU check from background-removal devtools pages
(worker auto-detects)
- Add `useModelPreload` composable for generic idle-time preloading
- Add `useInferencePreload` composable that reads provider config and
preloads configured local models
- Wire preloading into both `stage-web` and `stage-tamagotchi` App.vue
(Kokoro TTS preloads 3s after init)
- Add `ModelCacheManager.vue` settings component (cache size display,
per-model status, clear cache)
- Document GPU Device isolation architecture in protocol.ts
## After This PR
- All inference workers speak the same protocol → adding a new model
adapter is straightforward
- GPU memory is tracked across all models with automatic pressure
warnings at 80%/95% of VRAM budget
- Models load sequentially via priority queue → no bandwidth/VRAM
contention
- Workers auto-detect WebGPU and fall back to WASM → works on browsers
without WebGPU
- Kokoro TTS preloads during idle time → "instant" first use for
configured users
- All adapters auto-restart on worker crashes (max 3 attempts,
exponential backoff)
- 15 unit tests cover core infrastructure (mutex, queue, coordinator)
- Zero dead code remains in the inference pipeline
## Test Plan
- [x] `pnpm exec vitest run packages/stage-ui/src/libs/inference/` — 15
tests pass
- [x] `pnpm -F @proj-airi/stage-ui exec tsc --noEmit` — no TypeScript
errors
- [x] `pnpm lint:fix` — no lint errors in changed files
- [ ] Manual: verify Kokoro TTS works with fp16-webgpu on a supported
browser
- [ ] Manual: verify Whisper ASR loads and transcribes correctly
- [ ] Manual: verify Background Removal works in devtools page
- [ ] Manual: verify preloading triggers in console (`[Preload] Loading
kokoro-...`)
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Neko Ayaka