nanakura/moeka-project
1
0
Fork 0
Code Issues Pull Requests 1 Actions 1 Packages Projects Releases Wiki Activity
Files
e87c24d60febe56d65d6f7166bd9043bab7bb2f4
moeka-project/pnpm-workspace.yaml
T
NJXClaude Opus 4.6autofix-ci[bot]
 147d077aa7 feat(inference): unify and optimize WebGPU inference pipeline (#1622) 
## Problem

The WebGPU inference pipeline had several structural issues:

1. **No unified protocol** — Kokoro TTS, Whisper ASR, and Background
Removal workers each used their own ad-hoc message formats. Adding a new
model meant reinventing worker communication from scratch.
2. **Infrastructure existed but was disconnected** —
`GPUResourceCoordinator`, `LoadQueue`, `InferenceWorkerManager`, and
`protocol.ts` were all implemented but had zero consumers. The adapters
duplicated the same lifecycle/timeout/mutex patterns independently.
3. **Performance gaps** — Kokoro only offered fp32 on WebGPU (no fp16),
Whisper warm-up compiled shaders for 187.5s of dummy audio, audio
transfer went through unnecessary WAV blob encode/decode, and
`listVoices` reloaded the model every time.
4. **Silent failures** — Whisper worker's `generate()` had no try-catch;
errors were swallowed and the main thread waited until timeout.
5. **No graceful degradation** — Whisper and Background Removal workers
hardcoded `device: 'webgpu'` with no WASM fallback.
6. **No observability** — Only Kokoro had performance tracing. No
adapter reported status to `useInferenceStatus`. No cache management UI
existed.
7. **Dead code accumulation** — Old `KokoroWorkerManager` (232 lines),
legacy Whisper message types, and scattered duplicate constants.

## Changes

### Phase 0 — Critical Performance & Bugs
- Add `fp16-webgpu` dtype for Kokoro TTS (~2x inference speed on
supported GPUs)
- Fix Whisper warm-up tensor from `[1, 128, 3000]` → `[1, 128, 1]`
(minimal shader compilation)
- Fix Whisper worker silent error bug (add try-catch to `generate()` and
`load()`)

### Phase 1 — Data Transfer & Caching
- Switch Kokoro audio to Float32Array transferable (skip WAV blob encode
in worker, lightweight WAV encode on main thread)
- Cache `listVoices` results (skip redundant model reload when adapter
state is `ready`)
- Normalize progress reporting to 0-100 across all adapters,
differentiate `warmup` phase

### Phase 2 — Protocol Unification & Infrastructure
- Migrate all 3 workers + 3 adapters to unified `protocol.ts` message
types (`load-model`, `run-inference`, `model-ready`, `inference-result`,
`progress`, `error`)
- Wire `GPUResourceCoordinator` into all adapters (VRAM allocation
tracking, LRU ordering, memory pressure events)
- Wire `LoadQueue` into all adapters (priority-based sequential model
loading: TTS=10 > ASR=5 > BG_REMOVAL=1)
- Add `coordinator.ts` global singleton for GPU coordinator + load queue
- Add WebGPU detection + WASM fallback in Whisper and Background Removal
workers

### Phase 3 — Error Recovery & Observability
- Add restart logic with exponential backoff to Whisper adapter
(matching Kokoro's existing pattern)
- Integrate `classifyError()` (OOM / DEVICE_LOST / TIMEOUT
classification) in Whisper adapter
- Extend `defaultPerfTracer` to Whisper `transcribe()` and Background
Removal `processImage()`
- Wire `useInferenceStatus` into all 3 adapters (downloading → ready →
terminated lifecycle)

### Phase 4 — Tests
- Add unit tests for `AsyncMutex` (4 tests), `LoadQueue` (4 tests),
`GPUResourceCoordinator` (7 tests) — all 15 passing

### Phase 5 — Cleanup & Features
- Delete old `KokoroWorkerManager` (232 lines, zero consumers)
- Delete orphaned `libs/workers/types.ts` (old Whisper message types)
- Clean up `workers/kokoro/types.ts` (remove legacy message types, keep
domain types)
- Create centralized `constants.ts` (MODEL_IDS, MODEL_NAMES, TIMEOUTS,
MAX_RESTARTS)
- Remove hardcoded WebGPU check from background-removal devtools pages
(worker auto-detects)
- Add `useModelPreload` composable for generic idle-time preloading
- Add `useInferencePreload` composable that reads provider config and
preloads configured local models
- Wire preloading into both `stage-web` and `stage-tamagotchi` App.vue
(Kokoro TTS preloads 3s after init)
- Add `ModelCacheManager.vue` settings component (cache size display,
per-model status, clear cache)
- Document GPU Device isolation architecture in protocol.ts

## After This PR

- All inference workers speak the same protocol → adding a new model
adapter is straightforward
- GPU memory is tracked across all models with automatic pressure
warnings at 80%/95% of VRAM budget
- Models load sequentially via priority queue → no bandwidth/VRAM
contention
- Workers auto-detect WebGPU and fall back to WASM → works on browsers
without WebGPU
- Kokoro TTS preloads during idle time → "instant" first use for
configured users
- All adapters auto-restart on worker crashes (max 3 attempts,
exponential backoff)
- 15 unit tests cover core infrastructure (mutex, queue, coordinator)
- Zero dead code remains in the inference pipeline

## Test Plan

- [x] `pnpm exec vitest run packages/stage-ui/src/libs/inference/` — 15
tests pass
- [x] `pnpm -F @proj-airi/stage-ui exec tsc --noEmit` — no TypeScript
errors
- [x] `pnpm lint:fix` — no lint errors in changed files
- [ ] Manual: verify Kokoro TTS works with fp16-webgpu on a supported
browser
- [ ] Manual: verify Whisper ASR loads and transcribes correctly
- [ ] Manual: verify Background Removal works in devtools page
- [ ] Manual: verify preloading triggers in console (`[Preload] Loading
kokoro-...`)

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-13 23:19:27 +08:00

212 lines
5.7 KiB
YAML
Raw Blame History

catalogMode: prefer
shellEmulator: true
packages:
- packages/**
- plugins/**
- integrations/**
- services/**
- examples/**
- docs/**
- apps/**
- '!**/dist/**'
overrides:
array-flatten: npm:@nolyfill/array-flatten@^1.0.44
axios: npm:feaxios@^0.0.23
is-core-module: npm:@nolyfill/is-core-module@^1.0.39
isarray: npm:@nolyfill/isarray@^1.0.44
onnxruntime-web: npm:onnxruntime-web@^1.24.3
safe-buffer: npm:@nolyfill/safe-buffer@^1.0.44
safer-buffer: npm:@nolyfill/safer-buffer@^1.0.44
side-channel: npm:@nolyfill/side-channel@^1.0.44
string.prototype.matchall: npm:@nolyfill/string.prototype.matchall@^1.0.44
patchedDependencies:
'@mediapipe/tasks-vision': patches/@mediapipe__tasks-vision.patch
'@xsai/generate-text@0.5.0-beta.2': patches/@xsai__generate-text@0.5.0-beta.2.patch
'@xsai/shared-chat@0.5.0-beta.2': patches/@xsai__shared-chat@0.5.0-beta.2.patch
'@xsai/stream-text@0.5.0-beta.2': patches/@xsai__stream-text@0.5.0-beta.2.patch
crossws@0.4.4: patches/crossws@0.4.4.patch
mineflayer-pathfinder: patches/mineflayer-pathfinder.patch
# mineflayer@4.33.0: patches/mineflayer@4.33.0.patch
pixi-live2d-display: patches/pixi-live2d-display.patch
catalog:
'@better-auth/oauth-provider': 1.5.6
'@capacitor/android': ^8.3.0
'@capacitor/barcode-scanner': ^3.0.2
'@capacitor/cli': ^8.3.0
'@capacitor/core': ^8.3.0
'@capacitor/ios': ^8.3.0
'@capacitor/local-notifications': ^8.0.2
'@electric-sql/pglite': ^0.4.4
'@electron-toolkit/preload': ^3.0.2
'@electron/notarize': ^3.1.1
'@fontsource-variable/nunito': ^5.2.7
'@guiiai/logg': ^1.2.11
'@histoire/plugin-vue': 1.0.0-beta.1
'@hono/node-ws': ^1.3.0
'@iconify-json/line-md': ^1.2.16
'@iconify-json/logos': ^1.2.11
'@iconify-json/material-symbols': ^1.2.66
'@iconify-json/mdi': ^1.2.3
'@iconify-json/tabler': ^1.2.33
'@intlify/core': ^11.3.2
'@modelcontextprotocol/sdk': ^1.29.0
'@moeru/eslint-config': 0.1.0-beta.15
'@moeru/eventa': 1.0.0-beta.3
'@moeru/std': 0.1.0-beta.17
'@napi-rs/image': ^1.12.0
'@nekopaw/tempora': 0.4.0-alpha.1
'@pinia/testing': ^1.0.3
'@proj-airi/drizzle-duckdb-wasm': ^0.4.29
'@proj-airi/iconify-meteocons': ^0.1.4
'@proj-airi/unplugin-drizzle-orm-migrations': ^0.1.6
'@proj-airi/unplugin-fetch': ^0.2.2
'@shopify/draggable': ^1.2.1
'@takumi-rs/image-response': 1.0.0-beta.20
'@types/audioworklet': ^0.0.97
'@types/d3': ^7.4.3
'@types/hast': ^3.0.4
'@types/splitpanes': ^2.2.6
'@types/unist': ^3.0.3
'@types/ws': ^8.18.1
'@vueuse/core': 14.1.0
'@xsai-ext/providers': 0.5.0-beta.2
'@xsai/embed': 0.5.0-beta.2
'@xsai/generate-speech': 0.5.0-beta.2
'@xsai/generate-text': 0.5.0-beta.2
'@xsai/generate-transcription': 0.5.0-beta.2
'@xsai/model': 0.5.0-beta.2
'@xsai/shared': 0.5.0-beta.2
'@xsai/shared-chat': 0.5.0-beta.2
'@xsai/stream-text': 0.5.0-beta.2
'@xsai/stream-transcription': 0.5.0-beta.2
'@xsai/tool': 0.5.0-beta.2
'@xsai/utils-chat': 0.5.0-beta.2
alien-signals: ^3.1.2
async-mutex: 0.5.0
better-auth: ^1.6.2
better-call: ^2.0.3
builder-util-runtime: ^9.5.1
cac: ^7.0.0
capacitor-native-settings: ^8.1.0
crossws: ^0.4.4
d3: 7.9.0
date-fns: ^4.1.0
drizzle-kit: ^0.31.10
drizzle-orm: ^0.45.2
drizzle-valibot: ^0.4.2
electron: ^40.8.5
embla-carousel-vue: 9.0.0-rc02
es-toolkit: 1.43.0
get-port-please: ^3.2.0
histoire: 1.0.0-beta.1
hono: 4.11.3
hono-rate-limiter: ^0.5.3
idb-keyval: ^6.2.2
injeca: ^0.1.8
is-network-error: ^1.3.1
isolated-vm: ^6.1.2
jose: ^6.2.2
knip: ^6.3.1
meow: ^14.1.0
mkcert: ^3.2.0
nano-staged: ^1.0.2
nanoid: 5.1.6
node-pty: ^1.1.0
ofetch: ^1.5.1
oxc-minify: ^0.124.0
pinia: ^3.0.4
posthog-js: 1.306.1
splitpanes: ^4.0.4
std-env: ^4.0.0
superjson: ^2.2.6
tsdown: ^0.21.7
tsx: ^4.21.0
uncrypto: ^0.1.3
unplugin-info: ^1.3.2
unstorage: ^1.17.5
uqr: ^0.1.3
valibot: 1.2.0
vite: ^8.0.8
vite-plugin-inspect: 12.0.0-beta.1
vite-plugin-mkcert: ^2.0.0
vue: ^3.5.32
vue-sonner: 2.0.9
web-haptics: ^0.0.6
ws: ^8.20.0
xsschema: 0.5.0-beta.2
yaml: ^2.8.3
zod: ^4.3.6
catalogs:
vitest:
'@vitest/browser-playwright': ^4.1.4
'@vitest/coverage-v8': ^4.1.4
vitest: ^4.1.4
xsai:
unspeech: ^0.1.11
ignoredBuiltDependencies:
- '@prisma/client'
- better-sqlite3
- simple-git-hooks # [workaround] postinstall script bundled in simple-git-hooks fails to execute with `enableGlobalVirtualStore: true`. Using local postinstall script to run `npx simple-git-hooks` instead.
onlyBuiltDependencies:
- '@discordjs/opus'
- '@ffmpeg-installer/darwin-arm64'
- '@ffmpeg-installer/linux-x64'
- '@napi-rs/image'
- '@parcel/watcher'
- bufferutil
- canvas
- core-js
- electron
- electron-click-drag-plugin
- electron-winstaller
- es5-ext
- esbuild
- ffmpeg-static
- isolated-vm
- less
- msw
- node-pty
- onnxruntime-node
- protobufjs
- sharp
- spawn-sync
- utf-8-validate
- vue-demi
packageExtensions:
'@formkit/auto-animate':
peerDependencies:
vue: '*'
'@huggingface/transformers':
optionalDependencies:
'onnxruntime-common': '*'
'@intlify/unplugin-vue-i18n':
peerDependencies:
vite: '*'
'@masknet/eslint-plugin':
peerDependencies:
typescript: '*'
'@pixiv/three-vrm-animation':
peerDependencies:
'@types/three': '*'
'@pixiv/three-vrm-core':
peerDependencies:
'@types/three': '*'
'@pmndrs/pointer-events':
peerDependencies:
three: '*'
'@tresjs/core':
peerDependencies:
'@types/three': '*'
'@unocss/eslint-plugin':
peerDependencies:
'@typescript-eslint/types': '*'
'vitepress':
peerDependencies:
vite: '*'
vue: '*'
'vue-sonner':
peerDependencies:
vue: '^3.2.0'
Reference in New Issue View Git Blame Copy Permalink