## Summary
Addresses **Phase 1** of the inference infrastructure roadmap (#1661).
Six correctness and resilience issues fixed across adapters, workers,
and the unified protocol layer.
**Scope:** `packages/stage-ui/src/libs/inference/`,
`packages/stage-ui/src/libs/workers/`, `packages/stage-ui/src/workers/`
---
## Problems & Motivation
### 1. Kokoro Singleton cannot recover from terminal state
`getKokoroAdapter()` creates a singleton that is never reset. Once the
adapter reaches `terminated` or `error` (restarts exhausted), the
singleton is permanently dead -- users must refresh the page.
**Root cause:** `globalAdapter` is only re-created when `null`, never
checks the state of the existing instance. `scheduleRestart()` silently
returns when retries are exhausted, leaving state stuck at `'error'`
instead of transitioning to `'terminated'`.
**Fix:**
- `getKokoroAdapter()` now detects `terminated` / `error` states and
re-creates the adapter automatically
- `scheduleRestart()` in both Kokoro and Whisper adapters transitions to
`'terminated'` when max restarts are exhausted, making the dead state
detectable
### 2. `voices!` non-null assertion after nullable assignment
`kokoro.ts:275` assigns `voices = (response.metadata?.voices as Voices)
?? null` (can be null), but L288 returns `voices!` -- if the worker
returns unexpected metadata, callers receive `null` but the type system
promises `Voices`, causing downstream crashes.
**Fix:** Replaced with explicit null check + throw, routing into the
existing error recovery path.
### 3. Whisper Worker silently drops concurrent requests
The worker's `runInference()` has `if (processing) return` which
silently drops the request. The adapter's `waitForMessage()` never
receives a response, causing the Promise to hang for 120 seconds until
timeout.
**Root cause:** The bare boolean is safe in a single-threaded worker --
the real bug is the silent `return` with no error response.
**Fix:** Replaced with `sendError(requestId, ...)` so the adapter
receives an immediate rejection.
### 4. Whisper adapter leaks event listeners on worker restart
`ensureWorker()` attaches anonymous `message` and `error` listeners each
time a new Worker is created. `destroyWorker()` only calls
`worker.terminate()` without removing listeners. On restart paths
(`scheduleRestart -> ensureWorker`), old Worker listener closures may
prevent GC.
**Impact:** Whisper is most critical (2 listeners + restart logic).
Kokoro and Background Removal have the same pattern but lower risk.
**Fix:** All 3 adapters now store listener references and call
`removeEventListener` before `terminate()`. Whisper's
`terminateAdapter()` routes through `destroyWorker()` instead of direct
`worker.terminate()`.
### 5. Error classification is incomplete
`classifyError()` only matches 3 string patterns (OOM, DEVICE_LOST,
TIMEOUT). `LOAD_FAILED` and `INFERENCE_FAILED` are defined in the
`InferenceErrorCode` type but never produced. All workers hardcode
`recoverable: false`.
**Fix:**
- `classifyError(error, phase?)` gains a `phase` parameter: specific
patterns (OOM etc.) take priority, unmatched errors fall through to
`LOAD_FAILED` / `INFERENCE_FAILED` based on phase
- New `isRecoverable(code)` helper: TIMEOUT and DEVICE_LOST are
recoverable
- All 3 workers' `sendError()` updated to pass phase and use dynamic
recoverability
### 6. No dtype/device fallback chain
Kokoro Worker attempts model loading once with the requested dtype;
failure is terminal. Whisper Worker hardcodes `encoder_model: 'fp16'`
which crashes on devices that don't support it.
**Fix:**
- **Kokoro Worker:** New `DTYPE_FALLBACK` and `DEVICE_FALLBACK` maps.
Tries `fp16 -> fp32 -> q8 -> q4` on the same device, then `webgpu ->
wasm` device fallback. Reports `actualDtype` and `actualDevice` in
`ModelReadyResponse.metadata`
- **Whisper Worker:** Wraps fp16 encoder load in try-catch; falls back
to fp32
---
## State after changes
| Component | Before | After |
|-----------|--------|-------|
| Kokoro singleton | Returns dead instance forever after terminal error
| Auto-detects and re-creates |
| `voices` return | `voices!` can return null, lying to type system |
Explicit throw, enters error recovery path |
| Whisper Worker concurrency | Silent drop, Promise hangs 120s |
Immediate error response |
| Event listeners | Anonymous, never cleaned up | Stored references,
removeEventListener on destroy |
| Error classification | 4/6 codes reachable, recoverable hardcoded
false | 6/6 codes reachable, dynamic recoverability |
| dtype loading | Single attempt, failure is terminal | Progressive
fallback chain, reports actual config used |
---
## Files changed
| File | Changes |
|------|---------|
| `libs/inference/protocol.ts` | `classifyError` phase param, new
`isRecoverable` helper |
| `libs/inference/protocol.test.ts` | **New** -- 14 tests for
classification and recoverability |
| `libs/workers/worker.ts` | Silent drop fix, sendError phase, fp16
fallback |
| `workers/kokoro/worker.ts` | sendError phase, dtype/device fallback
chain |
| `workers/background-removal/worker.ts` | sendError phase |
| `libs/inference/adapters/whisper.ts` | Listener cleanup,
terminateAdapter uses destroyWorker, scheduleRestart terminal state |
| `libs/inference/adapters/kokoro.ts` | Singleton recovery, voices
assertion fix, listener cleanup, scheduleRestart terminal state |
| `libs/inference/adapters/kokoro.test.ts` | **New** -- 5 tests for
singleton and state transitions |
| `libs/inference/adapters/background-removal.ts` | Listener cleanup,
extracted destroyWorker |
## Test plan
- [x] `pnpm exec vitest run packages/stage-ui/src/libs/inference/` --
30/30 tests pass (protocol, kokoro adapter, GPU coordinator, load queue)
- [x] `pnpm -F @proj-airi/stage-ui typecheck` -- clean
- [x] `npx eslint` on all 9 changed files -- clean
- [x] Pre-commit hooks pass
Co-authored-by-agent: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
## Summary
Address maintainer feedback from #1622 ([@nekomeowww's
comment](https://github.com/moeru-ai/airi/pull/1622#discussion_r2342632282)).
The project already uses `async-mutex` in other packages
(`stage-tamagotchi`, `electron-screen-capture`), and it's in the
workspace catalog. This replaces the custom `AsyncMutex` implementation
with the shared dependency.
- Replace `AsyncMutex.run()` → `Mutex.runExclusive()`
- Replace `AsyncMutex.reset()` → `Mutex.cancel()`
- Remove `async-mutex.ts` and its unit tests
- Remove `AsyncMutex` from inference barrel exports
- Add `async-mutex` as direct dependency of `@proj-airi/stage-ui`
**Regarding `@moeru/eventa` suggestion**
([comment](https://github.com/moeru-ai/airi/pull/1622#discussion_r2342631692)):
The `waitForMessage` pattern is a thin request-response abstraction over
`postMessage`, where the worker is driven by `@huggingface/transformers`
internally. Eventa's transport-agnostic RPC is better suited for
bidirectional channels (Electron IPC, WebSocket) than for this one-way
"post and wait" pattern. No change for now.
## Test plan
- [x] `pnpm exec vitest run packages/stage-ui/src/libs/inference/` — 11
tests pass (4 removed with custom impl)
- [x] `pnpm -F @proj-airi/stage-ui typecheck` — no errors
- [x] `pnpm lint:fix` — no new errors in changed files
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
## Problem
The WebGPU inference pipeline had several structural issues:
1. **No unified protocol** — Kokoro TTS, Whisper ASR, and Background
Removal workers each used their own ad-hoc message formats. Adding a new
model meant reinventing worker communication from scratch.
2. **Infrastructure existed but was disconnected** —
`GPUResourceCoordinator`, `LoadQueue`, `InferenceWorkerManager`, and
`protocol.ts` were all implemented but had zero consumers. The adapters
duplicated the same lifecycle/timeout/mutex patterns independently.
3. **Performance gaps** — Kokoro only offered fp32 on WebGPU (no fp16),
Whisper warm-up compiled shaders for 187.5s of dummy audio, audio
transfer went through unnecessary WAV blob encode/decode, and
`listVoices` reloaded the model every time.
4. **Silent failures** — Whisper worker's `generate()` had no try-catch;
errors were swallowed and the main thread waited until timeout.
5. **No graceful degradation** — Whisper and Background Removal workers
hardcoded `device: 'webgpu'` with no WASM fallback.
6. **No observability** — Only Kokoro had performance tracing. No
adapter reported status to `useInferenceStatus`. No cache management UI
existed.
7. **Dead code accumulation** — Old `KokoroWorkerManager` (232 lines),
legacy Whisper message types, and scattered duplicate constants.
## Changes
### Phase 0 — Critical Performance & Bugs
- Add `fp16-webgpu` dtype for Kokoro TTS (~2x inference speed on
supported GPUs)
- Fix Whisper warm-up tensor from `[1, 128, 3000]` → `[1, 128, 1]`
(minimal shader compilation)
- Fix Whisper worker silent error bug (add try-catch to `generate()` and
`load()`)
### Phase 1 — Data Transfer & Caching
- Switch Kokoro audio to Float32Array transferable (skip WAV blob encode
in worker, lightweight WAV encode on main thread)
- Cache `listVoices` results (skip redundant model reload when adapter
state is `ready`)
- Normalize progress reporting to 0-100 across all adapters,
differentiate `warmup` phase
### Phase 2 — Protocol Unification & Infrastructure
- Migrate all 3 workers + 3 adapters to unified `protocol.ts` message
types (`load-model`, `run-inference`, `model-ready`, `inference-result`,
`progress`, `error`)
- Wire `GPUResourceCoordinator` into all adapters (VRAM allocation
tracking, LRU ordering, memory pressure events)
- Wire `LoadQueue` into all adapters (priority-based sequential model
loading: TTS=10 > ASR=5 > BG_REMOVAL=1)
- Add `coordinator.ts` global singleton for GPU coordinator + load queue
- Add WebGPU detection + WASM fallback in Whisper and Background Removal
workers
### Phase 3 — Error Recovery & Observability
- Add restart logic with exponential backoff to Whisper adapter
(matching Kokoro's existing pattern)
- Integrate `classifyError()` (OOM / DEVICE_LOST / TIMEOUT
classification) in Whisper adapter
- Extend `defaultPerfTracer` to Whisper `transcribe()` and Background
Removal `processImage()`
- Wire `useInferenceStatus` into all 3 adapters (downloading → ready →
terminated lifecycle)
### Phase 4 — Tests
- Add unit tests for `AsyncMutex` (4 tests), `LoadQueue` (4 tests),
`GPUResourceCoordinator` (7 tests) — all 15 passing
### Phase 5 — Cleanup & Features
- Delete old `KokoroWorkerManager` (232 lines, zero consumers)
- Delete orphaned `libs/workers/types.ts` (old Whisper message types)
- Clean up `workers/kokoro/types.ts` (remove legacy message types, keep
domain types)
- Create centralized `constants.ts` (MODEL_IDS, MODEL_NAMES, TIMEOUTS,
MAX_RESTARTS)
- Remove hardcoded WebGPU check from background-removal devtools pages
(worker auto-detects)
- Add `useModelPreload` composable for generic idle-time preloading
- Add `useInferencePreload` composable that reads provider config and
preloads configured local models
- Wire preloading into both `stage-web` and `stage-tamagotchi` App.vue
(Kokoro TTS preloads 3s after init)
- Add `ModelCacheManager.vue` settings component (cache size display,
per-model status, clear cache)
- Document GPU Device isolation architecture in protocol.ts
## After This PR
- All inference workers speak the same protocol → adding a new model
adapter is straightforward
- GPU memory is tracked across all models with automatic pressure
warnings at 80%/95% of VRAM budget
- Models load sequentially via priority queue → no bandwidth/VRAM
contention
- Workers auto-detect WebGPU and fall back to WASM → works on browsers
without WebGPU
- Kokoro TTS preloads during idle time → "instant" first use for
configured users
- All adapters auto-restart on worker crashes (max 3 attempts,
exponential backoff)
- 15 unit tests cover core infrastructure (mutex, queue, coordinator)
- Zero dead code remains in the inference pipeline
## Test Plan
- [x] `pnpm exec vitest run packages/stage-ui/src/libs/inference/` — 15
tests pass
- [x] `pnpm -F @proj-airi/stage-ui exec tsc --noEmit` — no TypeScript
errors
- [x] `pnpm lint:fix` — no lint errors in changed files
- [ ] Manual: verify Kokoro TTS works with fp16-webgpu on a supported
browser
- [ ] Manual: verify Whisper ASR loads and transcribes correctly
- [ ] Manual: verify Background Removal works in devtools page
- [ ] Manual: verify preloading triggers in console (`[Preload] Loading
kokoro-...`)
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
NJX