Track per-app TTS concurrency in Redis, route capped upstreams by
available pool capacity, and surface pool saturation metrics. Document
the Voice Pack plan so the remaining backend and card-binding work has
an explicit implementation map.
## Summary
- Fix the first Live2D -> VRM switch camera regression where
OrbitControls could clamp the camera into the model.
- Remove the VRM render scale control from the model settings panel.
- Remove the now-unused `settings.vrm.render-scale` locale entries.
- Ignore Godot/.NET `bin/` and `obj/` build artifacts under
`engines/stage-tamagotchi-godot`.
## Root Cause
On the first VRM load, `OrbitControls` can mount before the VRM
bootstrap writes a real `modelSize`. The previous watcher copied
`modelSize.z = 0` into `minDistance` and `maxDistance`, so later camera
updates were clamped onto the target before the bootstrap camera pose
finished applying.
## Changes
- Skip OrbitControls distance-bound updates until `modelSize.z` is
finite and non-zero.
- Keep the render scale runtime state intact, but remove the VRM
settings UI entry so it is no longer hot-updated from the panel.
- Add Godot project ignores for local .NET build output.
---------
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
The streaming speech provider settings page was seeding its model
picker with a hardcoded `volcengine/seed-tts-2.0`, which contradicts
the just-landed "streaming model catalog comes from the server"
contract. Now the default also comes from configKV.
- `UNSPEECH_UPSTREAM.streaming` gains `defaultModel?: string`.
- `GET /api/v1/audio/models/streaming` response gains `default: string
| null` reading that field.
- admin slice apply: `streaming.defaultModel` survives key/URL
rotation alongside `streaming.models`.
- `providerOfficialSpeechStreaming.listModels` stashes the value in a
module-scope ref, exposed via `getDefaultStreamingModel()`.
- Streaming speech settings page reads the helper instead of the
hardcoded id; falls back to the first server-returned model if the
operator hasn't curated a default. `handleGenerateSpeech` no longer
has a `seed-tts-2.0` backstop — a missing backend prefix in the
selected model id now throws instead of silently defaulting.
## Summary
This PR adds the first usable camera and view-state loop for the
experimental Godot stage.
The main design direction is that Godot owns the stage view state, while
Electron and the settings page act as lifecycle/transport/control
surfaces. The settings UI can now edit Godot camera state, and camera
changes from the Godot window can flow back to the UI.
## What Changed
- Added the shared Godot view-state contract for camera pose, avatar
bounds, snapshots, patches, and errors.
- Added Electron main bridge APIs for Godot status, scene input, view
snapshots, and view patches.
- Added Godot-side camera runtime:
- initial camera bootstrap from loaded avatar bounds
- avatar facing normalisation
- orbit, pan, wheel movement, and WASD-style camera movement
- idle persistence for camera position, yaw, pitch, and FOV
- Added settings-page Godot camera controls with coalesced updates,
restart-safe session handling, and stale request protection.
- Kept scene input simple: Electron does not cache the selected model
for replay. Each Godot start receives the current model selection from
the renderer.
## File Guide
Shared contracts:
- `packages/stage-shared/src/godot-stage/view-state.ts` defines the
Godot view-state contract and schema validation.
- `packages/stage-shared/src/godot-stage/index.ts` exports the Godot
shared contract surface.
- `packages/stage-shared/package.json` exposes the new shared entry.
Electron / renderer bridge:
- `apps/stage-tamagotchi/src/shared/eventa/index.ts` adds the Eventa
contracts for Godot status, scene input, view snapshot, patch, and error
events.
- `apps/stage-tamagotchi/src/main/services/airi/godot-stage/index.ts`
owns the Godot sidecar lifecycle, local WebSocket bridge, scene input
forwarding, and view-state forwarding.
Settings page:
- `apps/stage-tamagotchi/src/renderer/pages/settings/models/index.vue`
wires the settings page to Godot status, scene input, snapshots, and
patches.
-
`apps/stage-tamagotchi/src/renderer/pages/settings/models/godot-view-patch-queue.ts`
coalesces high-frequency camera edits before sending them to Godot.
-
`apps/stage-tamagotchi/src/renderer/pages/settings/models/godot-view-session.ts`
tracks renderer-local Godot session epochs so stale async results are
ignored.
Shared settings UI:
-
`packages/stage-ui/src/components/scenarios/settings/model-settings/godot.vue`
renders the Godot camera controls.
-
`packages/stage-ui/src/components/scenarios/settings/model-settings/panel.vue`
routes the model settings panel between Live2D, VRM, and Godot settings.
-
`packages/stage-ui/src/components/scenarios/settings/model-settings/runtime.ts`
resolves model-settings runtime display state and Godot camera control
range.
- `packages/stage-ui/src/components/data-pane/property-number.vue`
updates number control behavior so config changes such as min/max/range
are reflected correctly.
Godot runtime:
- `engines/stage-tamagotchi-godot/scripts/StageRoot.cs` wires scene
loading, view runtime, and transport together.
- `engines/stage-tamagotchi-godot/scripts/scene/StageSceneController.cs`
loads VRM avatars and normalizes avatar facing direction.
- `engines/stage-tamagotchi-godot/scripts/view/StageViewRuntime.cs` owns
Godot view-state updates, snapshots, patch application, and idle
persistence.
-
`engines/stage-tamagotchi-godot/scripts/view/StageCameraPoseController.cs`
computes avatar bounds, bootstraps the camera, applies camera pose, and
handles camera movement math.
-
`engines/stage-tamagotchi-godot/scripts/view/StageCameraInputController.cs`
maps mouse and keyboard input into camera operations.
- `engines/stage-tamagotchi-godot/scripts/view/StageViewStateRules.cs`
centralizes view-state defaults and validation rules.
- `engines/stage-tamagotchi-godot/scripts/view/StageViewStateStore.cs`
persists and loads Godot camera view state.
- `engines/stage-tamagotchi-godot/scripts/transport/StageViewJson.cs`
parses and serializes view-state transport payloads.
-
`engines/stage-tamagotchi-godot/scripts/transport/StageViewPayloads.cs`
defines the C# payload records used by the transport layer.
---------
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Frontend voice picker silently lost Azure voices because /audio/voices
was called without ?model=, so the server fell back to DEFAULT_TTS_MODEL
and only ever returned one model's catalog. /audio/models also hardcoded
a single `auto` alias, so the client could never request a specific
model id even if it wanted to. On top of that, the Azure adapter shipped
only 2 hand-typed voices in voices/azure.json — the rest of Microsoft's
catalog was unreachable.
Fixed in one pass:
- Drop voices/*.json. Every getVoiceCatalog now calls unspeech REST
(backend=microsoft|alibaba|volcengine). Azure proxies Microsoft's live
voices/list; cosyvoice + volcengine use unspeech's embedded catalogs.
- Drop direct upstream provider calls from send(). All three adapters
POST to <UNSPEECH_REST_BASE_URL>/v1/audio/speech with the openai-shaped
body unspeech expects (model: `<backend>/<id>`). Azure SSML still built
on our side so <prosody rate> speed survives — unspeech accepts
pre-built <speak> documents.
- Fail-fast on every voices endpoint: empty ?model= → 400 MISSING_MODEL
(no implicit DEFAULT_TTS_MODEL fallback), only `auto` resolves;
streaming upstream missing → 503; unspeech non-2xx / network err → 502;
malformed body → 502 with grepable message. No silent `{voices: []}`.
- /audio/models lists every LLM_ROUTER_CONFIG.tts.models key + `auto`.
- Frontend providerOfficialSpeech.listVoices passes ?model= and throws
on upstream failure instead of returning [].
- New UNSPEECH_REST_BASE_URL configKV entry (no default, missing → 503).
config-sync subscriber invalidates the voice cache when this key OR
LLM_ROUTER_CONFIG changes.
- Router voice catalog Redis cache covers all providers with per-provider
TTL (azure 6h live, alibaba/volcengine 24h since unspeech embeds them
at build time).
Pending billing-SKU verification: DashScope cosyvoice now goes through
unspeech's WS-internal alibaba backend (was REST `/SpeechSynthesizer`).
Functional output equivalent — confirm bill after deploy, revert via
git history if WS pricing is materially higher.
Streaming TTS model list was hardcoded in the official provider with two
Volcengine ids. This moves it server-side so operators control the
catalog without a client release.
Catalog lives on `STREAMING_TTS_UPSTREAM.models` (same configKV entry as
the ws upstream + key envelope) rather than a separate kv key — connection
target, keys, and surfaced ids are one deployment decision, splitting
them risks drift on rotation. `LLM_ROUTER_CONFIG.tts.models` is the
router/fallback domain for HTTP TTS; streaming is single-ws and has no
router semantics, so it gets its own `streamingTtsUpstreamSchema`
instead of reusing `ttsUpstreamSchema`.
New `GET /api/v1/audio/models/streaming` returns the configured list;
empty when upstream or models is unset (UI renders "no models" instead
of 5xx). Admin slice apply preserves existing `models` across key/connection
rotation so admin POSTs that only carry upstream+keys do not wipe the
catalog. Frontend `providerOfficialSpeechStreaming.listModels` fetches
the endpoint and throws on upstream errors (no silent empty array).
Why:
- Add a real bidirectional streaming TTS path: raw LLM tokens are
forwarded to the upstream model (Volcengine v3 via the unspeech ws
bridge) without client-side segmentation, so the model owns sentence
splitting and audio chunks play as they arrive.
- Move audio endpoints out of /api/v1/openai/. `/audio/voices`,
`/audio/models`, `/audio/voices/streaming` are not real OpenAI public
APIs, and the streaming TTS surface has nothing to do with OpenAI —
keeping them under /openai/ mislabelled the contract.
- Introduce `capabilities.speech.transport` on ProviderDefinition so
future streaming providers (ElevenLabs / Cartesia / OpenAI Realtime)
opt in without touching Stage.vue or the session factory.
- Unify Stage.vue's TTS path through a single StageTtsSession so the
chat-orchestrator hooks no longer branch on provider id.
What:
- apps/server: new ws proxy /api/v1/audio/speech/ws bridges client ↔
unspeech with auth, pre-flight flux check, billing from upstream
session.finished.usage, OTel spans.
- apps/server: audio routes moved from /api/v1/openai/audio/* to
/api/v1/audio/* (hard cutover; 404 sentinel tests added).
- apps/server: new /api/v1/audio/voices/streaming proxy reads voices
from unspeech /api/voices?provider=volcengine.
- apps/server: new STREAMING_TTS_UPSTREAM configKV entry +
scripts/seed-streaming-tts.ts.
- stage-ui: new libs/speech/streaming-pipeline.ts opens one ws per LLM
intent (appendText / finish / cancel + onSentence / onError / onDone).
- stage-ui: new libs/speech/tts-session.ts — StageTtsSession interface
with segmenter and streaming adapters; factory dispatches by
capabilities.speech.transport instead of hard-coded provider id.
- stage-ui: providerOfficialSpeechStreaming with capabilities.speech =
{ transport: 'bidirectional-ws' }; settings page with model/voice
picker + ws-based preview.
- stage-ui: Stage.vue chat hooks collapsed to a single currentSession;
hot-swap watcher cancels mid-session on provider/voice/model change;
unmount cancels and drains playback.
Tests:
- 9 streaming-pipeline tests (happy path / buffered / error / cancel /
truncation)
- 11 tts-session tests (factory branch coverage + adapter contracts)
- 4 audio-speech-ws route tests (forwarding / billing / pre-flight /
config-missing)
- 3 legacy-path 404 sentinels in v1 route tests
- Verification doc updated to reflect automated coverage.
- Added a new PostHog client for capturing server-side business events such as Stripe webhooks and subscription state changes.
- Implemented various tracking functions for pricing funnel steps, character creation, and chat session starts.
- Enhanced the flux meter tests to handle partial charges and report unbilled flux correctly.
- Updated the CharacterDialog and Flux settings pages to track user interactions with analytics events.
- Introduced a mechanism to identify users on PostHog based on authentication state to ensure accurate funnel tracking.
- Added necessary dependencies for PostHog integration in the project.
RainbowBird