Frontend voice picker silently lost Azure voices because /audio/voices
was called without ?model=, so the server fell back to DEFAULT_TTS_MODEL
and only ever returned one model's catalog. /audio/models also hardcoded
a single `auto` alias, so the client could never request a specific
model id even if it wanted to. On top of that, the Azure adapter shipped
only 2 hand-typed voices in voices/azure.json — the rest of Microsoft's
catalog was unreachable.
Fixed in one pass:
- Drop voices/*.json. Every getVoiceCatalog now calls unspeech REST
(backend=microsoft|alibaba|volcengine). Azure proxies Microsoft's live
voices/list; cosyvoice + volcengine use unspeech's embedded catalogs.
- Drop direct upstream provider calls from send(). All three adapters
POST to <UNSPEECH_REST_BASE_URL>/v1/audio/speech with the openai-shaped
body unspeech expects (model: `<backend>/<id>`). Azure SSML still built
on our side so <prosody rate> speed survives — unspeech accepts
pre-built <speak> documents.
- Fail-fast on every voices endpoint: empty ?model= → 400 MISSING_MODEL
(no implicit DEFAULT_TTS_MODEL fallback), only `auto` resolves;
streaming upstream missing → 503; unspeech non-2xx / network err → 502;
malformed body → 502 with grepable message. No silent `{voices: []}`.
- /audio/models lists every LLM_ROUTER_CONFIG.tts.models key + `auto`.
- Frontend providerOfficialSpeech.listVoices passes ?model= and throws
on upstream failure instead of returning [].
- New UNSPEECH_REST_BASE_URL configKV entry (no default, missing → 503).
config-sync subscriber invalidates the voice cache when this key OR
LLM_ROUTER_CONFIG changes.
- Router voice catalog Redis cache covers all providers with per-provider
TTL (azure 6h live, alibaba/volcengine 24h since unspeech embeds them
at build time).
Pending billing-SKU verification: DashScope cosyvoice now goes through
unspeech's WS-internal alibaba backend (was REST `/SpeechSynthesizer`).
Functional output equivalent — confirm bill after deploy, revert via
git history if WS pricing is materially higher.
Streaming TTS model list was hardcoded in the official provider with two
Volcengine ids. This moves it server-side so operators control the
catalog without a client release.
Catalog lives on `STREAMING_TTS_UPSTREAM.models` (same configKV entry as
the ws upstream + key envelope) rather than a separate kv key — connection
target, keys, and surfaced ids are one deployment decision, splitting
them risks drift on rotation. `LLM_ROUTER_CONFIG.tts.models` is the
router/fallback domain for HTTP TTS; streaming is single-ws and has no
router semantics, so it gets its own `streamingTtsUpstreamSchema`
instead of reusing `ttsUpstreamSchema`.
New `GET /api/v1/audio/models/streaming` returns the configured list;
empty when upstream or models is unset (UI renders "no models" instead
of 5xx). Admin slice apply preserves existing `models` across key/connection
rotation so admin POSTs that only carry upstream+keys do not wipe the
catalog. Frontend `providerOfficialSpeechStreaming.listModels` fetches
the endpoint and throws on upstream errors (no silent empty array).
Why:
- Add a real bidirectional streaming TTS path: raw LLM tokens are
forwarded to the upstream model (Volcengine v3 via the unspeech ws
bridge) without client-side segmentation, so the model owns sentence
splitting and audio chunks play as they arrive.
- Move audio endpoints out of /api/v1/openai/. `/audio/voices`,
`/audio/models`, `/audio/voices/streaming` are not real OpenAI public
APIs, and the streaming TTS surface has nothing to do with OpenAI —
keeping them under /openai/ mislabelled the contract.
- Introduce `capabilities.speech.transport` on ProviderDefinition so
future streaming providers (ElevenLabs / Cartesia / OpenAI Realtime)
opt in without touching Stage.vue or the session factory.
- Unify Stage.vue's TTS path through a single StageTtsSession so the
chat-orchestrator hooks no longer branch on provider id.
What:
- apps/server: new ws proxy /api/v1/audio/speech/ws bridges client ↔
unspeech with auth, pre-flight flux check, billing from upstream
session.finished.usage, OTel spans.
- apps/server: audio routes moved from /api/v1/openai/audio/* to
/api/v1/audio/* (hard cutover; 404 sentinel tests added).
- apps/server: new /api/v1/audio/voices/streaming proxy reads voices
from unspeech /api/voices?provider=volcengine.
- apps/server: new STREAMING_TTS_UPSTREAM configKV entry +
scripts/seed-streaming-tts.ts.
- stage-ui: new libs/speech/streaming-pipeline.ts opens one ws per LLM
intent (appendText / finish / cancel + onSentence / onError / onDone).
- stage-ui: new libs/speech/tts-session.ts — StageTtsSession interface
with segmenter and streaming adapters; factory dispatches by
capabilities.speech.transport instead of hard-coded provider id.
- stage-ui: providerOfficialSpeechStreaming with capabilities.speech =
{ transport: 'bidirectional-ws' }; settings page with model/voice
picker + ws-based preview.
- stage-ui: Stage.vue chat hooks collapsed to a single currentSession;
hot-swap watcher cancels mid-session on provider/voice/model change;
unmount cancels and drains playback.
Tests:
- 9 streaming-pipeline tests (happy path / buffered / error / cancel /
truncation)
- 11 tts-session tests (factory branch coverage + adapter contracts)
- 4 audio-speech-ws route tests (forwarding / billing / pre-flight /
config-missing)
- 3 legacy-path 404 sentinels in v1 route tests
- Verification doc updated to reflect automated coverage.
- Added a new PostHog client for capturing server-side business events such as Stripe webhooks and subscription state changes.
- Implemented various tracking functions for pricing funnel steps, character creation, and chat session starts.
- Enhanced the flux meter tests to handle partial charges and report unbilled flux correctly.
- Updated the CharacterDialog and Flux settings pages to track user interactions with analytics events.
- Introduced a mechanism to identify users on PostHog based on authentication state to ensure accurate funnel tracking.
- Added necessary dependencies for PostHog integration in the project.
Neko Ayaka