Track per-app TTS concurrency in Redis, route capped upstreams by
available pool capacity, and surface pool saturation metrics. Document
the Voice Pack plan so the remaining backend and card-binding work has
an explicit implementation map.
The streaming speech provider settings page was seeding its model
picker with a hardcoded `volcengine/seed-tts-2.0`, which contradicts
the just-landed "streaming model catalog comes from the server"
contract. Now the default also comes from configKV.
- `UNSPEECH_UPSTREAM.streaming` gains `defaultModel?: string`.
- `GET /api/v1/audio/models/streaming` response gains `default: string
| null` reading that field.
- admin slice apply: `streaming.defaultModel` survives key/URL
rotation alongside `streaming.models`.
- `providerOfficialSpeechStreaming.listModels` stashes the value in a
module-scope ref, exposed via `getDefaultStreamingModel()`.
- Streaming speech settings page reads the helper instead of the
hardcoded id; falls back to the first server-returned model if the
operator hasn't curated a default. `handleGenerateSpeech` no longer
has a `seed-tts-2.0` backstop — a missing backend prefix in the
selected model id now throws instead of silently defaulting.
One unspeech deployment = one configKV entry. The split into two keys
modelled a hypothetical split deployment (operators pointing REST and
WS at different unspeech instances) that nobody actually runs, and the
explanatory comments justifying the split were exactly the migration-
narrative anti-pattern we just banned from source.
Schema:
UNSPEECH_UPSTREAM = {
restBaseURL: string, // required, http(s)://host:port
streaming?: { // optional, only when ws TTS is on
baseURL: string, // ws(s)://host:port/...
keys: [{ id, ciphertext }],
adapterParams: {},
models?: [{ id, name?, description? }],
},
}
Admin slice surface flattens to one `kind: 'unspeech'`:
{ kind: 'unspeech', restBaseURL,
streaming?: { upstreamURL, plaintextKey, keyEntryId? } }
Read-site changes:
- routeTts + listTtsVoices read UNSPEECH_UPSTREAM.restBaseURL via
getOrThrow; absent entry → 503 CONFIG_NOT_SET.
- audio-speech-ws dials UNSPEECH_UPSTREAM.streaming and 1008-closes
with streaming_tts_not_configured when the subtree is absent.
- handleListStreamingVoices reads .restBaseURL directly (no more
ws→http scheme swap) and 503s on missing streaming subtree.
- handleListStreamingTTSModels reads .streaming.models.
- config-sync-subscriber listens for UNSPEECH_UPSTREAM invalidations
instead of UNSPEECH_REST_BASE_URL.
Drive-by: tighten ConfigKVService.getOrThrow return to
`Exclude<ConfigDefinitions[K], undefined>` so call sites stop needing
non-null assertions on optional schema entries — the runtime already
throws, the type now reflects it.
Migration: operators repost a single admin slice to rewrite the merged
entry. Old STREAMING_TTS_UPSTREAM and UNSPEECH_REST_BASE_URL rows
become dead data and can be deleted from configKV; nothing reads them.
Frontend voice picker silently lost Azure voices because /audio/voices
was called without ?model=, so the server fell back to DEFAULT_TTS_MODEL
and only ever returned one model's catalog. /audio/models also hardcoded
a single `auto` alias, so the client could never request a specific
model id even if it wanted to. On top of that, the Azure adapter shipped
only 2 hand-typed voices in voices/azure.json — the rest of Microsoft's
catalog was unreachable.
Fixed in one pass:
- Drop voices/*.json. Every getVoiceCatalog now calls unspeech REST
(backend=microsoft|alibaba|volcengine). Azure proxies Microsoft's live
voices/list; cosyvoice + volcengine use unspeech's embedded catalogs.
- Drop direct upstream provider calls from send(). All three adapters
POST to <UNSPEECH_REST_BASE_URL>/v1/audio/speech with the openai-shaped
body unspeech expects (model: `<backend>/<id>`). Azure SSML still built
on our side so <prosody rate> speed survives — unspeech accepts
pre-built <speak> documents.
- Fail-fast on every voices endpoint: empty ?model= → 400 MISSING_MODEL
(no implicit DEFAULT_TTS_MODEL fallback), only `auto` resolves;
streaming upstream missing → 503; unspeech non-2xx / network err → 502;
malformed body → 502 with grepable message. No silent `{voices: []}`.
- /audio/models lists every LLM_ROUTER_CONFIG.tts.models key + `auto`.
- Frontend providerOfficialSpeech.listVoices passes ?model= and throws
on upstream failure instead of returning [].
- New UNSPEECH_REST_BASE_URL configKV entry (no default, missing → 503).
config-sync subscriber invalidates the voice cache when this key OR
LLM_ROUTER_CONFIG changes.
- Router voice catalog Redis cache covers all providers with per-provider
TTL (azure 6h live, alibaba/volcengine 24h since unspeech embeds them
at build time).
Pending billing-SKU verification: DashScope cosyvoice now goes through
unspeech's WS-internal alibaba backend (was REST `/SpeechSynthesizer`).
Functional output equivalent — confirm bill after deploy, revert via
git history if WS pricing is materially higher.
Streaming TTS model list was hardcoded in the official provider with two
Volcengine ids. This moves it server-side so operators control the
catalog without a client release.
Catalog lives on `STREAMING_TTS_UPSTREAM.models` (same configKV entry as
the ws upstream + key envelope) rather than a separate kv key — connection
target, keys, and surfaced ids are one deployment decision, splitting
them risks drift on rotation. `LLM_ROUTER_CONFIG.tts.models` is the
router/fallback domain for HTTP TTS; streaming is single-ws and has no
router semantics, so it gets its own `streamingTtsUpstreamSchema`
instead of reusing `ttsUpstreamSchema`.
New `GET /api/v1/audio/models/streaming` returns the configured list;
empty when upstream or models is unset (UI renders "no models" instead
of 5xx). Admin slice apply preserves existing `models` across key/connection
rotation so admin POSTs that only carry upstream+keys do not wipe the
catalog. Frontend `providerOfficialSpeechStreaming.listModels` fetches
the endpoint and throws on upstream errors (no silent empty array).
- services/domain/llm-router and services/domain/user-deletion still had
colocated *.test.ts; moved them into the local tests/ subdir so every
service module follows the same layout as billing/, admin/*/, etc.
- routes/oidc/ wasn't an independent route group — it's wholly consumed
by routes/auth/index.ts (electron-callback relay + OIDC bearer token
helper). Moved to routes/auth/oidc/ to reflect that ownership.
- Fixed the relative imports in src/libs/tests/{auth,env,request-auth,
ws-auth}.test.ts. They were moved to libs/tests/ in the previous
commit without updating the './<source>' specifiers; typecheck only
caught it once the rewriter ran across the tree.
pnpm typecheck passes; eslint matches the main baseline.
Why
- src/services/ was an unordered mix of single-file services and module
directories with no shared classification axis, plus several long-dead
admin batch helpers that survived the move to the simpler synchronous
admin-flux-grants flow.
What
- services/ now has two top-level layers:
domain/ — DB state + business rules (billing, characters, chats,
flux, flux-transaction, llm-router, providers, request-log,
stripe, user-deletion, admin/{flux-grants,router-config})
adapters/ — thin wrappers over external SDKs / infra (config-kv, email,
posthog, tts/)
- admin/* moved under domain/admin/ with consistent plural names
(flux-grants, router-config).
- tts-adapters/ collapsed to adapters/tts/ (no redundant -adapters suffix
once nested under adapters/).
- 63 src files + scripts/e2e-llm-router.ts + tests/verifications/_harness.ts
had relative imports rewritten; git mv preserves blame.
- apps/server/CLAUDE.md and docs/ai-context/*.md updated to match new paths.
Dead code removed
- services/admin-flux-grant-batches/ (service + worker + tests, 1090 LOC) —
superseded by admin-flux-grants and never wired into app.ts.
- routes/admin/flux-grant-batches/ — same.
- utils/redis-compressed.ts + test — zero production call sites.
- llm-router/index.ts re-exports trimmed from 26 to 6; only symbols with
external consumers are kept.
Intentionally kept
- schemas/flux-grant-batch.ts and its schemas/index.ts export remain so the
drizzle-kit generate diff stays empty. Removing them is a separate PR
that owns the drop-table migration for flux_grant_batch /
flux_grant_batch_recipient.
Verification
- pnpm -F @proj-airi/server typecheck: passes.
- pnpm exec eslint apps/server: 49 errors, identical to main baseline
(all are pre-existing node/prefer-global/buffer in envelope-crypto and
scripts/e2e-llm-router; untouched by this change).
- Vitest passes per-file; the 6 mockDB hook timeouts under full-parallel
run are the known pushSchema-per-worker infra cost, not a regression.
Replaces routine use of `scripts/seed-router-config.ts` and
`scripts/seed-streaming-tts.ts` with `POST /api/admin/config/router`.
Operators can now patch one provider at a time without shelling into
the Railway runner; the seed scripts stay as break-glass tools for
cold-boot and disaster recovery.
The endpoint accepts a discriminated-union slice list (openrouter /
azure / dashscope-cosyvoice / streaming-tts), envelope-encrypts
plaintext keys in-process (never echoed back), and supports
merge/reset modes plus dryRun. Writes go through the existing
configKV + Redis `configkv:invalidate` channel so multi-instance
deployments pick up changes within the pub/sub propagation window.
Guarded by the existing `authGuard + adminGuard` pair
(`ADMIN_EMAILS` allowlist + verified email).
When `mapUpstreamError` produced the final 502/503/504, it only carried
`{triedKeys, triedUpstreams, lastStatusCode}` in `details`. The upstream
response body was `.cancel()`'d on the wire and the network error
message vanished into the catch arm — operators staring at a 502 had no
way to tell "OpenRouter region-blocked us" from "key revoked" from
"DNS failed" without re-probing the upstream by hand.
Now each recorded failure carries the diagnostic snippet:
- chat upstreams read at most 256 bytes of the failed body via a
drain-aware reader before cancelling the rest (socket still returns
to the pool, no fallback-storm pool exhaustion).
- TTS upstreams reuse `errorMessageFrom(err)` — adapters already bake
the status + body snippet into `err.message`, so one field carries
both.
- network / timeout attempts record `errorMessageFromUnknown(err)` so
"attempt-timeout" vs "ECONNRESET" vs "DNS failed" stays
distinguishable.
The collected `UpstreamAttempt[]` is attached to `ApiError.cause`
rather than `details`. SEC-5 (no upstream content in client-facing
response body) still holds — only the server-side logger + OTel pick
the cause up. `app.onError` now logs `{details, cause}` together so a
single log line tells the operator both the contract-level summary
and the actual upstream payload.
Adds a router.test.ts regression covering both shapes (HTTP 401 body
snippet + network ECONNRESET errorMessage), with an explicit assertion
that `details` does NOT contain the body text so SEC-5 doesn't drift.
DashScope dropped cosyvoice-v1 from its REST-supported model list. v2
(and v3+) speak a different shape: voice / format / sample_rate live
under `input`, not `parameters`; non-streaming responses return
`output.audio.url` (signed OSS URL) instead of inline `output.audio.data`
base64. The previous adapter sent v1-shaped bodies to a bare
`https://dashscope-intl.aliyuncs.com/api/v1` baseURL and parsed
`audio.data`, which 404'd before the migration and would 200-with-no-
audio after — both invisible regressions for the gateway.
Adapter changes:
- Rewrite request body to v2 schema (voice/format under input).
- Add follow-up GET against `output.audio.url`; stream into ArrayBuffer
with a 25 MB hard cap and explicit drain-tracking finally, so a
misbehaving URL cannot exhaust memory and a half-read body cannot
hang a connection.
- Re-document baseURL contract: adapters do NOT append path; ops must
configure the FULL endpoint URL (root cause of the original 404
storm). DEFAULT_COSYVOICE_MODEL bumped to `cosyvoice-v2`, default
voice to `longxiaochun_v2`.
Voice catalog: regenerated with 19 representative cosyvoice-v2 voices
(assistant / customer-service / child / en-US / en-GB / ja-JP / ko-KR)
so the frontend voice picker is no longer a 2-entry stub. Full catalog
(100+) remains on the Alibaba docs page — we'll sync on demand rather
than scrape.
Seed script: `--dashscope-region intl|cn` (default `intl`),
`--dashscope-upstream-model cosyvoice-v2`, baseURL now resolves to
`https://<host>/api/v1/services/audio/tts/SpeechSynthesizer` so a
mis-typed region or path cannot reintroduce the 404.
Tests: new dashscope-cosyvoice.test.ts covers v2 body shape (asserts
`parameters` absent — regression), audio.url follow-up fetch, 401
propagation with `.status`, empty-envelope falling back into the
router's recoverable-error path, and catalog freshness (no leftover v1
ids). Verified locally against the staging DashScope key: 200 +
playable mp3 end to end.
End-state of the multi-step KTD-5 / KTD-6 / U8 work. The knoway sidecar
is no longer reachable from server code; the router is required at boot
and now owns chat completions, TTS synthesis, and voice catalog listing.
Highlights:
- LLM_ROUTER_MASTER_KEY becomes required; app.ts drops the graceful-
skip branch and the chat fallback fetch path is gone.
- /audio/speech and /audio/voices route through new routeTts /
listTtsVoices entries that reuse the chat key-rotator + per-attempt
timeout + abort propagation.
- DEFAULT_CHAT_MODEL / DEFAULT_TTS_MODEL move from env to configKV so
default-model swaps are hot-reloadable via Pub/Sub.
- GATEWAY_BASE_URL removed from env schema, .env, .env.local, smoke,
verification harness. Redis upstream-voices cache deleted — catalogs
come from in-process adapter JSON.
- routeTts splits adapter error contract by ApiError statusCode:
4xx propagates without fallback; 5xx folds into the network-failure
fallback path. handleTTS wraps billing + span attribute in try/finally
to plug a span leak when ttsMeter.accumulate() throws.
- seed-router-config.ts rewritten with --merge (default) / --reset /
--dry-run modes and env-var key handoff (OPENROUTER_KEY / AZURE_KEY /
DASHSCOPE_KEY) so prod seed flows never put plaintext on the CLI.
Adds DashScope CosyVoice seeding.
Docs (CLAUDE.md, architecture-overview.md, transport-and-routes.md)
reflect the new boundary. verifications/llm-router.md replaces the
overstated "U1-U9 shipped" line with an evidence-vs-pending table.
Tests: full 40-file / 343-case server suite green. New regressions pin
ApiError 4xx → no-fallback, ApiError 5xx → fallback, TTS billing
failure → span closed and error propagated.
- Added a new PostHog client for capturing server-side business events such as Stripe webhooks and subscription state changes.
- Implemented various tracking functions for pricing funnel steps, character creation, and chat session starts.
- Enhanced the flux meter tests to handle partial charges and report unbilled flux correctly.
- Updated the CharacterDialog and Flux settings pages to track user interactions with analytics events.
- Introduced a mechanism to identify users on PostHog based on authentication state to ensure accurate funnel tracking.
- Added necessary dependencies for PostHog integration in the project.
- Moved RateLimitMetrics import path to a more centralized location.
- Introduced a new file for active sessions gauge to track user sessions in the database.
- Updated index.ts to include new metrics and ensure proper initialization of observability metrics.
- Modified various routes and services to utilize the new observability structure.
- Added smoke tests for HTTP and WebSocket metrics to ensure proper metric registration and functionality.
- Enhanced error handling for metrics reading failures to improve observability.
- Added new metrics for email service including send, failures, and duration tracking.
- Introduced rate limit metrics to monitor blocked requests and improve abuse detection.
- Enhanced billing metrics to track credited and unbilled flux, as well as TTS character processing.
- Updated OpenAI and Stripe routes to utilize new metrics for better revenue tracking and rate limiting.
- Implemented a smoke test for OpenTelemetry metrics registration to ensure visibility at startup.
The Redis Stream `billing-events` + `worker` Railway role +
advisory-lock poller layered together didn't actually buy us reliability
— `debitFlux` swallowed XADD failures, leaving the door open to "balance
updated, ledger row never written". Collapse the whole thing back to:
`creditFlux` and `debitFlux` write `flux_transaction` ledger rows inline
within the same DB transaction that mutates `user_flux`, and `(user_id,
request_id)` remains the partial unique index that keeps retries safe.
Concrete changes:
- Inline ledger inserts in `BillingService.{debitFlux, creditFlux,
creditFluxFromStripeCheckout, creditFluxFromInvoice}`; drop `billingMq`
and `publishEvent` plumbing entirely.
- `routes/openai/v1` writes `llm_request_log` synchronously via the
existing `requestLogService`; the duplicate `llm-request-log.ts` service
module is removed.
- `bin/run-worker.ts`, `libs/mq/*`,
`services/billing/billing-events.ts`,
`services/billing/billing-consumer-handler.ts`, and matching tests are
deleted. CLI now exposes only `api`.
- `BILLING_EVENTS_*` env vars and the `DEFAULT_BILLING_EVENTS_STREAM`
helper are dropped; `docker-compose.yml` no longer ships a worker
service.
- `docs/ai-context/{workers-and-runtime, billing-architecture,
redis-boundaries-and-pubsub, data-model-and-state,
architecture-overview, README}.md`, `CLAUDE.md`, and the existing
verification docs are updated to describe the single-process synchronous
pipeline.
Tests: 29 files / 247 cases pass. Production deployments need to drop
the worker Railway service after this lands.
- Added package.json for @proj-airi/server-protocol with necessary configurations.
- Implemented chat event types including WireMessage, SendMessagesRequest, and PullMessagesRequest.
- Defined WebSocket event types and structures for better integration with AIRI components.
- Updated server-runtime and server-sdk to utilize the new server-protocol package.
- Refactored imports across various packages to replace server-shared types with server-protocol types.
- Enhanced type definitions and added TypeScript configurations for better development experience.
RainbowBird