- Removed specific catalog references for '@unocss/core', 'es-toolkit', 'nanoid', and 'valibot' in pnpm-workspace.yaml and various service package.json files.
- Updated versions for '@vueuse/core', 'es-toolkit', 'nanoid', 'reka-ui', and 'valibot' to their latest compatible versions.
- Cleaned up unused catalog entries in pnpm-workspace.yaml.
Move Voice Pack creation and editing to dedicated admin routes, add catalog-backed free-text fields, and wire test audio generation through the public speech API.
Document the mock-API browser verification workflow so future local UI checks can avoid auth and tooling dead ends.
Signed-off-by: RainbowBird <git@luoling.moe>
Add PostHog product events for manual TTS stop clicks and conversation controls across Web, mobile, and Electron chat surfaces. Document the events in metrics ownership and cover the analytics API with Vitest.
Signed-off-by: RainbowBird <git@luoling.moe>
Commit-Message-Assisted-by: Codex
Match the mobile manual TTS stop control to the subtler Web chat button treatment by removing the bright filled circle and using neutral icon styling with primary hover feedback.
Signed-off-by: RainbowBird <git@luoling.moe>
Commit-Message-Assisted-by: Codex
Add the stop-speaking control to the mobile shared input layout and the Electron Tamagotchi chat input so Web/shared, mobile, and Electron all expose the same manual TTS interruption path.
Signed-off-by: RainbowBird <git@luoling.moe>
Commit-Message-Assisted-by: Codex
Track per-app TTS concurrency in Redis, route capped upstreams by
available pool capacity, and surface pool saturation metrics. Document
the Voice Pack plan so the remaining backend and card-binding work has
an explicit implementation map.
Upgrade @moeru/eventa to v1.0.0-beta.8 through the pnpm catalog and remove the local Hono adapter copy.
Keep AIRI on business-level Eventa usage while adapter contract tests live upstream in @moeru/eventa.
Signed-off-by: RainbowBird <git@luoling.moe>
Commit-Message-Assisted-by: Claude (via Claude Code)
Separate OpenAI-compatible route wiring from chat, speech, catalog, billing, and telemetry pipeline code.
Signed-off-by: RainbowBird <git@luoling.moe>
Commit-Message-Assisted-by: Claude (via Claude Code)
The packaged Electron renderer loads from file://, so its Stripe checkout
and portal requests carry no Referer and an opaque/absent Origin.
resolveTrustedRequestOrigin() returned undefined and the routes threw
INVALID_ORIGIN ("Missing trusted request origin"), blocking FLUX purchases
on desktop. Web and mobile were unaffected because they send a trusted web
origin. CORS was not the blocker: file:// requests already reach the API
(balance, providers, characters all work), so only the checkout-specific
redirect-base dependency failed.
Stripe success_url/cancel_url/return_url must be http(s) URLs, so file://
can never be a valid redirect base. Replace the trusted-origin requirement
with resolveCheckoutRedirectBase(), which prefers the request's trusted
origin (web/mobile return to where they started) and falls back to a
configured canonical web app URL (WEB_APP_URL, default https://airi.moeru.ai)
for origin-less clients like the desktop app.
On the client, the desktop app opens checkout in the system browser via
window.open (routed through setWindowOpenHandler -> shell.openExternal)
instead of navigating the file:// window to Stripe, and refreshes the FLUX
balance on window focus since it never receives the success_url redirect.
The streaming speech provider settings page was seeding its model
picker with a hardcoded `volcengine/seed-tts-2.0`, which contradicts
the just-landed "streaming model catalog comes from the server"
contract. Now the default also comes from configKV.
- `UNSPEECH_UPSTREAM.streaming` gains `defaultModel?: string`.
- `GET /api/v1/audio/models/streaming` response gains `default: string
| null` reading that field.
- admin slice apply: `streaming.defaultModel` survives key/URL
rotation alongside `streaming.models`.
- `providerOfficialSpeechStreaming.listModels` stashes the value in a
module-scope ref, exposed via `getDefaultStreamingModel()`.
- Streaming speech settings page reads the helper instead of the
hardcoded id; falls back to the first server-returned model if the
operator hasn't curated a default. `handleGenerateSpeech` no longer
has a `seed-tts-2.0` backstop — a missing backend prefix in the
selected model id now throws instead of silently defaulting.
One unspeech deployment = one configKV entry. The split into two keys
modelled a hypothetical split deployment (operators pointing REST and
WS at different unspeech instances) that nobody actually runs, and the
explanatory comments justifying the split were exactly the migration-
narrative anti-pattern we just banned from source.
Schema:
UNSPEECH_UPSTREAM = {
restBaseURL: string, // required, http(s)://host:port
streaming?: { // optional, only when ws TTS is on
baseURL: string, // ws(s)://host:port/...
keys: [{ id, ciphertext }],
adapterParams: {},
models?: [{ id, name?, description? }],
},
}
Admin slice surface flattens to one `kind: 'unspeech'`:
{ kind: 'unspeech', restBaseURL,
streaming?: { upstreamURL, plaintextKey, keyEntryId? } }
Read-site changes:
- routeTts + listTtsVoices read UNSPEECH_UPSTREAM.restBaseURL via
getOrThrow; absent entry → 503 CONFIG_NOT_SET.
- audio-speech-ws dials UNSPEECH_UPSTREAM.streaming and 1008-closes
with streaming_tts_not_configured when the subtree is absent.
- handleListStreamingVoices reads .restBaseURL directly (no more
ws→http scheme swap) and 503s on missing streaming subtree.
- handleListStreamingTTSModels reads .streaming.models.
- config-sync-subscriber listens for UNSPEECH_UPSTREAM invalidations
instead of UNSPEECH_REST_BASE_URL.
Drive-by: tighten ConfigKVService.getOrThrow return to
`Exclude<ConfigDefinitions[K], undefined>` so call sites stop needing
non-null assertions on optional schema entries — the runtime already
throws, the type now reflects it.
Migration: operators repost a single admin slice to rewrite the merged
entry. Old STREAMING_TTS_UPSTREAM and UNSPEECH_REST_BASE_URL rows
become dead data and can be deleted from configKV; nothing reads them.
Frontend voice picker silently lost Azure voices because /audio/voices
was called without ?model=, so the server fell back to DEFAULT_TTS_MODEL
and only ever returned one model's catalog. /audio/models also hardcoded
a single `auto` alias, so the client could never request a specific
model id even if it wanted to. On top of that, the Azure adapter shipped
only 2 hand-typed voices in voices/azure.json — the rest of Microsoft's
catalog was unreachable.
Fixed in one pass:
- Drop voices/*.json. Every getVoiceCatalog now calls unspeech REST
(backend=microsoft|alibaba|volcengine). Azure proxies Microsoft's live
voices/list; cosyvoice + volcengine use unspeech's embedded catalogs.
- Drop direct upstream provider calls from send(). All three adapters
POST to <UNSPEECH_REST_BASE_URL>/v1/audio/speech with the openai-shaped
body unspeech expects (model: `<backend>/<id>`). Azure SSML still built
on our side so <prosody rate> speed survives — unspeech accepts
pre-built <speak> documents.
- Fail-fast on every voices endpoint: empty ?model= → 400 MISSING_MODEL
(no implicit DEFAULT_TTS_MODEL fallback), only `auto` resolves;
streaming upstream missing → 503; unspeech non-2xx / network err → 502;
malformed body → 502 with grepable message. No silent `{voices: []}`.
- /audio/models lists every LLM_ROUTER_CONFIG.tts.models key + `auto`.
- Frontend providerOfficialSpeech.listVoices passes ?model= and throws
on upstream failure instead of returning [].
- New UNSPEECH_REST_BASE_URL configKV entry (no default, missing → 503).
config-sync subscriber invalidates the voice cache when this key OR
LLM_ROUTER_CONFIG changes.
- Router voice catalog Redis cache covers all providers with per-provider
TTL (azure 6h live, alibaba/volcengine 24h since unspeech embeds them
at build time).
Pending billing-SKU verification: DashScope cosyvoice now goes through
unspeech's WS-internal alibaba backend (was REST `/SpeechSynthesizer`).
Functional output equivalent — confirm bill after deploy, revert via
git history if WS pricing is materially higher.
Streaming TTS model list was hardcoded in the official provider with two
Volcengine ids. This moves it server-side so operators control the
catalog without a client release.
Catalog lives on `STREAMING_TTS_UPSTREAM.models` (same configKV entry as
the ws upstream + key envelope) rather than a separate kv key — connection
target, keys, and surfaced ids are one deployment decision, splitting
them risks drift on rotation. `LLM_ROUTER_CONFIG.tts.models` is the
router/fallback domain for HTTP TTS; streaming is single-ws and has no
router semantics, so it gets its own `streamingTtsUpstreamSchema`
instead of reusing `ttsUpstreamSchema`.
New `GET /api/v1/audio/models/streaming` returns the configured list;
empty when upstream or models is unset (UI renders "no models" instead
of 5xx). Admin slice apply preserves existing `models` across key/connection
rotation so admin POSTs that only carry upstream+keys do not wipe the
catalog. Frontend `providerOfficialSpeechStreaming.listModels` fetches
the endpoint and throws on upstream errors (no silent empty array).
- services/domain/llm-router and services/domain/user-deletion still had
colocated *.test.ts; moved them into the local tests/ subdir so every
service module follows the same layout as billing/, admin/*/, etc.
- routes/oidc/ wasn't an independent route group — it's wholly consumed
by routes/auth/index.ts (electron-callback relay + OIDC bearer token
helper). Moved to routes/auth/oidc/ to reflect that ownership.
- Fixed the relative imports in src/libs/tests/{auth,env,request-auth,
ws-auth}.test.ts. They were moved to libs/tests/ in the previous
commit without updating the './<source>' specifiers; typecheck only
caught it once the rewriter ran across the tree.
pnpm typecheck passes; eslint matches the main baseline.
Why
- Code, routes, service, worker, tests, and ai-context references for the
legacy flux_grant_batch flow were removed in the previous commit. The
schema file and the corresponding production tables were intentionally
left for a separate DDL-only PR (this one) so the destructive change is
easy to time and roll back.
What
- Delete src/schemas/flux-grant-batch.ts.
- Drop the re-export line in src/schemas/index.ts.
- drizzle-kit generate produced drizzle/0011_open_unus.sql:
DROP TABLE flux_grant_batch CASCADE;
DROP TABLE flux_grant_batch_recipient CASCADE;
CASCADE removes the 6 associated indexes in one shot.
- docs/ai-context/architecture-overview.md updated: the dead-code
reminder now points at the migration and explains the rollback story.
Also rolls in a pre-existing local move that was sitting uncommitted:
src/libs/{auth,env,request-auth,ws-auth}.test.ts → src/libs/tests/...
(aligning with the libs/tests/eventa-hono-adapter.test.ts placement that
was already on HEAD).
Deployment
- pnpm typecheck: passes.
- DO NOT run pnpm db:push on prod from this branch automatically. The
drop is intentionally a separate operator action that requires picking
a deploy window where no instance is still on an older image that
could try to read flux_grant_batch. Until 0011 is applied to prod the
table sits as an orphaned shell — safe to leave indefinitely.
Why
- src/services/ was an unordered mix of single-file services and module
directories with no shared classification axis, plus several long-dead
admin batch helpers that survived the move to the simpler synchronous
admin-flux-grants flow.
What
- services/ now has two top-level layers:
domain/ — DB state + business rules (billing, characters, chats,
flux, flux-transaction, llm-router, providers, request-log,
stripe, user-deletion, admin/{flux-grants,router-config})
adapters/ — thin wrappers over external SDKs / infra (config-kv, email,
posthog, tts/)
- admin/* moved under domain/admin/ with consistent plural names
(flux-grants, router-config).
- tts-adapters/ collapsed to adapters/tts/ (no redundant -adapters suffix
once nested under adapters/).
- 63 src files + scripts/e2e-llm-router.ts + tests/verifications/_harness.ts
had relative imports rewritten; git mv preserves blame.
- apps/server/CLAUDE.md and docs/ai-context/*.md updated to match new paths.
Dead code removed
- services/admin-flux-grant-batches/ (service + worker + tests, 1090 LOC) —
superseded by admin-flux-grants and never wired into app.ts.
- routes/admin/flux-grant-batches/ — same.
- utils/redis-compressed.ts + test — zero production call sites.
- llm-router/index.ts re-exports trimmed from 26 to 6; only symbols with
external consumers are kept.
Intentionally kept
- schemas/flux-grant-batch.ts and its schemas/index.ts export remain so the
drizzle-kit generate diff stays empty. Removing them is a separate PR
that owns the drop-table migration for flux_grant_batch /
flux_grant_batch_recipient.
Verification
- pnpm -F @proj-airi/server typecheck: passes.
- pnpm exec eslint apps/server: 49 errors, identical to main baseline
(all are pre-existing node/prefer-global/buffer in envelope-crypto and
scripts/e2e-llm-router; untouched by this change).
- Vitest passes per-file; the 6 mockDB hook timeouts under full-parallel
run are the known pushSchema-per-worker infra cost, not a regression.
The two seed scripts are fully superseded by the new admin endpoint
`POST /api/admin/config/router` — same encryption, same configKV
writes, same `configkv:invalidate` publish, plus auth + audit + body
limits. Keeping both code paths created a drift risk on the AAD label
and the merge semantics.
Doc + test fallout:
- `e2e-llm-router.ts` now points readers to the admin endpoint for
the prerequisite seed step.
- `docs/ai-context/verifications/llm-router.md` and
`streaming-tts.md` get curl-based seed instructions; the 2026-05-15
llm-router evidence stays intact with a note that the script it
used has since been removed.
- The U9 follow-up entry in `llm-router.md` flips from "not shipped"
to "partially shipped" — ETag + HMAC publish are still deferred,
so the `config_write` / `config_invalid_hmac` Grafana panels stay
parked.
- Self-edit on the admin route + `app.ts` docstrings to drop the
earlier "scripts stay as break-glass" wording.
Replaces routine use of `scripts/seed-router-config.ts` and
`scripts/seed-streaming-tts.ts` with `POST /api/admin/config/router`.
Operators can now patch one provider at a time without shelling into
the Railway runner; the seed scripts stay as break-glass tools for
cold-boot and disaster recovery.
The endpoint accepts a discriminated-union slice list (openrouter /
azure / dashscope-cosyvoice / streaming-tts), envelope-encrypts
plaintext keys in-process (never echoed back), and supports
merge/reset modes plus dryRun. Writes go through the existing
configKV + Redis `configkv:invalidate` channel so multi-instance
deployments pick up changes within the pub/sub propagation window.
Guarded by the existing `authGuard + adminGuard` pair
(`ADMIN_EMAILS` allowlist + verified email).
Why:
- Add a real bidirectional streaming TTS path: raw LLM tokens are
forwarded to the upstream model (Volcengine v3 via the unspeech ws
bridge) without client-side segmentation, so the model owns sentence
splitting and audio chunks play as they arrive.
- Move audio endpoints out of /api/v1/openai/. `/audio/voices`,
`/audio/models`, `/audio/voices/streaming` are not real OpenAI public
APIs, and the streaming TTS surface has nothing to do with OpenAI —
keeping them under /openai/ mislabelled the contract.
- Introduce `capabilities.speech.transport` on ProviderDefinition so
future streaming providers (ElevenLabs / Cartesia / OpenAI Realtime)
opt in without touching Stage.vue or the session factory.
- Unify Stage.vue's TTS path through a single StageTtsSession so the
chat-orchestrator hooks no longer branch on provider id.
What:
- apps/server: new ws proxy /api/v1/audio/speech/ws bridges client ↔
unspeech with auth, pre-flight flux check, billing from upstream
session.finished.usage, OTel spans.
- apps/server: audio routes moved from /api/v1/openai/audio/* to
/api/v1/audio/* (hard cutover; 404 sentinel tests added).
- apps/server: new /api/v1/audio/voices/streaming proxy reads voices
from unspeech /api/voices?provider=volcengine.
- apps/server: new STREAMING_TTS_UPSTREAM configKV entry +
scripts/seed-streaming-tts.ts.
- stage-ui: new libs/speech/streaming-pipeline.ts opens one ws per LLM
intent (appendText / finish / cancel + onSentence / onError / onDone).
- stage-ui: new libs/speech/tts-session.ts — StageTtsSession interface
with segmenter and streaming adapters; factory dispatches by
capabilities.speech.transport instead of hard-coded provider id.
- stage-ui: providerOfficialSpeechStreaming with capabilities.speech =
{ transport: 'bidirectional-ws' }; settings page with model/voice
picker + ws-based preview.
- stage-ui: Stage.vue chat hooks collapsed to a single currentSession;
hot-swap watcher cancels mid-session on provider/voice/model change;
unmount cancels and drains playback.
Tests:
- 9 streaming-pipeline tests (happy path / buffered / error / cancel /
truncation)
- 11 tts-session tests (factory branch coverage + adapter contracts)
- 4 audio-speech-ws route tests (forwarding / billing / pre-flight /
config-missing)
- 3 legacy-path 404 sentinels in v1 route tests
- Verification doc updated to reflect automated coverage.
The `/v1/openai/{chat,audio}` handlers used to be silent past
`hono/logger`'s `<-- POST` / `--> 502` lines — no userId, no model,
no token counts, no flux billed. Operators looking at a real gateway
incident had to cross-reference traces, request-log rows, and billing
ledger entries by timestamp alone. For a gateway whose value is
auditable per-request metering, that's not enough.
Hoists `requestId = nanoid()` to handler entry so the same
correlation id flows through:
- the inbound log line (model / stream / messageCount or inputChars
for TTS)
- the per-stream / non-streaming delivered log (status / durationMs /
promptTokens / completionTokens / fluxConsumed)
- the upstream-error degraded path (warn level)
- the partial-debit and debit-failure paths (already used requestId)
- `billingService.consumeFluxForLLM` / `ttsMeter.accumulate` for
DB-level idempotency (replaces the previous local nanoid() calls)
handleListVoices gets a `debug`-level line — the route is high-
frequency from UI voice pickers and we don't want it in the regular
audit feed, but it's useful when chasing voice-picker drift bugs.
No new schemas, no metric emission changes; this is purely logger
output. Pairs with the cause-propagation change so errors carry the
upstream snippet AND the request can be traced end-to-end by id.
When `mapUpstreamError` produced the final 502/503/504, it only carried
`{triedKeys, triedUpstreams, lastStatusCode}` in `details`. The upstream
response body was `.cancel()`'d on the wire and the network error
message vanished into the catch arm — operators staring at a 502 had no
way to tell "OpenRouter region-blocked us" from "key revoked" from
"DNS failed" without re-probing the upstream by hand.
Now each recorded failure carries the diagnostic snippet:
- chat upstreams read at most 256 bytes of the failed body via a
drain-aware reader before cancelling the rest (socket still returns
to the pool, no fallback-storm pool exhaustion).
- TTS upstreams reuse `errorMessageFrom(err)` — adapters already bake
the status + body snippet into `err.message`, so one field carries
both.
- network / timeout attempts record `errorMessageFromUnknown(err)` so
"attempt-timeout" vs "ECONNRESET" vs "DNS failed" stays
distinguishable.
The collected `UpstreamAttempt[]` is attached to `ApiError.cause`
rather than `details`. SEC-5 (no upstream content in client-facing
response body) still holds — only the server-side logger + OTel pick
the cause up. `app.onError` now logs `{details, cause}` together so a
single log line tells the operator both the contract-level summary
and the actual upstream payload.
Adds a router.test.ts regression covering both shapes (HTTP 401 body
snippet + network ECONNRESET errorMessage), with an explicit assertion
that `details` does NOT contain the body text so SEC-5 doesn't drift.
DashScope dropped cosyvoice-v1 from its REST-supported model list. v2
(and v3+) speak a different shape: voice / format / sample_rate live
under `input`, not `parameters`; non-streaming responses return
`output.audio.url` (signed OSS URL) instead of inline `output.audio.data`
base64. The previous adapter sent v1-shaped bodies to a bare
`https://dashscope-intl.aliyuncs.com/api/v1` baseURL and parsed
`audio.data`, which 404'd before the migration and would 200-with-no-
audio after — both invisible regressions for the gateway.
Adapter changes:
- Rewrite request body to v2 schema (voice/format under input).
- Add follow-up GET against `output.audio.url`; stream into ArrayBuffer
with a 25 MB hard cap and explicit drain-tracking finally, so a
misbehaving URL cannot exhaust memory and a half-read body cannot
hang a connection.
- Re-document baseURL contract: adapters do NOT append path; ops must
configure the FULL endpoint URL (root cause of the original 404
storm). DEFAULT_COSYVOICE_MODEL bumped to `cosyvoice-v2`, default
voice to `longxiaochun_v2`.
Voice catalog: regenerated with 19 representative cosyvoice-v2 voices
(assistant / customer-service / child / en-US / en-GB / ja-JP / ko-KR)
so the frontend voice picker is no longer a 2-entry stub. Full catalog
(100+) remains on the Alibaba docs page — we'll sync on demand rather
than scrape.
Seed script: `--dashscope-region intl|cn` (default `intl`),
`--dashscope-upstream-model cosyvoice-v2`, baseURL now resolves to
`https://<host>/api/v1/services/audio/tts/SpeechSynthesizer` so a
mis-typed region or path cannot reintroduce the 404.
Tests: new dashscope-cosyvoice.test.ts covers v2 body shape (asserts
`parameters` absent — regression), audio.url follow-up fetch, 401
propagation with `.status`, empty-envelope falling back into the
router's recoverable-error path, and catalog freshness (no leftover v1
ids). Verified locally against the staging DashScope key: 200 +
playable mp3 end to end.
End-state of the multi-step KTD-5 / KTD-6 / U8 work. The knoway sidecar
is no longer reachable from server code; the router is required at boot
and now owns chat completions, TTS synthesis, and voice catalog listing.
Highlights:
- LLM_ROUTER_MASTER_KEY becomes required; app.ts drops the graceful-
skip branch and the chat fallback fetch path is gone.
- /audio/speech and /audio/voices route through new routeTts /
listTtsVoices entries that reuse the chat key-rotator + per-attempt
timeout + abort propagation.
- DEFAULT_CHAT_MODEL / DEFAULT_TTS_MODEL move from env to configKV so
default-model swaps are hot-reloadable via Pub/Sub.
- GATEWAY_BASE_URL removed from env schema, .env, .env.local, smoke,
verification harness. Redis upstream-voices cache deleted — catalogs
come from in-process adapter JSON.
- routeTts splits adapter error contract by ApiError statusCode:
4xx propagates without fallback; 5xx folds into the network-failure
fallback path. handleTTS wraps billing + span attribute in try/finally
to plug a span leak when ttsMeter.accumulate() throws.
- seed-router-config.ts rewritten with --merge (default) / --reset /
--dry-run modes and env-var key handoff (OPENROUTER_KEY / AZURE_KEY /
DASHSCOPE_KEY) so prod seed flows never put plaintext on the CLI.
Adds DashScope CosyVoice seeding.
Docs (CLAUDE.md, architecture-overview.md, transport-and-routes.md)
reflect the new boundary. verifications/llm-router.md replaces the
overstated "U1-U9 shipped" line with an evidence-vs-pending table.
Tests: full 40-file / 343-case server suite green. New regressions pin
ApiError 4xx → no-fallback, ApiError 5xx → fallback, TTS billing
failure → span closed and error propagated.
- Added a new ObservableGauge for distinct active users to track real active user count, mitigating session row inflation issues.
- Updated the Grafana dashboard to reflect changes, including the removal of redundant WS Connections panel and the addition of new metrics for active sessions and distinct users.
- Improved documentation for verification automation processes, outlining a structured approach to automate verification steps and maintain evidence of tests.
- Added a new PostHog client for capturing server-side business events such as Stripe webhooks and subscription state changes.
- Implemented various tracking functions for pricing funnel steps, character creation, and chat session starts.
- Enhanced the flux meter tests to handle partial charges and report unbilled flux correctly.
- Updated the CharacterDialog and Flux settings pages to track user interactions with analytics events.
- Introduced a mechanism to identify users on PostHog based on authentication state to ensure accurate funnel tracking.
- Added necessary dependencies for PostHog integration in the project.
- Moved RateLimitMetrics import path to a more centralized location.
- Introduced a new file for active sessions gauge to track user sessions in the database.
- Updated index.ts to include new metrics and ensure proper initialization of observability metrics.
- Modified various routes and services to utilize the new observability structure.
- Added smoke tests for HTTP and WebSocket metrics to ensure proper metric registration and functionality.
- Enhanced error handling for metrics reading failures to improve observability.
- Updated the EngagementMetrics interface to use ObservableGauge for tracking active WebSocket connections.
- Added detailed comments explaining the rationale for this change, highlighting the benefits of using a pull-based gauge over a delta-based counter.
- Implemented the ObservableGauge in the createChatWsHandlers function, ensuring it accurately reflects the live count of active connections.
- Removed the previous UpDownCounter logic to prevent issues with connection drift during process crashes or network interruptions.
- Added new metrics for email service including send, failures, and duration tracking.
- Introduced rate limit metrics to monitor blocked requests and improve abuse detection.
- Enhanced billing metrics to track credited and unbilled flux, as well as TTS character processing.
- Updated OpenAI and Stripe routes to utilize new metrics for better revenue tracking and rate limiting.
- Implemented a smoke test for OpenTelemetry metrics registration to ensure visibility at startup.
The Redis Stream `billing-events` + `worker` Railway role +
advisory-lock poller layered together didn't actually buy us reliability
— `debitFlux` swallowed XADD failures, leaving the door open to "balance
updated, ledger row never written". Collapse the whole thing back to:
`creditFlux` and `debitFlux` write `flux_transaction` ledger rows inline
within the same DB transaction that mutates `user_flux`, and `(user_id,
request_id)` remains the partial unique index that keeps retries safe.
Concrete changes:
- Inline ledger inserts in `BillingService.{debitFlux, creditFlux,
creditFluxFromStripeCheckout, creditFluxFromInvoice}`; drop `billingMq`
and `publishEvent` plumbing entirely.
- `routes/openai/v1` writes `llm_request_log` synchronously via the
existing `requestLogService`; the duplicate `llm-request-log.ts` service
module is removed.
- `bin/run-worker.ts`, `libs/mq/*`,
`services/billing/billing-events.ts`,
`services/billing/billing-consumer-handler.ts`, and matching tests are
deleted. CLI now exposes only `api`.
- `BILLING_EVENTS_*` env vars and the `DEFAULT_BILLING_EVENTS_STREAM`
helper are dropped; `docker-compose.yml` no longer ships a worker
service.
- `docs/ai-context/{workers-and-runtime, billing-architecture,
redis-boundaries-and-pubsub, data-model-and-state,
architecture-overview, README}.md`, `CLAUDE.md`, and the existing
verification docs are updated to describe the single-process synchronous
pipeline.
Tests: 29 files / 247 cases pass. Production deployments need to drop
the worker Railway service after this lands.
RainbowBird