refactor(server): proxy TTS through unspeech, drop implicit fallbacks

Frontend voice picker silently lost Azure voices because /audio/voices
was called without ?model=, so the server fell back to DEFAULT_TTS_MODEL
and only ever returned one model's catalog. /audio/models also hardcoded
a single `auto` alias, so the client could never request a specific
model id even if it wanted to. On top of that, the Azure adapter shipped
only 2 hand-typed voices in voices/azure.json — the rest of Microsoft's
catalog was unreachable.

Fixed in one pass:

- Drop voices/*.json. Every getVoiceCatalog now calls unspeech REST
  (backend=microsoft|alibaba|volcengine). Azure proxies Microsoft's live
  voices/list; cosyvoice + volcengine use unspeech's embedded catalogs.
- Drop direct upstream provider calls from send(). All three adapters
  POST to <UNSPEECH_REST_BASE_URL>/v1/audio/speech with the openai-shaped
  body unspeech expects (model: `<backend>/<id>`). Azure SSML still built
  on our side so <prosody rate> speed survives — unspeech accepts
  pre-built <speak> documents.
- Fail-fast on every voices endpoint: empty ?model= → 400 MISSING_MODEL
  (no implicit DEFAULT_TTS_MODEL fallback), only `auto` resolves;
  streaming upstream missing → 503; unspeech non-2xx / network err → 502;
  malformed body → 502 with grepable message. No silent `{voices: []}`.
- /audio/models lists every LLM_ROUTER_CONFIG.tts.models key + `auto`.
- Frontend providerOfficialSpeech.listVoices passes ?model= and throws
  on upstream failure instead of returning [].
- New UNSPEECH_REST_BASE_URL configKV entry (no default, missing → 503).
  config-sync subscriber invalidates the voice cache when this key OR
  LLM_ROUTER_CONFIG changes.
- Router voice catalog Redis cache covers all providers with per-provider
  TTL (azure 6h live, alibaba/volcengine 24h since unspeech embeds them
  at build time).

Pending billing-SKU verification: DashScope cosyvoice now goes through
unspeech's WS-internal alibaba backend (was REST `/SpeechSynthesizer`).
Functional output equivalent — confirm bill after deploy, revert via
git history if WS pricing is materially higher.
This commit is contained in:
RainbowBird
2026-05-19 23:14:16 +08:00
parent 6b0788dd60
commit 615e0441e8
17 changed files with 993 additions and 749 deletions
@@ -92,11 +92,11 @@ export const providerOfficialSpeech = defineProvider({
listModels: async (): Promise<ModelInfo[]> => {
const res = await globalThis.fetch(`${SERVER_URL}/api/v1/audio/models`, { headers: authHeaders() })
if (!res.ok)
return []
throw new Error(`audio models upstream ${res.status}: ${await res.text().catch(() => '')}`.slice(0, 256))
const data = await res.json() as { models?: { id: string, name: string }[] }
if (!Array.isArray(data.models))
return []
throw new Error('audio models upstream returned malformed body')
return data.models.map(m => ({
id: m.id,
@@ -104,10 +104,18 @@ export const providerOfficialSpeech = defineProvider({
provider: OFFICIAL_SPEECH_PROVIDER_ID,
}))
},
listVoices: async (): Promise<VoiceInfo[]> => {
const res = await globalThis.fetch(`${SERVER_URL}/api/v1/audio/voices`, { headers: authHeaders() })
listVoices: async (_config, _provider, model): Promise<VoiceInfo[]> => {
// Voice catalogs are model-scoped on the server side. Pass the active
// model through so Azure / cosyvoice / future provider voices route to
// the right adapter. `auto` defers to the server's DEFAULT_TTS_MODEL,
// but it MUST be sent explicitly — an absent `model` is treated as a
// client bug and returns 400.
const target = model && model.length > 0 ? model : 'auto'
const url = new URL(`${SERVER_URL}/api/v1/audio/voices`)
url.searchParams.set('model', target)
const res = await globalThis.fetch(url.toString(), { headers: authHeaders() })
if (!res.ok)
return []
throw new Error(`audio voices upstream ${res.status}: ${await res.text().catch(() => '')}`.slice(0, 256))
// Shape aligned with unspeech's types.ListVoicesResponse, plus the
// `recommended` field our server injects from configKV DEFAULT_TTS_VOICES.
@@ -132,7 +140,7 @@ export const providerOfficialSpeech = defineProvider({
recommendedVoicesByLocale = (data.recommended && typeof data.recommended === 'object') ? data.recommended : {}
if (!Array.isArray(data.voices))
return []
throw new Error('audio voices upstream returned malformed body')
return data.voices.map((v) => {
// unspeech surfaces gender inside labels rather than as a top-level field.
@@ -205,6 +213,11 @@ export const providerOfficialSpeechStreaming = defineProvider({
validationRequiredWhen: () => false,
extraMethods: {
listModels: async (): Promise<ModelInfo[]> => {
// Streaming TTS catalog is operator-controlled via configKV
// (`STREAMING_TTS_MODELS`). The wire `model` field uses the
// `<backend>/<api_resource_id>` shape unspeech expects (see
// `unspeech/docs/wire-protocols/audio-speech-stream-v1.md`); the server
// returns whatever the operator put there, no client-side defaults.
const res = await globalThis.fetch(`${SERVER_URL}/api/v1/audio/models/streaming`, { headers: authHeaders() })
if (!res.ok)
throw new Error(`streaming models upstream ${res.status}: ${await res.text().catch(() => '')}`.slice(0, 256))
@@ -240,7 +253,7 @@ export const providerOfficialSpeechStreaming = defineProvider({
{ headers: authHeaders() },
)
if (!res.ok)
return []
throw new Error(`streaming voices upstream ${res.status}: ${await res.text().catch(() => '')}`.slice(0, 256))
const data = await res.json() as {
voices?: {
@@ -253,7 +266,7 @@ export const providerOfficialSpeechStreaming = defineProvider({
}[]
}
if (!Array.isArray(data.voices))
return []
throw new Error('streaming voices upstream returned malformed body')
return data.voices.map((v) => {
const rawGender = typeof v.labels?.gender === 'string' ? (v.labels.gender as string) : undefined