refactor(server): proxy TTS through unspeech, drop implicit fallbacks

Frontend voice picker silently lost Azure voices because /audio/voices
was called without ?model=, so the server fell back to DEFAULT_TTS_MODEL
and only ever returned one model's catalog. /audio/models also hardcoded
a single `auto` alias, so the client could never request a specific
model id even if it wanted to. On top of that, the Azure adapter shipped
only 2 hand-typed voices in voices/azure.json — the rest of Microsoft's
catalog was unreachable.

Fixed in one pass:

- Drop voices/*.json. Every getVoiceCatalog now calls unspeech REST
  (backend=microsoft|alibaba|volcengine). Azure proxies Microsoft's live
  voices/list; cosyvoice + volcengine use unspeech's embedded catalogs.
- Drop direct upstream provider calls from send(). All three adapters
  POST to <UNSPEECH_REST_BASE_URL>/v1/audio/speech with the openai-shaped
  body unspeech expects (model: `<backend>/<id>`). Azure SSML still built
  on our side so <prosody rate> speed survives — unspeech accepts
  pre-built <speak> documents.
- Fail-fast on every voices endpoint: empty ?model= → 400 MISSING_MODEL
  (no implicit DEFAULT_TTS_MODEL fallback), only `auto` resolves;
  streaming upstream missing → 503; unspeech non-2xx / network err → 502;
  malformed body → 502 with grepable message. No silent `{voices: []}`.
- /audio/models lists every LLM_ROUTER_CONFIG.tts.models key + `auto`.
- Frontend providerOfficialSpeech.listVoices passes ?model= and throws
  on upstream failure instead of returning [].
- New UNSPEECH_REST_BASE_URL configKV entry (no default, missing → 503).
  config-sync subscriber invalidates the voice cache when this key OR
  LLM_ROUTER_CONFIG changes.
- Router voice catalog Redis cache covers all providers with per-provider
  TTL (azure 6h live, alibaba/volcengine 24h since unspeech embeds them
  at build time).

Pending billing-SKU verification: DashScope cosyvoice now goes through
unspeech's WS-internal alibaba backend (was REST `/SpeechSynthesizer`).
Functional output equivalent — confirm bill after deploy, revert via
git history if WS pricing is materially higher.
This commit is contained in:
RainbowBird
2026-05-19 23:14:16 +08:00
parent 6b0788dd60
commit 615e0441e8
17 changed files with 993 additions and 749 deletions
+2 -1
View File
@@ -635,11 +635,12 @@ export async function createApp() {
// LLM_ROUTER_MASTER_KEY is required at env-parse time, so this provider
// always builds a real router — the legacy `null` fallback path is gone.
const llmRouter = injeca.provide('services:llmRouter', {
dependsOn: { configKV, envelopeCrypto, otel },
dependsOn: { configKV, envelopeCrypto, otel, redis },
build: ({ dependsOn }) => createLlmRouterService({
configKV: dependsOn.configKV,
envelopeCrypto: dependsOn.envelopeCrypto,
gatewayMetrics: dependsOn.otel?.gateway ?? null,
redis: dependsOn.redis,
}),
})
+60 -39
View File
@@ -20,7 +20,7 @@ import { configGuard } from '../../../middlewares/config-guard'
import { rateLimiter } from '../../../middlewares/rate-limit'
import { captureSafe } from '../../../services/adapters/posthog'
import { calculateFluxFromUsage, extractUsageFromBody } from '../../../services/domain/billing/billing'
import { createPaymentRequiredError } from '../../../utils/error'
import { createBadGatewayError, createBadRequestError, createPaymentRequiredError, createServiceUnavailableError } from '../../../utils/error'
import { nanoid } from '../../../utils/id'
import {
AIRI_ATTR_BILLING_FLUX_CONSUMED,
@@ -615,14 +615,21 @@ export function createV1Routes(
}
async function handleListVoices(c: Context<HonoEnv>) {
// Voice catalogs are per-model (different TTS models expose different
// voices). Catalog content comes from the adapter's compiled-in JSON
// (apps/server/src/services/tts-adapters/voices/*.json), so there's
// nothing to fetch and the Redis upstream cache is no longer needed —
// adapter-side JSON is already in-process. Recommended map stays in
// configKV so operators can edit it without a deploy.
// Voice catalogs are per-model. Live providers (Azure) call upstream
// via unspeech; static providers (cosyvoice, volcengine) return their
// bundled JSON. The Redis cache + invalidation lives one layer down
// in the router so route-level changes don't leak into the cache
// contract. Recommended map stays in configKV so operators can edit it
// without a deploy.
//
// No implicit fallback: an empty `?model=` is a client bug (the UI is
// expected to pass either an explicit model id or the `auto` alias) and
// returns 400 instead of silently resolving to DEFAULT_TTS_MODEL.
const requested = c.req.query('model')
const model = (!requested || requested === 'auto')
if (requested === undefined || requested === '')
throw createBadRequestError('audio voices: ?model= is required (use `auto` to defer to DEFAULT_TTS_MODEL)', 'MISSING_MODEL')
const model = requested === 'auto'
? await configKV.getOrThrow('DEFAULT_TTS_MODEL')
: requested
@@ -638,19 +645,23 @@ export function createV1Routes(
/**
* Voice catalog for the streaming TTS provider (`/audio/speech/ws`).
*
* The HTTP `/audio/voices?model=…` endpoint above queries
* `LLM_ROUTER_CONFIG.tts.models` and is unaware of the streaming
* surface. Streaming uses `STREAMING_TTS_UPSTREAM` (a single unspeech
* instance) instead, and unspeech ships an embed-time voice catalog
* for Volcengine that doesn't require credentials — so we proxy
* straight to it. Falls back to an empty list if streaming isn't
* configured yet so the client can render "no voices" instead of
* exploding.
* Streaming uses `STREAMING_TTS_UPSTREAM` (a single unspeech instance)
* to actually open the ws session, but the REST voices catalog lives
* at `UNSPEECH_REST_BASE_URL` — kept as a separate configKV entry so
* operators can split the streaming endpoint from the catalog source
* if they want, and so this path doesn't have to derive HTTPS from a
* `wss://` URL (which has bitten us once already).
*
* Errors propagate verbatim: missing config → 503, malformed upstream
* URL → 500, unspeech network failure → 502, unspeech non-2xx → 502.
* No empty-array fallback — the UI surfaces a real failure state.
*/
async function handleListStreamingVoices(c: Context<HonoEnv>) {
const upstream = await configKV.getOptional('STREAMING_TTS_UPSTREAM')
if (!upstream || !upstream.baseURL)
return Response.json({ voices: [], recommended: {} })
const streaming = await configKV.getOptional('STREAMING_TTS_UPSTREAM')
if (!streaming || !streaming.baseURL)
throw createServiceUnavailableError('streaming tts upstream not configured', 'STREAMING_TTS_NOT_CONFIGURED')
const unspeechBaseURL = await configKV.getOrThrow('UNSPEECH_REST_BASE_URL')
// Pass through the api_resource_id (e.g. `seed-tts-2.0`). unspeech
// filters the embedded Volcengine catalogue server-side; absent model
@@ -659,10 +670,7 @@ export function createV1Routes(
let voicesURL: string
try {
const u = new URL(upstream.baseURL)
// ws:// → http://, wss:// → https://. unspeech serves both the WS
// stream and the REST voices endpoint on the same listener.
u.protocol = u.protocol === 'wss:' ? 'https:' : 'http:'
const u = new URL(unspeechBaseURL)
u.pathname = '/api/voices'
const params = new URLSearchParams({ provider: 'volcengine' })
if (model)
@@ -671,8 +679,8 @@ export function createV1Routes(
voicesURL = u.toString()
}
catch (err) {
logger.withError(err).withFields({ baseURL: upstream.baseURL }).warn('streaming-voices: bad upstream URL')
return Response.json({ voices: [], recommended: {} })
logger.withError(err).withFields({ unspeechBaseURL }).warn('streaming-voices: bad UNSPEECH_REST_BASE_URL')
throw createBadGatewayError('UNSPEECH_REST_BASE_URL is malformed')
}
let res: Response
@@ -683,32 +691,45 @@ export function createV1Routes(
}
catch (err) {
logger.withError(err).withFields({ voicesURL }).warn('streaming-voices: unspeech fetch failed')
return Response.json({ voices: [], recommended: {} })
throw createBadGatewayError('streaming voices upstream fetch failed')
}
if (!res.ok) {
logger.withFields({ voicesURL, status: res.status }).warn('streaming-voices: unspeech non-2xx')
return Response.json({ voices: [], recommended: {} })
const snippet = await res.text().catch(() => '')
logger.withFields({ voicesURL, status: res.status, snippet: snippet.slice(0, 256) }).warn('streaming-voices: unspeech non-2xx')
throw createBadGatewayError(`streaming voices upstream ${res.status}`, { lastStatusCode: res.status })
}
const data = await res.json().catch(() => ({})) as { voices?: unknown[] }
const data = await res.json() as { voices: unknown[] }
if (!Array.isArray(data.voices))
throw createBadGatewayError('streaming voices upstream missing voices[]')
const recommended = model
? ((await configKV.getOptional('DEFAULT_TTS_VOICES'))?.[model] ?? {})
: {}
return Response.json({
voices: Array.isArray(data.voices) ? data.voices : [],
recommended,
})
return Response.json({ voices: data.voices, recommended })
}
async function handleListTTSModels(_c: Context<HonoEnv>) {
// Mirror the chat provider: expose a single 'auto' routing alias instead
// of the concrete DEFAULT_TTS_MODEL id. Keeps clients insulated from
// backend model swaps and stays symmetric with /chat listModels.
// /audio/speech and /audio/voices already translate 'auto' into the
// configKV DEFAULT_TTS_MODEL alias before hitting upstream.
// Surface the concrete TTS models the operator has configured plus the
// `auto` alias. Clients need real model ids to pass `?model=<id>` to
// `/audio/voices`, otherwise the voice catalog endpoint can never resolve
// anything beyond the DEFAULT_TTS_MODEL catalog — which is the bug that
// hid the Azure voices from the UI.
//
// `auto` is kept on top as an explicit "use the operator default" knob
// for clients that don't care which concrete model handles them.
const config = await configKV.getOrThrow('LLM_ROUTER_CONFIG')
// `LLM_ROUTER_CONFIG` is `optional()` at the schema, so its inferred type
// tolerates `undefined`. `getOrThrow` already throws on missing entries,
// so by this line we know `config` is present — the `?.` here is purely
// a TS narrowing aid.
const modelIds = Object.keys(config?.tts?.models ?? {}).sort()
return Response.json({
models: [{ id: 'auto', name: 'Auto' }],
models: [
{ id: 'auto', name: 'Auto' },
...modelIds.map(id => ({ id, name: id })),
],
})
}
+140 -9
View File
@@ -110,6 +110,7 @@ function createMockLlmRouter(impl?: Partial<LlmRouterService>): LlmRouterService
}),
listTtsVoices: vi.fn(async () => []),
invalidateConfig: vi.fn(),
invalidateTtsVoicesCache: vi.fn(async () => undefined),
...impl,
} as LlmRouterService
}
@@ -747,10 +748,42 @@ describe('v1CompletionsRoutes', () => {
})
describe('gET /api/v1/audio/models', () => {
it('exposes only the auto routing alias regardless of DEFAULT_TTS_MODEL', async () => {
it('exposes auto alias plus every configured tts model id', async () => {
const app = createTestApp(
createMockFluxService(),
createMockConfigKV({ DEFAULT_TTS_MODEL: 'microsoft/v1' }),
createMockConfigKV({
LLM_ROUTER_CONFIG: {
llm: { models: {} },
tts: {
models: {
'microsoft/v1': { provider: 'azure', upstreams: [] as unknown[] },
'alibaba/cosyvoice-v2': { provider: 'dashscope-cosyvoice', upstreams: [] as unknown[] },
},
},
},
}),
)
const res = await app.fetch(
new Request('http://localhost/api/v1/audio/models', { method: 'GET' }),
{ user: testUser } as any,
)
expect(res.status).toBe(200)
const data = await res.json() as { models: { id: string, name: string }[] }
expect(data.models[0]).toEqual({ id: 'auto', name: 'Auto' })
expect(data.models.slice(1).map(m => m.id)).toEqual([
'alibaba/cosyvoice-v2',
'microsoft/v1',
])
})
it('returns only the auto alias when no tts models are configured', async () => {
const app = createTestApp(
createMockFluxService(),
createMockConfigKV({
LLM_ROUTER_CONFIG: { llm: { models: {} }, tts: { models: {} } },
}),
)
const res = await app.fetch(
@@ -843,7 +876,7 @@ describe('v1CompletionsRoutes', () => {
})
describe('gET /api/v1/audio/voices', () => {
it('returns the recommended bucket scoped to the resolved model', async () => {
it('returns the recommended bucket scoped to the explicit model id', async () => {
const voices = [
{ id: 'en-US-JennyNeural', name: 'Jenny', provider: 'azure', locale: 'en-US', gender: 'Female' },
{ id: 'en-US-AvaMultilingualNeural', name: 'Ava', provider: 'azure', locale: 'en-US', gender: 'Female' },
@@ -853,7 +886,7 @@ describe('v1CompletionsRoutes', () => {
})
const configKV = createMockConfigKV({
DEFAULT_TTS_VOICES: {
'tts-1': { 'en-US': 'en-US-AvaMultilingualNeural' },
'microsoft/v1': { 'en-US': 'en-US-AvaMultilingualNeural' },
'other-model': { 'en-US': 'should-not-leak' },
},
})
@@ -861,7 +894,7 @@ describe('v1CompletionsRoutes', () => {
const app = createTestApp(createMockFluxService(), configKV, undefined, undefined, undefined, llmRouter)
const res = await app.fetch(
new Request('http://localhost/api/v1/audio/voices', { method: 'GET' }),
new Request('http://localhost/api/v1/audio/voices?model=microsoft/v1', { method: 'GET' }),
{ user: testUser } as any,
)
@@ -869,7 +902,7 @@ describe('v1CompletionsRoutes', () => {
const data = await res.json() as { voices: typeof voices, recommended: Record<string, string> }
expect(data.voices).toEqual(voices)
expect(data.recommended).toEqual({ 'en-US': 'en-US-AvaMultilingualNeural' })
expect(llmRouter.listTtsVoices).toHaveBeenCalledWith('tts-1')
expect(llmRouter.listTtsVoices).toHaveBeenCalledWith('microsoft/v1')
})
it('returns an empty recommended map when the resolved model has no bucket', async () => {
@@ -916,6 +949,40 @@ describe('v1CompletionsRoutes', () => {
await app.fetch(new Request('http://localhost/api/v1/audio/voices?model=auto'), { user: testUser } as any)
expect(llmRouter.listTtsVoices).toHaveBeenCalledWith('microsoft/v1')
})
it('returns 400 MISSING_MODEL when ?model= is omitted (no implicit fallback)', async () => {
const llmRouter = createMockLlmRouter({
listTtsVoices: vi.fn(async () => []) as any,
})
const app = createTestApp(createMockFluxService(), createMockConfigKV(), undefined, undefined, undefined, llmRouter)
const res = await app.fetch(
new Request('http://localhost/api/v1/audio/voices', { method: 'GET' }),
{ user: testUser } as any,
)
expect(res.status).toBe(400)
const body = await res.json() as { error?: string, message?: string }
expect(body.error).toBe('MISSING_MODEL')
expect(llmRouter.listTtsVoices).not.toHaveBeenCalled()
})
it('returns 400 MISSING_MODEL when ?model= is empty string', async () => {
const llmRouter = createMockLlmRouter({
listTtsVoices: vi.fn(async () => []) as any,
})
const app = createTestApp(createMockFluxService(), createMockConfigKV(), undefined, undefined, undefined, llmRouter)
const res = await app.fetch(
new Request('http://localhost/api/v1/audio/voices?model=', { method: 'GET' }),
{ user: testUser } as any,
)
expect(res.status).toBe(400)
expect(llmRouter.listTtsVoices).not.toHaveBeenCalled()
})
})
describe('gET /api/v1/audio/voices/streaming', () => {
@@ -926,10 +993,15 @@ describe('v1CompletionsRoutes', () => {
})) as any
}
function mockUnspeechFailure(status: number, body = 'boom') {
globalThis.fetch = vi.fn(async () => new Response(body, { status })) as any
}
it('returns the streaming-model bucket of DEFAULT_TTS_VOICES when ?model= matches', async () => {
mockUnspeechVoices([{ id: 'zh_female_vv_uranus_bigtts', name: 'Vivi 2.0' }])
const configKV = createMockConfigKV({
STREAMING_TTS_UPSTREAM: { baseURL: 'http://unspeech.local' },
STREAMING_TTS_UPSTREAM: { baseURL: 'ws://unspeech.local:5933/v1/audio/speech/stream' },
UNSPEECH_REST_BASE_URL: 'http://unspeech.local:5933',
DEFAULT_TTS_VOICES: {
'seed-tts-2.0': { 'zh-cn': 'zh_female_vv_uranus_bigtts' },
'seed-tts-1.0': { 'zh-cn': 'should-not-leak' },
@@ -951,7 +1023,8 @@ describe('v1CompletionsRoutes', () => {
it('returns empty recommended when ?model= is omitted', async () => {
mockUnspeechVoices([])
const configKV = createMockConfigKV({
STREAMING_TTS_UPSTREAM: { baseURL: 'http://unspeech.local' },
STREAMING_TTS_UPSTREAM: { baseURL: 'ws://unspeech.local:5933/v1/audio/speech/stream' },
UNSPEECH_REST_BASE_URL: 'http://unspeech.local:5933',
DEFAULT_TTS_VOICES: { 'seed-tts-2.0': { 'zh-cn': 'x' } },
})
@@ -969,7 +1042,8 @@ describe('v1CompletionsRoutes', () => {
it('returns empty recommended when the requested model has no configKV bucket', async () => {
mockUnspeechVoices([])
const configKV = createMockConfigKV({
STREAMING_TTS_UPSTREAM: { baseURL: 'http://unspeech.local' },
STREAMING_TTS_UPSTREAM: { baseURL: 'ws://unspeech.local:5933/v1/audio/speech/stream' },
UNSPEECH_REST_BASE_URL: 'http://unspeech.local:5933',
DEFAULT_TTS_VOICES: { 'seed-tts-2.0': { 'zh-cn': 'x' } },
})
@@ -983,6 +1057,63 @@ describe('v1CompletionsRoutes', () => {
const data = await res.json() as { recommended: Record<string, string> }
expect(data.recommended).toEqual({})
})
it('returns 503 STREAMING_TTS_NOT_CONFIGURED when STREAMING_TTS_UPSTREAM is absent', async () => {
mockUnspeechVoices([])
const configKV = createMockConfigKV({
STREAMING_TTS_UPSTREAM: undefined,
UNSPEECH_REST_BASE_URL: 'http://unspeech.local:5933',
})
const app = createTestApp(createMockFluxService(), configKV)
const res = await app.fetch(
new Request('http://localhost/api/v1/audio/voices/streaming'),
{ user: testUser } as any,
)
expect(res.status).toBe(503)
const body = await res.json() as { error?: string }
expect(body.error).toBe('STREAMING_TTS_NOT_CONFIGURED')
})
it('returns 502 BAD_GATEWAY when unspeech responds non-2xx', async () => {
mockUnspeechFailure(503, 'unspeech is sleeping')
const configKV = createMockConfigKV({
STREAMING_TTS_UPSTREAM: { baseURL: 'ws://unspeech.local:5933/v1/audio/speech/stream' },
UNSPEECH_REST_BASE_URL: 'http://unspeech.local:5933',
})
const app = createTestApp(createMockFluxService(), configKV)
const res = await app.fetch(
new Request('http://localhost/api/v1/audio/voices/streaming?model=seed-tts-2.0'),
{ user: testUser } as any,
)
expect(res.status).toBe(502)
const body = await res.json() as { error?: string }
expect(body.error).toBe('BAD_GATEWAY')
})
it('returns 502 BAD_GATEWAY when unspeech fetch throws', async () => {
globalThis.fetch = vi.fn(async () => {
throw new Error('ECONNREFUSED')
}) as any
const configKV = createMockConfigKV({
STREAMING_TTS_UPSTREAM: { baseURL: 'ws://unspeech.local:5933/v1/audio/speech/stream' },
UNSPEECH_REST_BASE_URL: 'http://unspeech.local:5933',
})
const app = createTestApp(createMockFluxService(), configKV)
const res = await app.fetch(
new Request('http://localhost/api/v1/audio/voices/streaming?model=seed-tts-2.0'),
{ user: testUser } as any,
)
expect(res.status).toBe(502)
})
})
describe('route matching', () => {
@@ -147,6 +147,18 @@ const ConfigEntrySchemas = {
// carry the upstream-provider API key (e.g. Volcengine X-Api-Key), not an
// unspeech tenant token.
STREAMING_TTS_UPSTREAM: optional(streamingTtsUpstreamSchema),
// unspeech REST base URL (e.g. `https://airi-unspeech.railway.internal:5933`).
// Used by:
// - HTTP voice catalog lookup for live providers (Azure): the Azure TTS
// adapter calls `<base>/api/voices?backend=microsoft&region=<region>`.
// - Streaming voice catalog lookup: the streaming voices handler calls
// `<base>/api/voices?provider=volcengine&model=<api_resource_id>`.
// Kept explicit (no derivation from STREAMING_TTS_UPSTREAM.baseURL) so
// operators can point REST/voices lookups at a different unspeech instance
// from the streaming ws upstream if they want. Naked schema (no default):
// missing entry surfaces CONFIG_NOT_SET and the request fails fast instead
// of silently returning an empty voices list.
UNSPEECH_REST_BASE_URL: pipe(string(), nonEmpty('UNSPEECH_REST_BASE_URL must not be empty')),
} as const
type ConfigDefinitions = {
+74 -17
View File
@@ -1,12 +1,10 @@
import type { Voice } from 'unspeech'
import type { TtsAdapter, TtsAdapterContext, TtsInput, TtsResult } from './types'
import type { TtsAdapter, TtsAdapterContext, TtsInput, TtsResult, TtsVoiceCatalogContext } from './types'
import { errorMessageFrom } from '@moeru/std'
import azureVoices from './voices/azure.json' with { type: 'json' }
import { createBadRequestError, createInternalError } from '../../../utils/error'
import { createBadGatewayError, createBadRequestError, createInternalError, createServiceUnavailableError } from '../../../utils/error'
// NOTICE:
// Voice IDs Azure accepts are stable strings like `en-US-AvaMultilingualNeural`.
@@ -159,24 +157,39 @@ export const azureAdapter: TtsAdapter = {
const outputFormat = resolveAzureFormat(input.responseFormat)
const disableSsml = input.extraOptions?.disableSsml === true
// When disableSsml is set the caller is responsible for shipping valid
// SSML themselves; we forward as-is to support callers wiring their own
// <speak> documents.
const body = disableSsml
// We build the SSML envelope on our side rather than letting unspeech do
// it, because (a) unspeech's `processSSML` does not honor `<prosody rate>`
// so speed multipliers would be silently dropped, and (b) the SSML escape
// (`escapeForSsml`) is a security boundary — keeping it in this process
// means a misbehaving unspeech can never accidentally accept raw text and
// surface an injection. unspeech detects a pre-built `<speak>` document
// in the `input` field and forwards it verbatim.
const ssml = disableSsml
? input.text
: buildAzureSsml(input.text, voice, input.speed)
const headers: Record<string, string> = {
'Ocp-Apim-Subscription-Key': ctx.keyPlaintext.toString('utf8'),
'X-Microsoft-OutputFormat': outputFormat,
'Content-Type': 'application/ssml+xml',
}
const region = ctx.adapterParams?.region
if (typeof region !== 'string' || !region)
throw createInternalError('azure tts upstream is missing adapterParams.region')
const body = JSON.stringify({
model: 'microsoft/v1',
input: ssml,
voice,
response_format: outputFormat,
extra_body: { region },
})
let response: Response
try {
response = await ctx.fetchImpl(ctx.baseURL, {
response = await ctx.fetchImpl(`${ctx.unspeechBaseURL.replace(/\/+$/, '')}/v1/audio/speech`, {
method: 'POST',
headers,
headers: {
// unspeech's microsoft backend reads the bearer token verbatim as
// the Azure subscription key (unspeech/pkg/backend/microsoft/speech.go:264).
'Authorization': `Bearer ${ctx.keyPlaintext.toString('utf8')}`,
'Content-Type': 'application/json',
},
body,
signal: ctx.abortSignal,
})
@@ -189,6 +202,8 @@ export const azureAdapter: TtsAdapter = {
if (!response.ok) {
// Bubble the upstream status. Router (U3) maps to fallback or 5xx.
// unspeech preserves the upstream Azure status code on its
// `apierrors.NewUpstreamError(status)` envelope.
const text = await response.text().catch(() => '')
const err = new Error(`azure tts upstream ${response.status}: ${text.slice(0, 256)}`) as Error & { status?: number }
err.status = response.status
@@ -201,8 +216,50 @@ export const azureAdapter: TtsAdapter = {
return { contentType, body: arrayBuffer }
},
getVoiceCatalog() {
return azureVoices as Voice[]
async getVoiceCatalog(ctx: TtsVoiceCatalogContext): Promise<Voice[]> {
// Azure has no static catalog. Voices live at Microsoft's `voices/list`
// REST endpoint, which we reach via the unspeech `microsoft` backend
// because unspeech already maps the proprietary response shape to
// `types.Voice` (full formats table, masterpiece preview URLs, locale
// metadata). Calling unspeech also keeps a single integration point for
// every other provider that could grow this way later.
if (!ctx.region)
throw createServiceUnavailableError('azure tts region not configured', 'AZURE_TTS_NOT_CONFIGURED')
if (!ctx.keyPlaintext)
throw createServiceUnavailableError('azure tts key not configured', 'AZURE_TTS_NOT_CONFIGURED')
const url = `${ctx.unspeechBaseURL.replace(/\/+$/, '')}/api/voices?backend=microsoft&region=${encodeURIComponent(ctx.region)}`
let response: Response
try {
response = await ctx.fetchImpl(url, {
method: 'GET',
headers: {
// unspeech's microsoft backend reads the bearer token verbatim as
// the Azure subscription key (see unspeech/pkg/backend/microsoft/voices.go).
Authorization: `Bearer ${ctx.keyPlaintext.toString('utf8')}`,
Accept: 'application/json',
},
signal: ctx.abortSignal,
})
}
catch (error) {
throw createBadGatewayError(`azure voices fetch failed: ${errorMessageFrom(error) ?? 'unknown'}`)
}
if (!response.ok) {
const text = await response.text().catch(() => '')
throw createBadGatewayError(
`azure voices upstream ${response.status}: ${text.slice(0, 256)}`,
{ lastStatusCode: response.status },
)
}
const data = await response.json() as { voices: Voice[] }
if (!Array.isArray(data.voices))
throw createBadGatewayError('azure voices upstream missing voices[]')
return data.voices
},
}
@@ -4,15 +4,8 @@ import { describe, expect, it, vi } from 'vitest'
import { dashscopeCosyvoiceAdapter } from './dashscope-cosyvoice'
const FULL_ENDPOINT = 'https://dashscope-intl.aliyuncs.com/api/v1/services/audio/tts/SpeechSynthesizer'
const AUDIO_URL = 'https://dashscope-internal.aliyuncs.com/audio/abc.mp3'
function jsonResponse(body: object, status = 200) {
return new Response(JSON.stringify(body), {
status,
headers: { 'content-type': 'application/json' },
})
}
const UNSPEECH = 'http://unspeech.local:5933'
const SPEECH_URL = `${UNSPEECH}/v1/audio/speech`
function binaryResponse(bytes: Uint8Array, status = 200) {
return new Response(bytes, {
@@ -22,58 +15,36 @@ function binaryResponse(bytes: Uint8Array, status = 200) {
}
describe('dashscopeCosyvoiceAdapter', () => {
it('sends v2-shaped body: voice / format under input, not parameters (regression — old shape returned no audio.data)', async () => {
const fetchImpl = vi.fn()
.mockResolvedValueOnce(jsonResponse({ output: { audio: { url: AUDIO_URL } } }))
.mockResolvedValueOnce(binaryResponse(new Uint8Array([1, 2, 3, 4])))
it('forwards to unspeech with model=alibaba/<adapterParams.model>, voice + response_format passthrough', async () => {
const audioBytes = new Uint8Array([0x49, 0x44, 0x33, 0x04, 0x00, 0x00]) // ID3v2 mp3 header
const fetchImpl = vi.fn().mockResolvedValueOnce(binaryResponse(audioBytes))
await dashscopeCosyvoiceAdapter.send(
const result = await dashscopeCosyvoiceAdapter.send(
{ text: 'hi there', voice: 'longxiaochun_v2', responseFormat: 'mp3' },
{
keyPlaintext: Buffer.from('sk-test', 'utf8'),
baseURL: FULL_ENDPOINT,
baseURL: 'https://dashscope-intl.aliyuncs.com/api/v1/services/audio/tts/SpeechSynthesizer',
unspeechBaseURL: UNSPEECH,
adapterParams: { model: 'cosyvoice-v2' },
fetchImpl: fetchImpl as unknown as typeof fetch,
},
)
expect(fetchImpl).toHaveBeenCalledTimes(2)
const [synthesizeUrl, synthesizeInit] = fetchImpl.mock.calls[0]
expect(synthesizeUrl).toBe(FULL_ENDPOINT)
expect(synthesizeInit.method).toBe('POST')
expect(fetchImpl).toHaveBeenCalledTimes(1)
const [calledURL, init] = fetchImpl.mock.calls[0]
expect(calledURL).toBe(SPEECH_URL)
expect(init.method).toBe('POST')
const body = JSON.parse(synthesizeInit.body as string)
const body = JSON.parse(init.body as string)
expect(body).toEqual({
model: 'cosyvoice-v2',
input: {
text: 'hi there',
voice: 'longxiaochun_v2',
format: 'mp3',
},
model: 'alibaba/cosyvoice-v2',
input: 'hi there',
voice: 'longxiaochun_v2',
response_format: 'mp3',
})
// Critical: voice / format must NOT leak into a top-level `parameters` block.
expect(body.parameters).toBeUndefined()
})
it('follows output.audio.url to fetch the actual audio bytes and returns them as ArrayBuffer (regression — v1 parsed base64 from output.audio.data, v2 returns a URL instead)', async () => {
const audioBytes = new Uint8Array([0x49, 0x44, 0x33, 0x04, 0x00, 0x00]) // ID3v2 mp3 header
const fetchImpl = vi.fn()
.mockResolvedValueOnce(jsonResponse({ output: { audio: { url: AUDIO_URL, data: '' } } }))
.mockResolvedValueOnce(binaryResponse(audioBytes))
const result = await dashscopeCosyvoiceAdapter.send(
{ text: 'hi', responseFormat: 'mp3' },
{
keyPlaintext: Buffer.from('sk-test', 'utf8'),
baseURL: FULL_ENDPOINT,
adapterParams: {},
fetchImpl: fetchImpl as unknown as typeof fetch,
},
)
const [audioFetchUrl, audioFetchInit] = fetchImpl.mock.calls[1]
expect(audioFetchUrl).toBe(AUDIO_URL)
expect(audioFetchInit.method).toBe('GET')
const headers = init.headers as Record<string, string>
expect(headers.Authorization).toBe('Bearer sk-test')
expect(result.contentType).toBe('audio/mpeg')
expect(result.body).toBeInstanceOf(ArrayBuffer)
@@ -81,16 +52,16 @@ describe('dashscopeCosyvoiceAdapter', () => {
expect(Array.from(out)).toEqual(Array.from(audioBytes))
})
it('throws Error with .status when synthesis endpoint returns non-2xx (router maps to fallback chain)', async () => {
const fetchImpl = vi.fn()
.mockResolvedValueOnce(jsonResponse({ code: 'InvalidApiKey', message: 'bad key' }, 401))
it('throws Error with .status when unspeech returns non-2xx (router walks to next key)', async () => {
const fetchImpl = vi.fn().mockResolvedValueOnce(new Response('bad key', { status: 401 }))
await expect(
dashscopeCosyvoiceAdapter.send(
{ text: 'hi' },
{
keyPlaintext: Buffer.from('sk-test', 'utf8'),
baseURL: FULL_ENDPOINT,
baseURL: 'https://dashscope-intl.aliyuncs.com/api/v1/services/audio/tts/SpeechSynthesizer',
unspeechBaseURL: UNSPEECH,
adapterParams: {},
fetchImpl: fetchImpl as unknown as typeof fetch,
},
@@ -100,58 +71,39 @@ describe('dashscopeCosyvoiceAdapter', () => {
expect(fetchImpl).toHaveBeenCalledTimes(1)
})
it('throws Error with .status when output envelope contains no audio.url (treat as recoverable upstream error)', async () => {
// ROOT CAUSE:
//
// CosyVoice v2 non-streaming returns audio.url; if upstream returns 200
// with an empty envelope (rare — policy reject / region edge case), the
// v1 adapter silently returned "no audio data" while modeled status was
// 200. Router can't decide whether to fall back without a status. We
// attach the response status to the error so the router treats it as a
// recoverable upstream failure and walks to the next key.
const fetchImpl = vi.fn()
.mockResolvedValueOnce(jsonResponse({ output: {} }))
await expect(
dashscopeCosyvoiceAdapter.send(
{ text: 'hi' },
{
keyPlaintext: Buffer.from('sk-test', 'utf8'),
baseURL: FULL_ENDPOINT,
adapterParams: {},
fetchImpl: fetchImpl as unknown as typeof fetch,
},
),
).rejects.toMatchObject({ status: 200, message: expect.stringContaining('no audio.url') })
})
it('uses cosyvoice-v2 + longxiaochun_v2 as defaults when caller omits model / voice (regression — v1 defaults broke against the v2 endpoint)', async () => {
const fetchImpl = vi.fn()
.mockResolvedValueOnce(jsonResponse({ output: { audio: { url: AUDIO_URL } } }))
.mockResolvedValueOnce(binaryResponse(new Uint8Array([0])))
it('falls back to cosyvoice-v2 + longxiaochun_v2 when caller omits model / voice', async () => {
const fetchImpl = vi.fn().mockResolvedValueOnce(binaryResponse(new Uint8Array([0])))
await dashscopeCosyvoiceAdapter.send(
{ text: 'hi' },
{
keyPlaintext: Buffer.from('sk-test', 'utf8'),
baseURL: FULL_ENDPOINT,
baseURL: 'https://dashscope-intl.aliyuncs.com/api/v1/services/audio/tts/SpeechSynthesizer',
unspeechBaseURL: UNSPEECH,
adapterParams: {},
fetchImpl: fetchImpl as unknown as typeof fetch,
},
)
const body = JSON.parse(fetchImpl.mock.calls[0][1].body as string)
expect(body.model).toBe('cosyvoice-v2')
expect(body.input.voice).toBe('longxiaochun_v2')
expect(body.input.format).toBe('mp3')
expect(body.model).toBe('alibaba/cosyvoice-v2')
expect(body.voice).toBe('longxiaochun_v2')
expect(body.response_format).toBe('mp3')
})
it('voice catalog contains v2-suffixed ids and is non-empty', () => {
const catalog = dashscopeCosyvoiceAdapter.getVoiceCatalog()
expect(catalog.length).toBeGreaterThan(0)
expect(catalog.some(v => v.id === 'longxiaochun_v2')).toBe(true)
// No bare v1 ids should survive the migration.
expect(catalog.find(v => v.id === 'longxiaochun')).toBeUndefined()
expect(catalog.find(v => v.id === 'longxiaobai')).toBeUndefined()
it('voice catalog is proxied through unspeech (alibaba backend)', async () => {
// The catalog itself is unspeech-owned now (embedded JSON in
// unspeech/pkg/backend/alibaba/voices.go). This test only verifies the
// wire contract — fixture content is intentionally minimal so an
// unspeech-side roster change doesn't break us.
const fetchImpl = vi.fn(async () => new Response(JSON.stringify({
voices: [{ id: 'longxiaochun_v2', name: 'Longxiaochun v2' }],
}), { status: 200 })) as unknown as typeof fetch
const catalog = await dashscopeCosyvoiceAdapter.getVoiceCatalog({
adapterParams: {},
unspeechBaseURL: UNSPEECH,
fetchImpl,
})
expect(catalog).toEqual([{ id: 'longxiaochun_v2', name: 'Longxiaochun v2' }])
})
})
@@ -1,14 +1,10 @@
import type { Voice } from 'unspeech'
import type { TtsAdapter, TtsAdapterContext, TtsInput, TtsResult } from './types'
import { Buffer } from 'node:buffer'
import type { TtsAdapter, TtsAdapterContext, TtsInput, TtsResult, TtsVoiceCatalogContext } from './types'
import { errorMessageFrom } from '@moeru/std'
import cosyvoiceVoices from './voices/dashscope-cosyvoice.json' with { type: 'json' }
import { createInternalError } from '../../../utils/error'
import { createBadGatewayError, createInternalError } from '../../../utils/error'
/**
* Default DashScope cosyvoice voice id. v2 voice ids carry an explicit `_v2`
@@ -38,15 +34,7 @@ const DEFAULT_COSYVOICE_FORMAT = 'mp3'
const DEFAULT_COSYVOICE_MODEL = 'cosyvoice-v2'
/**
* Hard cap on the audio bytes we will pull from the `output.audio.url`
* follow-up fetch. CosyVoice non-streaming responses for normal TTS prompts
* stay well under this; the limit exists so a misbehaving / hijacked URL
* cannot exhaust memory on the gateway instance.
*/
const MAX_AUDIO_BYTES = 25 * 1024 * 1024
/**
* DashScope cosyvoice non-streaming REST adapter.
* DashScope cosyvoice adapter.
*
* Use when:
* - Routing a hosted TTS request to Alibaba DashScope's cosyvoice v2 / v3
@@ -78,31 +66,22 @@ export const dashscopeCosyvoiceAdapter: TtsAdapter = {
const voice = input.voice ?? DEFAULT_COSYVOICE_VOICE
const format = input.responseFormat ?? DEFAULT_COSYVOICE_FORMAT
// v2 / v3 request shape: voice / format / sample_rate live under `input`
// (NOT `parameters`, which was the v1 multimodal-generation schema). Speed
// is currently dropped on v2 non-streaming — there is no documented field
// for it on this endpoint; SSML rate is the supported substitute on
// SSML-enabled voices.
const body: Record<string, unknown> = {
model,
input: {
text: input.text,
voice,
format,
},
}
const headers: Record<string, string> = {
'Authorization': `Bearer ${ctx.keyPlaintext.toString('utf8')}`,
'Content-Type': 'application/json',
}
const body = JSON.stringify({
model: `alibaba/${model}`,
input: input.text,
voice,
response_format: format,
})
let response: Response
try {
response = await ctx.fetchImpl(ctx.baseURL, {
response = await ctx.fetchImpl(`${ctx.unspeechBaseURL.replace(/\/+$/, '')}/v1/audio/speech`, {
method: 'POST',
headers,
body: JSON.stringify(body),
headers: {
'Authorization': `Bearer ${ctx.keyPlaintext.toString('utf8')}`,
'Content-Type': 'application/json',
},
body,
signal: ctx.abortSignal,
})
}
@@ -117,121 +96,47 @@ export const dashscopeCosyvoiceAdapter: TtsAdapter = {
throw err
}
let payload: unknown
try {
payload = await response.json()
}
catch (error) {
throw createInternalError(`dashscope-cosyvoice tts response parse failed: ${errorMessageFrom(error) ?? 'unknown'}`)
}
// unspeech aggregates the WS binary frames and returns the audio buffer
// directly, so we no longer parse a JSON envelope or follow a signed URL.
const audioBytes = await response.arrayBuffer()
const contentType = response.headers.get('content-type') ?? formatToMime(format)
const audioUrl = extractCosyvoiceAudioUrl(payload)
if (!audioUrl) {
// Could be a request-mode mismatch (SSE was enabled, response shape
// changed) or an upstream-policy reject that returned 200 with an empty
// envelope. Treat as a recoverable upstream error so the router can
// try the next key / upstream.
const err = new Error(`dashscope-cosyvoice tts upstream returned no audio.url (envelope: ${stringifyEnvelope(payload)})`) as Error & { status?: number }
err.status = response.status
throw err
}
let audioBytes: ArrayBuffer
try {
audioBytes = await fetchAudioBytes(ctx.fetchImpl, audioUrl, ctx.abortSignal)
}
catch (error) {
throw createInternalError(`dashscope-cosyvoice tts audio download failed: ${errorMessageFrom(error) ?? 'unknown'}`)
}
const contentType = formatToMime(format)
return { contentType, body: audioBytes }
},
getVoiceCatalog() {
return cosyvoiceVoices as Voice[]
},
}
async getVoiceCatalog(ctx: TtsVoiceCatalogContext): Promise<Voice[]> {
// unspeech's alibaba backend embeds the catalog at build time
// (unspeech/pkg/backend/alibaba/voices.go `//go:embed voices.json`),
// so this call is in-memory on unspeech's side and only crosses a TCP
// hop. No upstream credential is required.
const url = `${ctx.unspeechBaseURL.replace(/\/+$/, '')}/api/voices?backend=alibaba`
/**
* Pulls the `output.audio.url` short-lived signed URL out of a cosyvoice
* non-streaming JSON envelope. Returns `null` if the response shape doesn't
* match (e.g. error envelope, SSE leak, or v1-style `audio.data` payload),
* so the caller can surface a clear upstream error.
*/
function extractCosyvoiceAudioUrl(payload: unknown): string | null {
if (payload == null || typeof payload !== 'object')
return null
const output = (payload as { output?: unknown }).output
if (output == null || typeof output !== 'object')
return null
const audio = (output as { audio?: unknown }).audio
if (audio == null || typeof audio !== 'object')
return null
const url = (audio as { url?: unknown }).url
if (typeof url !== 'string' || url.length === 0)
return null
return url
}
/**
* Stringify just enough of the upstream JSON to make the "no audio.url" error
* actionable, without leaking secrets like API keys. Keeps the snippet small
* so it fits inside the router's bodySnippet propagation path.
*/
function stringifyEnvelope(payload: unknown): string {
try {
return JSON.stringify(payload).slice(0, 256)
}
catch {
return '<unserializable>'
}
}
/**
* Follow-up GET against the short-lived signed URL the cosyvoice endpoint
* returns. Streamed into an ArrayBuffer with a hard size cap so a misbehaving
* URL cannot exhaust memory on the gateway.
*/
async function fetchAudioBytes(fetchImpl: typeof fetch, url: string, abortSignal: AbortSignal | undefined): Promise<ArrayBuffer> {
const audioResp = await fetchImpl(url, { method: 'GET', signal: abortSignal })
if (!audioResp.ok) {
const err = new Error(`audio.url responded ${audioResp.status}`) as Error & { status?: number }
err.status = audioResp.status
throw err
}
if (audioResp.body == null) {
throw new Error('audio.url response had no body')
}
const reader = audioResp.body.getReader()
const chunks: Uint8Array[] = []
let total = 0
let drained = false
try {
while (true) {
const { value, done } = await reader.read()
if (done) {
drained = true
break
}
total += value.length
if (total > MAX_AUDIO_BYTES)
throw new Error(`audio payload exceeded ${MAX_AUDIO_BYTES} bytes`)
chunks.push(value)
let response: Response
try {
response = await ctx.fetchImpl(url, {
method: 'GET',
headers: { Accept: 'application/json' },
signal: ctx.abortSignal,
})
}
catch (error) {
throw createBadGatewayError(`cosyvoice voices fetch failed: ${errorMessageFrom(error) ?? 'unknown'}`)
}
}
finally {
// Cancel only if we exited early; double-cancel after a clean drain is
// a no-op in spec but `reader.closed` is a Promise (always truthy), so
// we track the drain explicitly instead of testing `closed`.
if (!drained)
reader.cancel().catch(() => {})
}
const buf = Buffer.concat(chunks.map(c => Buffer.from(c)))
return buf.buffer.slice(buf.byteOffset, buf.byteOffset + buf.byteLength)
if (!response.ok) {
const text = await response.text().catch(() => '')
throw createBadGatewayError(
`cosyvoice voices upstream ${response.status}: ${text.slice(0, 256)}`,
{ lastStatusCode: response.status },
)
}
const data = await response.json() as { voices: Voice[] }
if (!Array.isArray(data.voices))
throw createBadGatewayError('cosyvoice voices upstream missing voices[]')
return data.voices
},
}
/**
@@ -1,4 +1,6 @@
import { describe, expect, it } from 'vitest'
import { Buffer } from 'node:buffer'
import { describe, expect, it, vi } from 'vitest'
import { ApiError } from '../../../utils/error'
import { getAdapter } from './index'
@@ -38,13 +40,258 @@ describe('getAdapter', () => {
}
})
it('each adapter has send and getVoiceCatalog functions', () => {
for (const id of ['azure', 'dashscope-cosyvoice', 'volcengine'] as const) {
it('every adapter delegates getVoiceCatalog to unspeech and returns the parsed list', async () => {
for (const id of ['dashscope-cosyvoice', 'volcengine'] as const) {
const adapter = getAdapter(id)
expect(typeof adapter.send).toBe('function')
expect(typeof adapter.getVoiceCatalog).toBe('function')
// U6 wires real catalogs; U5 stub returns []
expect(Array.isArray(adapter.getVoiceCatalog())).toBe(true)
const fetchImpl = vi.fn(async () => new Response(JSON.stringify({
voices: [{ id: 'v1', name: 'v1' }],
}), { status: 200 })) as unknown as typeof fetch
const voices = await adapter.getVoiceCatalog({
adapterParams: {},
unspeechBaseURL: 'http://unspeech.local',
fetchImpl,
})
expect(voices).toEqual([{ id: 'v1', name: 'v1' }])
expect(fetchImpl).toHaveBeenCalledTimes(1)
}
})
})
describe('dashscopeCosyvoiceAdapter.getVoiceCatalog', () => {
it('calls unspeech with backend=alibaba (no Bearer)', async () => {
const adapter = getAdapter('dashscope-cosyvoice')
const fetchImpl = vi.fn(async () => new Response(JSON.stringify({
voices: [{ id: 'longxiaochun_v2', name: 'Longxiaochun v2' }],
}), { status: 200 })) as unknown as typeof fetch
const voices = await adapter.getVoiceCatalog({
adapterParams: {},
unspeechBaseURL: 'http://unspeech.local',
fetchImpl,
})
expect(voices).toEqual([{ id: 'longxiaochun_v2', name: 'Longxiaochun v2' }])
const [calledUrl, init] = (fetchImpl as unknown as { mock: { calls: [string, RequestInit][] } }).mock.calls[0]
expect(calledUrl).toBe('http://unspeech.local/api/voices?backend=alibaba')
const headers = (init.headers ?? {}) as Record<string, string>
expect(headers.Authorization).toBeUndefined()
})
it('throws 502 BAD_GATEWAY when unspeech non-2xx', async () => {
const adapter = getAdapter('dashscope-cosyvoice')
const fetchImpl = vi.fn(async () => new Response('boom', { status: 502 })) as unknown as typeof fetch
await expect(adapter.getVoiceCatalog({
adapterParams: {},
unspeechBaseURL: 'http://unspeech.local',
fetchImpl,
})).rejects.toMatchObject({ statusCode: 502 })
})
})
describe('volcengineAdapter.getVoiceCatalog', () => {
it('calls unspeech with backend=volcengine and forwards adapterParams.model as ?model=', async () => {
const adapter = getAdapter('volcengine')
const fetchImpl = vi.fn(async () => new Response(JSON.stringify({
voices: [{ id: 'zh_female_x', name: 'X' }],
}), { status: 200 })) as unknown as typeof fetch
const voices = await adapter.getVoiceCatalog({
adapterParams: { model: 'seed-tts-2.0' },
unspeechBaseURL: 'http://unspeech.local',
fetchImpl,
})
expect(voices).toEqual([{ id: 'zh_female_x', name: 'X' }])
const [calledUrl] = (fetchImpl as unknown as { mock: { calls: [string, RequestInit][] } }).mock.calls[0]
expect(calledUrl).toBe('http://unspeech.local/api/voices?backend=volcengine&model=seed-tts-2.0')
})
it('omits ?model= when adapterParams.model is not set', async () => {
const adapter = getAdapter('volcengine')
const fetchImpl = vi.fn(async () => new Response(JSON.stringify({ voices: [] }), { status: 200 })) as unknown as typeof fetch
await adapter.getVoiceCatalog({
adapterParams: {},
unspeechBaseURL: 'http://unspeech.local',
fetchImpl,
})
const [calledUrl] = (fetchImpl as unknown as { mock: { calls: [string, RequestInit][] } }).mock.calls[0]
expect(calledUrl).toBe('http://unspeech.local/api/voices?backend=volcengine')
})
})
describe('azureAdapter.getVoiceCatalog', () => {
it('sends bearer + region to unspeech and returns voices on 200', async () => {
const adapter = getAdapter('azure')
const fetchImpl = vi.fn(async () => new Response(JSON.stringify({
voices: [{ id: 'en-US-AvaMultilingualNeural', name: 'Ava' }],
}), { status: 200, headers: { 'Content-Type': 'application/json' } })) as unknown as typeof fetch
const voices = await adapter.getVoiceCatalog({
keyPlaintext: Buffer.from('subscription-key-XYZ', 'utf8'),
region: 'eastasia',
adapterParams: { region: 'eastasia' },
unspeechBaseURL: 'http://unspeech.local:5933',
fetchImpl,
})
expect(voices).toEqual([{ id: 'en-US-AvaMultilingualNeural', name: 'Ava' }])
expect(fetchImpl).toHaveBeenCalledTimes(1)
const [calledUrl, init] = (fetchImpl as unknown as { mock: { calls: [string, RequestInit][] } }).mock.calls[0]
expect(calledUrl).toBe('http://unspeech.local:5933/api/voices?backend=microsoft&region=eastasia')
const headers = init.headers as Record<string, string>
expect(headers.Authorization).toBe('Bearer subscription-key-XYZ')
})
it('throws 503 AZURE_TTS_NOT_CONFIGURED when region is missing', async () => {
const adapter = getAdapter('azure')
await expect(adapter.getVoiceCatalog({
keyPlaintext: Buffer.from('k', 'utf8'),
adapterParams: {},
unspeechBaseURL: 'http://unspeech.local',
fetchImpl: vi.fn() as unknown as typeof fetch,
})).rejects.toMatchObject({ statusCode: 503, errorCode: 'AZURE_TTS_NOT_CONFIGURED' })
})
it('throws 503 AZURE_TTS_NOT_CONFIGURED when keyPlaintext is missing', async () => {
const adapter = getAdapter('azure')
await expect(adapter.getVoiceCatalog({
region: 'eastasia',
adapterParams: { region: 'eastasia' },
unspeechBaseURL: 'http://unspeech.local',
fetchImpl: vi.fn() as unknown as typeof fetch,
})).rejects.toMatchObject({ statusCode: 503, errorCode: 'AZURE_TTS_NOT_CONFIGURED' })
})
it('throws 502 BAD_GATEWAY when unspeech responds non-2xx', async () => {
const adapter = getAdapter('azure')
const fetchImpl = vi.fn(async () => new Response('upstream down', { status: 502 })) as unknown as typeof fetch
await expect(adapter.getVoiceCatalog({
keyPlaintext: Buffer.from('k', 'utf8'),
region: 'eastasia',
adapterParams: { region: 'eastasia' },
unspeechBaseURL: 'http://unspeech.local',
fetchImpl,
})).rejects.toMatchObject({ statusCode: 502 })
})
it('throws 502 BAD_GATEWAY when unspeech fetch throws', async () => {
const adapter = getAdapter('azure')
const fetchImpl = vi.fn(async () => {
throw new Error('ECONNREFUSED')
}) as unknown as typeof fetch
await expect(adapter.getVoiceCatalog({
keyPlaintext: Buffer.from('k', 'utf8'),
region: 'eastasia',
adapterParams: { region: 'eastasia' },
unspeechBaseURL: 'http://unspeech.local',
fetchImpl,
})).rejects.toMatchObject({ statusCode: 502 })
})
})
describe('azureAdapter.send', () => {
it('posts SSML to unspeech /v1/audio/speech with model=microsoft/v1 + region extra_body', async () => {
const adapter = getAdapter('azure')
const fetchImpl = vi.fn(async () => new Response(new Uint8Array([1, 2, 3]), {
status: 200,
headers: { 'content-type': 'audio/mpeg' },
})) as unknown as typeof fetch
await adapter.send(
{ text: 'hi there', voice: 'en-US-AvaMultilingualNeural', speed: 1.2 },
{
keyPlaintext: Buffer.from('azure-sub-key', 'utf8'),
baseURL: 'https://eastasia.tts.speech.microsoft.com/cognitiveservices/v1',
unspeechBaseURL: 'http://unspeech.local:5933',
adapterParams: { region: 'eastasia' },
fetchImpl,
},
)
const [calledURL, init] = (fetchImpl as unknown as { mock: { calls: [string, RequestInit][] } }).mock.calls[0]
expect(calledURL).toBe('http://unspeech.local:5933/v1/audio/speech')
const body = JSON.parse(init.body as string) as Record<string, unknown>
expect(body.model).toBe('microsoft/v1')
expect(body.voice).toBe('en-US-AvaMultilingualNeural')
expect((body.extra_body as { region?: string }).region).toBe('eastasia')
// SSML is built on our side so speed survives — verify the prosody tag is in
// the input field unspeech receives.
expect(body.input).toContain('<prosody rate=\'+20%\'>')
expect(body.input).toContain('hi there')
const headers = init.headers as Record<string, string>
expect(headers.Authorization).toBe('Bearer azure-sub-key')
})
it('throws Error with .status when unspeech non-2xx', async () => {
const adapter = getAdapter('azure')
const fetchImpl = vi.fn(async () => new Response('upstream rejected', { status: 401 })) as unknown as typeof fetch
await expect(adapter.send(
{ text: 'hi', voice: 'en-US-AvaMultilingualNeural' },
{
keyPlaintext: Buffer.from('k', 'utf8'),
baseURL: 'https://eastasia.tts.speech.microsoft.com/cognitiveservices/v1',
unspeechBaseURL: 'http://unspeech.local:5933',
adapterParams: { region: 'eastasia' },
fetchImpl,
},
)).rejects.toMatchObject({ status: 401 })
})
})
describe('volcengineAdapter.send', () => {
it('posts to unspeech with model=volcengine/<api_resource_id> and app/cluster in extra_body', async () => {
const adapter = getAdapter('volcengine')
const fetchImpl = vi.fn(async () => new Response(new Uint8Array([0x49, 0x44, 0x33]), {
status: 200,
headers: { 'content-type': 'audio/mpeg' },
})) as unknown as typeof fetch
const result = await adapter.send(
{ text: 'hi', voice: 'BV001_streaming', responseFormat: 'mp3', speed: 1.0 },
{
keyPlaintext: Buffer.from('volc-token', 'utf8'),
baseURL: 'https://openspeech.bytedance.com/api/v1/tts',
unspeechBaseURL: 'http://unspeech.local:5933',
adapterParams: { appid: 'APP-123', cluster: 'volcano_tts', model: 'seed-tts-2.0' },
fetchImpl,
},
)
const [calledURL, init] = (fetchImpl as unknown as { mock: { calls: [string, RequestInit][] } }).mock.calls[0]
expect(calledURL).toBe('http://unspeech.local:5933/v1/audio/speech')
const body = JSON.parse(init.body as string) as Record<string, any>
expect(body.model).toBe('volcengine/seed-tts-2.0')
expect(body.voice).toBe('BV001_streaming')
expect(body.response_format).toBe('mp3')
expect(body.extra_body.app).toEqual({ appid: 'APP-123', cluster: 'volcano_tts' })
expect(typeof body.extra_body.request.reqid).toBe('string')
expect(body.extra_body.request.operation).toBe('query')
// Plain Bearer — unspeech itself re-attaches as `Bearer; <token>` to the
// upstream Volcengine call.
const headers = init.headers as Record<string, string>
expect(headers.Authorization).toBe('Bearer volc-token')
expect(result.contentType).toBe('audio/mpeg')
expect(result.body).toBeInstanceOf(ArrayBuffer)
})
it('rejects when adapterParams.appid is missing', async () => {
const adapter = getAdapter('volcengine')
const fetchImpl = vi.fn() as unknown as typeof fetch
await expect(adapter.send(
{ text: 'hi' },
{
keyPlaintext: Buffer.from('k', 'utf8'),
baseURL: 'https://openspeech.bytedance.com/api/v1/tts',
unspeechBaseURL: 'http://unspeech.local:5933',
adapterParams: {},
fetchImpl,
},
)).rejects.toMatchObject({ statusCode: 500 })
})
})
+47 -5
View File
@@ -39,8 +39,20 @@ export interface TtsInput {
export interface TtsAdapterContext {
/** Decrypted upstream credential. Plain text — keep in-memory only. */
keyPlaintext: Buffer
/** Upstream HTTP base URL (no trailing slash). */
/**
* Per-upstream baseURL from `LLM_ROUTER_CONFIG.tts.upstreams[i].baseURL`.
*
* Historically the upstream provider URL (e.g.
* `https://eastasia.tts.speech.microsoft.com/cognitiveservices/v1`). After
* the Phase-B unspeech migration, adapters no longer call upstreams
* directly every `send()` forwards through unspeech REST so this field
* is informational only and adapters MAY ignore it. Kept on the context so
* existing operator configs continue to validate (the schema requires a
* non-empty string).
*/
baseURL: string
/** unspeech REST base URL (no trailing slash) — adapters POST to `<this>/v1/audio/speech`. */
unspeechBaseURL: string
/** Free-form adapter-specific params from `tts.upstreams[i].adapterParams` (e.g. Volcengine `appid` / `cluster`). */
adapterParams: Record<string, unknown>
/** Fetch implementation. Tests inject a `vi.fn()`; production passes `globalThis.fetch`. */
@@ -72,6 +84,34 @@ export interface TtsResult {
*/
export type TtsAdapterId = 'azure' | 'dashscope-cosyvoice' | 'volcengine'
/**
* Per-call context for {@link TtsAdapter.getVoiceCatalog}.
*
* `keyPlaintext` and `region` are mandatory for live providers (Azure) that
* proxy through unspeech and call the upstream provider with a subscription
* key; the router decrypts the envelope key and forwards `adapterParams.region`
* verbatim. Providers with static, credential-less catalogs (DashScope
* cosyvoice, Volcengine) ignore both fields.
*
* `unspeechBaseURL` is the configKV `UNSPEECH_REST_BASE_URL` resolved by the
* router. Passing it through the context keeps adapters free of configKV
* coupling they receive a fully-resolved URL string.
*/
export interface TtsVoiceCatalogContext {
/** Decrypted upstream credential (live providers only). */
keyPlaintext?: Buffer
/** Provider region (live providers only). */
region?: string
/** Free-form adapter-specific params (mirrors `tts.upstreams[i].adapterParams`). */
adapterParams: Record<string, unknown>
/** unspeech REST base URL, no trailing slash. */
unspeechBaseURL: string
/** Fetch implementation. Tests inject `vi.fn()`; production passes `globalThis.fetch`. */
fetchImpl: typeof fetch
/** Caller-side abort signal — propagated to the upstream fetch. */
abortSignal?: AbortSignal
}
/**
* Pure protocol translator between OpenAI-shaped `/v1/audio/speech` requests
* and one upstream TTS provider.
@@ -97,10 +137,12 @@ export interface TtsAdapter {
/** Dispatches one TTS request and resolves with the audio payload. */
send: (input: TtsInput, ctx: TtsAdapterContext) => Promise<TtsResult>
/**
* Returns the committed voice catalog for the provider.
* Returns the voice catalog for the provider.
*
* U5 stub returns `[]`; U6 wires per-provider static JSON files under
* `./voices/<id>.json`.
* Live providers (Azure) call upstream via unspeech using the supplied
* region + plaintext key. Static providers (dashscope-cosyvoice, volcengine)
* return their compiled-in JSON and ignore the context fields. Adapters
* MUST throw on upstream failure no empty-array fallback.
*/
getVoiceCatalog: () => Voice[]
getVoiceCatalog: (ctx: TtsVoiceCatalogContext) => Promise<Voice[]>
}
@@ -1,46 +0,0 @@
[
{
"id": "en-US-AvaMultilingualNeural",
"name": "Ava (Multilingual Neural)",
"description": "General-purpose multilingual neural voice",
"compatible_models": ["azure-tts"],
"formats": [
{
"name": "mp3-24khz-48kbps",
"format_code": "audio-24khz-48kbitrate-mono-mp3",
"mime_type": "audio/mpeg",
"extension": "mp3",
"bitrate": 48,
"sample_rate": 24000
}
],
"labels": { "gender": "Female", "type": "neural" },
"languages": [
{ "code": "en-US", "title": "English (US)" },
{ "code": "zh-CN", "title": "Chinese (Simplified)" }
],
"tags": ["multilingual", "neural", "general"]
},
{
"id": "en-US-AndrewMultilingualNeural",
"name": "Andrew (Multilingual Neural)",
"description": "General-purpose multilingual neural voice (male)",
"compatible_models": ["azure-tts"],
"formats": [
{
"name": "mp3-24khz-48kbps",
"format_code": "audio-24khz-48kbitrate-mono-mp3",
"mime_type": "audio/mpeg",
"extension": "mp3",
"bitrate": 48,
"sample_rate": 24000
}
],
"labels": { "gender": "Male", "type": "neural" },
"languages": [
{ "code": "en-US", "title": "English (US)" },
{ "code": "zh-CN", "title": "Chinese (Simplified)" }
],
"tags": ["multilingual", "neural", "general"]
}
]
@@ -1,265 +0,0 @@
[
{
"id": "longxiaochun_v2",
"name": "Longxiaochun (龙小淳)",
"description": "Knowing and energetic female assistant voice",
"compatible_models": ["cosyvoice-v2"],
"formats": [
{ "name": "mp3", "format_code": "mp3", "mime_type": "audio/mpeg", "extension": "mp3", "bitrate": 64, "sample_rate": 22050 }
],
"labels": { "gender": "Female", "type": "neural", "scenario": "assistant" },
"languages": [
{ "code": "zh-CN", "title": "Chinese (Simplified)" },
{ "code": "en-US", "title": "English (United States)" }
],
"tags": ["assistant", "chinese", "english", "neural"]
},
{
"id": "longxiaoxia_v2",
"name": "Longxiaoxia (龙小夏)",
"description": "Calm authoritative female assistant voice",
"compatible_models": ["cosyvoice-v2"],
"formats": [
{ "name": "mp3", "format_code": "mp3", "mime_type": "audio/mpeg", "extension": "mp3", "bitrate": 64, "sample_rate": 22050 }
],
"labels": { "gender": "Female", "type": "neural", "scenario": "assistant" },
"languages": [
{ "code": "zh-CN", "title": "Chinese (Simplified)" },
{ "code": "en-US", "title": "English (United States)" }
],
"tags": ["assistant", "chinese", "english", "neural"]
},
{
"id": "longyumi_v2",
"name": "Yumi",
"description": "Composed young female assistant voice",
"compatible_models": ["cosyvoice-v2"],
"formats": [
{ "name": "mp3", "format_code": "mp3", "mime_type": "audio/mpeg", "extension": "mp3", "bitrate": 64, "sample_rate": 22050 }
],
"labels": { "gender": "Female", "type": "neural", "scenario": "assistant" },
"languages": [
{ "code": "zh-CN", "title": "Chinese (Simplified)" },
{ "code": "en-US", "title": "English (United States)" }
],
"tags": ["assistant", "chinese", "english", "neural"]
},
{
"id": "longanli",
"name": "Longanli (龙安莉)",
"description": "Crisp confident female assistant voice",
"compatible_models": ["cosyvoice-v2"],
"formats": [
{ "name": "mp3", "format_code": "mp3", "mime_type": "audio/mpeg", "extension": "mp3", "bitrate": 64, "sample_rate": 22050 }
],
"labels": { "gender": "Female", "type": "neural", "scenario": "assistant" },
"languages": [
{ "code": "zh-CN", "title": "Chinese (Simplified)" },
{ "code": "en-US", "title": "English (United States)" }
],
"tags": ["assistant", "chinese", "english", "neural"]
},
{
"id": "longanlang",
"name": "Longanlang (龙安朗)",
"description": "Cool and clean male assistant voice",
"compatible_models": ["cosyvoice-v2"],
"formats": [
{ "name": "mp3", "format_code": "mp3", "mime_type": "audio/mpeg", "extension": "mp3", "bitrate": 64, "sample_rate": 22050 }
],
"labels": { "gender": "Male", "type": "neural", "scenario": "assistant" },
"languages": [
{ "code": "zh-CN", "title": "Chinese (Simplified)" },
{ "code": "en-US", "title": "English (United States)" }
],
"tags": ["assistant", "chinese", "english", "neural"]
},
{
"id": "longanwen",
"name": "Longanwen (龙安温)",
"description": "Elegant and graceful female assistant voice",
"compatible_models": ["cosyvoice-v2"],
"formats": [
{ "name": "mp3", "format_code": "mp3", "mime_type": "audio/mpeg", "extension": "mp3", "bitrate": 64, "sample_rate": 22050 }
],
"labels": { "gender": "Female", "type": "neural", "scenario": "assistant" },
"languages": [
{ "code": "zh-CN", "title": "Chinese (Simplified)" },
{ "code": "en-US", "title": "English (United States)" }
],
"tags": ["assistant", "chinese", "english", "neural"]
},
{
"id": "longanyun",
"name": "Longanyun (龙安昂)",
"description": "Warm at-home male assistant voice",
"compatible_models": ["cosyvoice-v2"],
"formats": [
{ "name": "mp3", "format_code": "mp3", "mime_type": "audio/mpeg", "extension": "mp3", "bitrate": 64, "sample_rate": 22050 }
],
"labels": { "gender": "Male", "type": "neural", "scenario": "assistant" },
"languages": [
{ "code": "zh-CN", "title": "Chinese (Simplified)" },
{ "code": "en-US", "title": "English (United States)" }
],
"tags": ["assistant", "chinese", "english", "neural"]
},
{
"id": "longyingmu",
"name": "Longyingmu (龙应沐)",
"description": "Polished service representative female voice",
"compatible_models": ["cosyvoice-v2"],
"formats": [
{ "name": "mp3", "format_code": "mp3", "mime_type": "audio/mpeg", "extension": "mp3", "bitrate": 64, "sample_rate": 22050 }
],
"labels": { "gender": "Female", "type": "neural", "scenario": "customer_service" },
"languages": [
{ "code": "zh-CN", "title": "Chinese (Simplified)" },
{ "code": "en-US", "title": "English (United States)" }
],
"tags": ["customer-service", "chinese", "english", "neural"]
},
{
"id": "longyingtian",
"name": "Longyingtian (龙应甜)",
"description": "Warm sweet service representative female voice",
"compatible_models": ["cosyvoice-v2"],
"formats": [
{ "name": "mp3", "format_code": "mp3", "mime_type": "audio/mpeg", "extension": "mp3", "bitrate": 64, "sample_rate": 22050 }
],
"labels": { "gender": "Female", "type": "neural", "scenario": "customer_service" },
"languages": [
{ "code": "zh-CN", "title": "Chinese (Simplified)" },
{ "code": "en-US", "title": "English (United States)" }
],
"tags": ["customer-service", "chinese", "english", "neural"]
},
{
"id": "longhuhu",
"name": "Longhuhu (龙呼呼)",
"description": "Bright young girl voice for child-facing content",
"compatible_models": ["cosyvoice-v2"],
"formats": [
{ "name": "mp3", "format_code": "mp3", "mime_type": "audio/mpeg", "extension": "mp3", "bitrate": 64, "sample_rate": 22050 }
],
"labels": { "gender": "Female", "type": "neural", "scenario": "child" },
"languages": [
{ "code": "zh-CN", "title": "Chinese (Simplified)" },
{ "code": "en-US", "title": "English (United States)" }
],
"tags": ["child", "chinese", "english", "neural"]
},
{
"id": "longniuniu",
"name": "Longniuniu (龙牛牛)",
"description": "Sunny young boy voice for child-facing content",
"compatible_models": ["cosyvoice-v2"],
"formats": [
{ "name": "mp3", "format_code": "mp3", "mime_type": "audio/mpeg", "extension": "mp3", "bitrate": 64, "sample_rate": 22050 }
],
"labels": { "gender": "Male", "type": "neural", "scenario": "child" },
"languages": [
{ "code": "zh-CN", "title": "Chinese (Simplified)" },
{ "code": "en-US", "title": "English (United States)" }
],
"tags": ["child", "chinese", "english", "neural"]
},
{
"id": "loongabby_v2",
"name": "Abby",
"description": "American English female voice",
"compatible_models": ["cosyvoice-v2"],
"formats": [
{ "name": "mp3", "format_code": "mp3", "mime_type": "audio/mpeg", "extension": "mp3", "bitrate": 64, "sample_rate": 22050 }
],
"labels": { "gender": "Female", "type": "neural", "scenario": "narration" },
"languages": [
{ "code": "en-US", "title": "English (United States)" }
],
"tags": ["english", "us", "neural"]
},
{
"id": "loongdavid_v2",
"name": "David",
"description": "American English male voice",
"compatible_models": ["cosyvoice-v2"],
"formats": [
{ "name": "mp3", "format_code": "mp3", "mime_type": "audio/mpeg", "extension": "mp3", "bitrate": 64, "sample_rate": 22050 }
],
"labels": { "gender": "Male", "type": "neural", "scenario": "narration" },
"languages": [
{ "code": "en-US", "title": "English (United States)" }
],
"tags": ["english", "us", "neural"]
},
{
"id": "loongbrian_v2",
"name": "Brian",
"description": "British English male voice",
"compatible_models": ["cosyvoice-v2"],
"formats": [
{ "name": "mp3", "format_code": "mp3", "mime_type": "audio/mpeg", "extension": "mp3", "bitrate": 64, "sample_rate": 22050 }
],
"labels": { "gender": "Male", "type": "neural", "scenario": "narration" },
"languages": [
{ "code": "en-GB", "title": "English (United Kingdom)" }
],
"tags": ["english", "uk", "neural"]
},
{
"id": "loongeva_v2",
"name": "Eva",
"description": "British English female voice",
"compatible_models": ["cosyvoice-v2"],
"formats": [
{ "name": "mp3", "format_code": "mp3", "mime_type": "audio/mpeg", "extension": "mp3", "bitrate": 64, "sample_rate": 22050 }
],
"labels": { "gender": "Female", "type": "neural", "scenario": "narration" },
"languages": [
{ "code": "en-GB", "title": "English (United Kingdom)" }
],
"tags": ["english", "uk", "neural"]
},
{
"id": "loongyuuna_v2",
"name": "Yuuna",
"description": "Genki Japanese female voice",
"compatible_models": ["cosyvoice-v2"],
"formats": [
{ "name": "mp3", "format_code": "mp3", "mime_type": "audio/mpeg", "extension": "mp3", "bitrate": 64, "sample_rate": 22050 }
],
"labels": { "gender": "Female", "type": "neural", "scenario": "narration" },
"languages": [
{ "code": "ja-JP", "title": "Japanese (Japan)" }
],
"tags": ["japanese", "neural"]
},
{
"id": "loongyuuma_v2",
"name": "Yuuma",
"description": "Steady Japanese male voice",
"compatible_models": ["cosyvoice-v2"],
"formats": [
{ "name": "mp3", "format_code": "mp3", "mime_type": "audio/mpeg", "extension": "mp3", "bitrate": 64, "sample_rate": 22050 }
],
"labels": { "gender": "Male", "type": "neural", "scenario": "narration" },
"languages": [
{ "code": "ja-JP", "title": "Japanese (Japan)" }
],
"tags": ["japanese", "neural"]
},
{
"id": "loongjihun_v2",
"name": "Jihun",
"description": "Bright Korean male voice",
"compatible_models": ["cosyvoice-v2"],
"formats": [
{ "name": "mp3", "format_code": "mp3", "mime_type": "audio/mpeg", "extension": "mp3", "bitrate": 64, "sample_rate": 22050 }
],
"labels": { "gender": "Male", "type": "neural", "scenario": "narration" },
"languages": [
{ "code": "ko-KR", "title": "Korean (Korea)" }
],
"tags": ["korean", "neural"]
}
]
@@ -1,30 +0,0 @@
[
{
"id": "BV001_streaming",
"name": "BV001 (Streaming)",
"description": "Volcengine default streaming voice",
"compatible_models": ["volcano_tts"],
"formats": [
{ "name": "mp3", "format_code": "mp3", "mime_type": "audio/mpeg", "extension": "mp3", "bitrate": 64, "sample_rate": 24000 }
],
"labels": { "gender": "Female", "type": "neural" },
"languages": [
{ "code": "zh-CN", "title": "Chinese (Simplified)" }
],
"tags": ["chinese", "streaming"]
},
{
"id": "BV002_streaming",
"name": "BV002 (Streaming)",
"description": "Volcengine male streaming voice",
"compatible_models": ["volcano_tts"],
"formats": [
{ "name": "mp3", "format_code": "mp3", "mime_type": "audio/mpeg", "extension": "mp3", "bitrate": 64, "sample_rate": 24000 }
],
"labels": { "gender": "Male", "type": "neural" },
"languages": [
{ "code": "zh-CN", "title": "Chinese (Simplified)" }
],
"tags": ["chinese", "streaming"]
}
]
@@ -1,14 +1,10 @@
import type { Voice } from 'unspeech'
import type { TtsAdapter, TtsAdapterContext, TtsInput, TtsResult } from './types'
import { Buffer } from 'node:buffer'
import type { TtsAdapter, TtsAdapterContext, TtsInput, TtsResult, TtsVoiceCatalogContext } from './types'
import { errorMessageFrom } from '@moeru/std'
import volcengineVoices from './voices/volcengine.json' with { type: 'json' }
import { createInternalError } from '../../../utils/error'
import { createBadGatewayError, createInternalError } from '../../../utils/error'
import { nanoid } from '../../../utils/id'
/**
@@ -50,55 +46,51 @@ export const volcengineAdapter: TtsAdapter = {
id: 'volcengine',
async send(input: TtsInput, ctx: TtsAdapterContext): Promise<TtsResult> {
const appid = typeof ctx.adapterParams.appid === 'string' ? ctx.adapterParams.appid : undefined
if (!appid) {
// Misconfigured upstream — the router config validation should have
// caught this earlier, but defending here keeps the adapter total.
const appid = ctx.adapterParams.appid
if (typeof appid !== 'string' || !appid)
throw createInternalError('volcengine tts: adapterParams.appid is required')
}
const cluster = typeof ctx.adapterParams.cluster === 'string'
? ctx.adapterParams.cluster
: DEFAULT_VOLCENGINE_CLUSTER
const apiResourceId = typeof ctx.adapterParams.model === 'string'
? ctx.adapterParams.model
: undefined
const voice = input.voice ?? DEFAULT_VOLCENGINE_VOICE
const encoding = input.responseFormat ?? DEFAULT_VOLCENGINE_FORMAT
const speed = input.speed ?? 1
const token = ctx.keyPlaintext.toString('utf8')
// Volcengine TTS request envelope (v1 non-streaming "query" mode).
// Per docs the `reqid` must be unique per request; we use nanoid for that.
const body = {
app: { appid, token, cluster },
user: { uid: 'airi-server' },
audio: {
voice_type: voice,
encoding,
speed_ratio: speed,
// unspeech volcengine backend (unspeech/pkg/backend/volcengine/speech.go):
// - reads token from `Authorization: Bearer <token>` (strips "Bearer "
// prefix), then re-attaches as `Bearer; <token>` to the upstream — so
// we send a normal Bearer here, NOT the `Bearer; ` form.
// - takes `app.appid`, `app.cluster`, `user.uid`, `request.reqid`,
// `audio.encoding`, `audio.speed_ratio` from `extra_body` jsonpath.
// - decodes the upstream base64 audio frame itself and returns binary.
const body = JSON.stringify({
model: apiResourceId ? `volcengine/${apiResourceId}` : 'volcengine',
input: input.text,
voice,
response_format: encoding,
extra_body: {
app: { appid, cluster },
user: { uid: 'airi-server' },
audio: { speed_ratio: speed },
request: { reqid: nanoid(), operation: 'query' },
},
request: {
reqid: nanoid(),
text: input.text,
operation: 'query',
},
}
const headers: Record<string, string> = {
// NOTICE:
// Volcengine's auth header uses `Bearer; <token>` (note the semicolon),
// not standard `Bearer <token>`. Documented at
// https://www.volcengine.com/docs/6561/79817 — sending a normal Bearer
// returns 401.
'Authorization': `Bearer; ${token}`,
'Content-Type': 'application/json',
}
})
let response: Response
try {
response = await ctx.fetchImpl(ctx.baseURL, {
response = await ctx.fetchImpl(`${ctx.unspeechBaseURL.replace(/\/+$/, '')}/v1/audio/speech`, {
method: 'POST',
headers,
body: JSON.stringify(body),
headers: {
'Authorization': `Bearer ${ctx.keyPlaintext.toString('utf8')}`,
'Content-Type': 'application/json',
},
body,
signal: ctx.abortSignal,
})
}
@@ -113,42 +105,54 @@ export const volcengineAdapter: TtsAdapter = {
throw err
}
let payload: unknown
try {
payload = await response.json()
}
catch (error) {
throw createInternalError(`volcengine tts response parse failed: ${errorMessageFrom(error) ?? 'unknown'}`)
}
const audioData = extractVolcengineAudioBase64(payload)
if (!audioData) {
const err = new Error('volcengine tts upstream returned no audio data') as Error & { status?: number }
err.status = response.status
throw err
}
const buf = Buffer.from(audioData, 'base64')
const arrayBuffer = buf.buffer.slice(buf.byteOffset, buf.byteOffset + buf.byteLength)
const contentType = encodingToMime(encoding)
// unspeech decodes the base64 audio frame and returns binary audio
// directly — no more JSON envelope on this side.
const arrayBuffer = await response.arrayBuffer()
const contentType = response.headers.get('content-type') ?? encodingToMime(encoding)
return { contentType, body: arrayBuffer }
},
getVoiceCatalog() {
return volcengineVoices as Voice[]
},
}
async getVoiceCatalog(ctx: TtsVoiceCatalogContext): Promise<Voice[]> {
// unspeech embeds the Volcengine catalog at build time
// (unspeech/pkg/backend/volcengine/voices.go), filtered server-side to
// streaming-compatible voices. Passing `model=<api_resource_id>` narrows
// further by `compatible_models` — adapterParams.model is the operator-
// configured resource id (e.g. `seed-tts-2.0`).
const url = new URL(`${ctx.unspeechBaseURL.replace(/\/+$/, '')}/api/voices`)
url.searchParams.set('backend', 'volcengine')
const apiResourceId = typeof ctx.adapterParams?.model === 'string'
? ctx.adapterParams.model
: undefined
if (apiResourceId)
url.searchParams.set('model', apiResourceId)
/**
* Reads the `data` base64 field from Volcengine's JSON response. Returns
* `null` if the response shape doesn't carry audio (e.g. error envelope).
*/
function extractVolcengineAudioBase64(payload: unknown): string | null {
if (!payload || typeof payload !== 'object')
return null
const data = (payload as Record<string, unknown>).data
return typeof data === 'string' ? data : null
let response: Response
try {
response = await ctx.fetchImpl(url.toString(), {
method: 'GET',
headers: { Accept: 'application/json' },
signal: ctx.abortSignal,
})
}
catch (error) {
throw createBadGatewayError(`volcengine voices fetch failed: ${errorMessageFrom(error) ?? 'unknown'}`)
}
if (!response.ok) {
const text = await response.text().catch(() => '')
throw createBadGatewayError(
`volcengine voices upstream ${response.status}: ${text.slice(0, 256)}`,
{ lastStatusCode: response.status },
)
}
const data = await response.json() as { voices: Voice[] }
if (!Array.isArray(data.voices))
throw createBadGatewayError('volcengine voices upstream missing voices[]')
return data.voices
},
}
/**
@@ -80,16 +80,26 @@ export function createConfigSyncSubscriber(opts: ConfigSyncSubscriberOptions): C
return
try {
const payload = JSON.parse(message) as { key?: unknown }
// Only LLM_ROUTER_CONFIG drives a router invalidation right now;
// future configKV entries with their own subscribers should route
// through additional channels, not this branch.
if (payload?.key !== 'LLM_ROUTER_CONFIG')
// LLM_ROUTER_CONFIG drives a model-config cache + voice-catalog cache
// invalidation (key rotation, model add/remove, region swap all need to
// surface immediately). UNSPEECH_REST_BASE_URL only affects the voice
// catalog cache because no other in-process structure references it.
if (payload?.key === 'LLM_ROUTER_CONFIG') {
opts.llmRouter.invalidateConfig()
void opts.llmRouter.invalidateTtsVoicesCache().catch((err) => {
opts.logger.withError(err).warn('Failed to invalidate tts voices cache on LLM_ROUTER_CONFIG change')
})
opts.gatewayMetrics?.configReload.add(1, {
source: 'pubsub',
service_instance_id: opts.instanceId,
})
return
opts.llmRouter.invalidateConfig()
opts.gatewayMetrics?.configReload.add(1, {
source: 'pubsub',
service_instance_id: opts.instanceId,
})
}
if (payload?.key === 'UNSPEECH_REST_BASE_URL') {
void opts.llmRouter.invalidateTtsVoicesCache().catch((err) => {
opts.logger.withError(err).warn('Failed to invalidate tts voices cache on UNSPEECH_REST_BASE_URL change')
})
}
}
catch (err) {
opts.logger.withError(err).warn('Failed to parse configkv:invalidate payload')
@@ -1,5 +1,7 @@
import type { Buffer } from 'node:buffer'
import type Redis from 'ioredis'
import type { GatewayMetrics } from '../../../otel'
import type { EnvelopeCrypto } from '../../../utils/envelope-crypto'
import type { ConfigKVService } from '../../adapters/config-kv'
@@ -95,6 +97,12 @@ export interface CreateLlmRouterServiceOptions {
envelopeCrypto: EnvelopeCrypto
/** OTel gateway metric bundle. `null` when OTel is disabled. */
gatewayMetrics: GatewayMetrics | null
/**
* Redis client used as the TTS voice catalog cache. Live catalogs (Azure)
* are stable but heavy; caching avoids hammering Microsoft on every voice
* picker open while keeping freshness within {@link TTS_VOICES_CACHE_TTL_S}.
*/
redis: Redis
/**
* Fetch implementation. Defaults to `globalThis.fetch`. Tests inject a
* `vi.fn` so we never touch the real network.
@@ -106,6 +114,39 @@ export interface CreateLlmRouterServiceOptions {
* @default 5_000
*/
configCacheTtlMs?: number
/**
* TTL for the Redis voice catalog cache in seconds.
* @default 21_600 (6h)
*/
ttsVoiceCacheTtlSeconds?: number
}
/**
* Default TTL for the TTS voice catalog Redis cache, per provider.
*
* - Azure (`microsoft`): live `voices/list` REST. Stable on a weekly cadence
* so 6h trades a tolerable freshness window for a big upstream call
* reduction.
* - alibaba / volcengine: unspeech embeds the catalog at build time, so the
* only way the catalog changes is unspeech redeploy. 24h is conservative
* and avoids hammering unspeech on every voice-picker open.
*
* Admin config writes invalidate every cache entry directly through
* `invalidateTtsVoicesCache`, so a key rotation or unspeech URL change
* propagates immediately and doesn't have to wait out the TTL.
*/
const TTS_VOICES_CACHE_TTL_S_BY_PROVIDER: Record<string, number> = {
'azure': 21_600,
'dashscope-cosyvoice': 86_400,
'volcengine': 86_400,
}
function ttsVoicesCacheTtl(provider: string): number {
return TTS_VOICES_CACHE_TTL_S_BY_PROVIDER[provider] ?? 21_600
}
function ttsVoicesCacheKey(provider: string, modelName: string): string {
return `tts:voices:${provider}:${modelName}`
}
/**
@@ -376,6 +417,7 @@ export function createLlmRouterService(options: CreateLlmRouterServiceOptions) {
abortSignal: AbortSignal | undefined,
perAttemptTimeoutMs: number,
fallbackHttpCodes: number[],
unspeechBaseURL: string,
onAttemptFailure: (failure: { keyId: string, status: number | 'timeout', errorMessage?: string }) => void,
): Promise<
| { kind: 'ok', contentType: string, body: ArrayBuffer | ReadableStream<Uint8Array>, attemptIndex: number }
@@ -406,6 +448,7 @@ export function createLlmRouterService(options: CreateLlmRouterServiceOptions) {
result = await adapter.send(input, {
keyPlaintext: key.plaintext,
baseURL: upstream.baseURL.replace(/\/+$/, ''),
unspeechBaseURL,
adapterParams: upstream.adapterParams ?? {},
fetchImpl,
abortSignal: attemptCtrl.signal,
@@ -503,6 +546,11 @@ export function createLlmRouterService(options: CreateLlmRouterServiceOptions) {
const defaults = slice.defaults ?? { perAttemptTimeoutMs: 30000, fullChainTimeoutMs: 60000, fallbackHttpCodes: [401, 402, 403, 429, 500, 502, 503, 504] }
const fallbackHttpCodes = slice.model.fallbackTriggers?.httpCodes ?? defaults.fallbackHttpCodes ?? [401, 402, 403, 429, 500, 502, 503, 504]
// Adapters POST to unspeech `/v1/audio/speech`; resolve the base URL once
// per request rather than per upstream attempt so a single configKV miss
// surfaces as a clean 503 before any key rotation happens.
const unspeechBaseURL = await options.configKV.getOrThrow('UNSPEECH_REST_BASE_URL')
const allFailures: Array<{ provider: string, keyId: string, status: number | 'timeout', errorMessage?: string }> = []
let triedUpstreams = 0
@@ -524,6 +572,7 @@ export function createLlmRouterService(options: CreateLlmRouterServiceOptions) {
req.abortSignal,
perAttemptTimeoutMs,
fallbackHttpCodes,
unspeechBaseURL,
(failure) => { allFailures.push({ provider: providerTag, ...failure }) },
)
@@ -566,15 +615,107 @@ export function createLlmRouterService(options: CreateLlmRouterServiceOptions) {
}
/**
* Returns the static voice catalog for one TTS provider model. Read from
* the adapter's compiled-in JSON no network call, no envelope decrypt,
* no per-upstream variation (voice lists are provider-wide).
* Returns the voice catalog for one TTS provider model.
*
* For live providers (Azure) this proxies to unspeech REST with the
* decrypted upstream key + region resolved from the model's first
* upstream. Result is cached in Redis under
* `tts:voices:<provider>:<modelName>` with a {@link TTS_VOICES_CACHE_TTL_S}
* TTL. Upstream errors are NEVER swallowed they bubble through as 5xx
* so the UI can render a real failure state instead of an empty list.
* Cache writes only happen on success.
*
* Static providers (dashscope-cosyvoice, volcengine) return their bundled
* JSON and bypass the cache (no upstream call to amortize).
*/
async function listTtsVoices(modelName: string) {
const slice = await configLoader.getModelConfig('tts', modelName)
if (slice.kind !== 'tts')
throw new Error(`Expected tts model slice for ${modelName}, got ${slice.kind}`)
return getAdapter(slice.model.provider).getVoiceCatalog()
const adapter = getAdapter(slice.model.provider)
const upstream = slice.model.upstreams[0]
const cacheKey = ttsVoicesCacheKey(slice.model.provider, modelName)
const cached = await options.redis.get(cacheKey).catch(() => null)
if (cached != null) {
try {
const parsed = JSON.parse(cached) as unknown
if (Array.isArray(parsed))
return parsed
// Malformed cache entry — drop and refetch. Don't throw; the upstream
// path is the source of truth and a stale/poisoned cache row is not a
// caller-visible failure.
}
catch {
// fallthrough — refetch
}
}
const unspeechBaseURL = await options.configKV.getOrThrow('UNSPEECH_REST_BASE_URL')
// Live providers (Azure) need the decrypted Azure subscription key + region;
// static-catalog providers (alibaba, volcengine) ignore both. The router
// decrypts unconditionally so the adapter doesn't have to know which
// category it's in — adapters that don't need creds just won't read them.
const region = typeof upstream.adapterParams?.region === 'string'
? upstream.adapterParams.region
: undefined
const keyEntry = upstream.keys[0]
const plaintext = slice.model.provider === 'azure'
? options.envelopeCrypto.decryptKey(keyEntry.ciphertext, { modelName, keyEntryId: keyEntry.id })
: undefined
try {
const voices = await adapter.getVoiceCatalog({
keyPlaintext: plaintext,
region,
adapterParams: upstream.adapterParams ?? {},
unspeechBaseURL,
fetchImpl,
})
// Cache only on success — failure responses must NOT be persisted or
// the next admin reconfigure would have to wait out the TTL even after
// fixing credentials.
const ttl = options.ttsVoiceCacheTtlSeconds ?? ttsVoicesCacheTtl(slice.model.provider)
await options.redis.set(cacheKey, JSON.stringify(voices), 'EX', ttl)
.catch((err) => {
logger.withError(err).withFields({ cacheKey }).warn('failed to write tts voices cache')
})
return voices
}
finally {
plaintext?.fill(0)
}
}
/**
* Drops every cached TTS voice catalog. Called by the configkv invalidation
* subscriber when `LLM_ROUTER_CONFIG` or `UNSPEECH_REST_BASE_URL` changes
* a key rotation or unspeech endpoint move must propagate to in-flight
* voice-picker fetches without waiting for the 6h TTL.
*/
async function invalidateTtsVoicesCache(): Promise<void> {
// SCAN avoids blocking redis on a large keyspace; production deployments
// can have voice catalogs from many models. Using a stream keeps memory
// bounded.
const stream = options.redis.scanStream({ match: 'tts:voices:*', count: 100 })
const pipeline = options.redis.pipeline()
let queued = 0
for await (const keys of stream as AsyncIterable<string[]>) {
for (const key of keys) {
pipeline.del(key)
queued += 1
}
}
if (queued > 0) {
await pipeline.exec().catch((err) => {
logger.withError(err).warn('failed to invalidate tts voices cache')
})
}
}
return {
@@ -587,6 +728,12 @@ export function createLlmRouterService(options: CreateLlmRouterServiceOptions) {
* service wrapper.
*/
invalidateConfig: configLoader.invalidate,
/**
* Flush the Redis voice catalog cache. The config-sync subscriber calls
* this when LLM_ROUTER_CONFIG or UNSPEECH_REST_BASE_URL is rotated; admin
* writes invalidate it directly so the next voice-picker fetch repopulates.
*/
invalidateTtsVoicesCache,
}
}
@@ -1,6 +1,7 @@
import type { Buffer } from 'node:buffer'
import type { Counter } from '@opentelemetry/api'
import type Redis from 'ioredis'
import type { GatewayMetrics } from '../../../../otel'
import type { ConfigKVService } from '../../../adapters/config-kv'
@@ -14,6 +15,22 @@ import { createEnvelopeCrypto } from '../../../../utils/envelope-crypto'
import { ApiError } from '../../../../utils/error'
import { createLlmRouterService } from '../router'
/**
* Minimal redis stub shared across `createLlmRouterService` tests. The router
* only touches redis through the TTS voice catalog cache, which the LLM-side
* tests never exercise every method here is a no-op vi.fn so the type
* checker is happy without spinning a real client.
*/
function makeRedisStub(): Redis {
async function* emptyScan(): AsyncGenerator<string[]> {}
return {
get: vi.fn(async () => null),
set: vi.fn(async () => 'OK'),
scanStream: vi.fn(() => emptyScan()),
pipeline: vi.fn(() => ({ del: vi.fn(), exec: vi.fn(async () => []) })),
} as unknown as Redis
}
function freshMasterKey(): Buffer {
return randomBytes(32)
}
@@ -39,7 +56,14 @@ function makeMetrics(): GatewayMetrics {
function makeConfigKV(config: RouterConfig | null): ConfigKVService {
return {
getOptional: vi.fn(async (key: string) => (key === 'LLM_ROUTER_CONFIG' ? config : null)),
getOrThrow: vi.fn(),
// routeTts reads UNSPEECH_REST_BASE_URL once per request via getOrThrow.
// LLM-side tests never invoke routeTts so the value is irrelevant; TTS
// tests need a non-empty string.
getOrThrow: vi.fn(async (key: string) => {
if (key === 'UNSPEECH_REST_BASE_URL')
return 'http://unspeech.local:5933'
return undefined
}),
get: vi.fn(),
set: vi.fn(),
} as unknown as ConfigKVService
@@ -122,6 +146,7 @@ describe('createLlmRouterService', () => {
envelopeCrypto: crypto,
gatewayMetrics: metrics,
fetchImpl,
redis: makeRedisStub(),
})
const res = await router.route({ modelName: 'openai/gpt-5-mini', body: { messages: [] } })
@@ -140,6 +165,7 @@ describe('createLlmRouterService', () => {
envelopeCrypto: crypto,
gatewayMetrics: null,
fetchImpl,
redis: makeRedisStub(),
})
await router.route({ modelName: 'openai/gpt-5-mini', body: { messages: [{ role: 'user', content: 'hi' }] } })
@@ -164,6 +190,7 @@ describe('createLlmRouterService', () => {
envelopeCrypto: crypto,
gatewayMetrics: null,
fetchImpl,
redis: makeRedisStub(),
})
await router.route({ modelName: 'openai/gpt-5-mini', body: { messages: [] } })
@@ -184,6 +211,7 @@ describe('createLlmRouterService', () => {
envelopeCrypto: crypto,
gatewayMetrics: metrics,
fetchImpl,
redis: makeRedisStub(),
})
const res = await router.route({ modelName: 'openai/gpt-5-mini', body: {} })
@@ -217,6 +245,7 @@ describe('createLlmRouterService', () => {
envelopeCrypto: crypto,
gatewayMetrics: metrics,
fetchImpl,
redis: makeRedisStub(),
})
const res = await router.route({ modelName: 'openai/gpt-5-mini', body: {} })
@@ -242,6 +271,7 @@ describe('createLlmRouterService', () => {
envelopeCrypto: crypto,
gatewayMetrics: metrics,
fetchImpl,
redis: makeRedisStub(),
})
try {
@@ -286,6 +316,7 @@ describe('createLlmRouterService', () => {
envelopeCrypto: crypto,
gatewayMetrics: null,
fetchImpl,
redis: makeRedisStub(),
})
try {
@@ -330,6 +361,7 @@ describe('createLlmRouterService', () => {
envelopeCrypto: crypto,
gatewayMetrics: metrics,
fetchImpl,
redis: makeRedisStub(),
})
await expect(router.route({ modelName: 'openai/gpt-5-mini', body: {} })).rejects.toMatchObject({ statusCode: 503, errorCode: 'SERVICE_UNAVAILABLE' })
@@ -358,6 +390,7 @@ describe('createLlmRouterService', () => {
envelopeCrypto: crypto,
gatewayMetrics: null,
fetchImpl,
redis: makeRedisStub(),
})
try {
@@ -403,6 +436,7 @@ describe('createLlmRouterService', () => {
envelopeCrypto: crypto,
gatewayMetrics: null,
fetchImpl,
redis: makeRedisStub(),
})
const res = await router.route({ modelName: 'openai/gpt-5-mini', body: {} })
@@ -432,6 +466,7 @@ describe('createLlmRouterService', () => {
envelopeCrypto: crypto,
gatewayMetrics: null,
fetchImpl,
redis: makeRedisStub(),
})
try {
@@ -456,6 +491,7 @@ describe('createLlmRouterService', () => {
envelopeCrypto: crypto,
gatewayMetrics: metrics,
fetchImpl,
redis: makeRedisStub(),
})
try {
@@ -480,6 +516,7 @@ describe('createLlmRouterService', () => {
envelopeCrypto: crypto,
gatewayMetrics: null,
fetchImpl,
redis: makeRedisStub(),
})
await expect(router.route({ modelName: 'whatever', body: {} })).rejects.toMatchObject({ statusCode: 503, errorCode: 'CONFIG_NOT_SET' })
@@ -497,6 +534,7 @@ describe('createLlmRouterService', () => {
envelopeCrypto: crypto,
gatewayMetrics: null,
fetchImpl,
redis: makeRedisStub(),
})
await expect(router.route({ modelName: 'openai/gpt-5-mini', body: {}, abortSignal: ctrl.signal })).rejects.toThrow(/client-disconnected/)
@@ -527,6 +565,7 @@ describe('createLlmRouterService', () => {
envelopeCrypto: crypto,
gatewayMetrics: null,
fetchImpl,
redis: makeRedisStub(),
})
await expect(router.route({ modelName: 'openai/gpt-5-mini', body: {}, abortSignal: ctrl.signal })).rejects.toThrow(/client-disconnected/)
@@ -544,6 +583,7 @@ describe('createLlmRouterService', () => {
envelopeCrypto: crypto,
gatewayMetrics: null,
fetchImpl,
redis: makeRedisStub(),
})
await router.route({ modelName: 'openai/gpt-5-mini', body: {} })
@@ -619,6 +659,7 @@ describe('createLlmRouterService', () => {
envelopeCrypto: crypto,
gatewayMetrics: metrics,
fetchImpl,
redis: makeRedisStub(),
})
let caught: unknown
@@ -645,7 +686,7 @@ describe('createLlmRouterService', () => {
// azure adapter wraps a fetch reject as createInternalError(500).
// The router should treat that as a fallback-eligible network failure
// and try the second key — not propagate the 500 as a final error.
const { config, crypto } = makeTtsConfig({ upstreams: [{ baseURL: 'https://az.example', keyIds: ['kA1', 'kA2'] }] })
const { config, crypto } = makeTtsConfig({ upstreams: [{ baseURL: 'https://az.example', keyIds: ['kA1', 'kA2'], adapterParams: { region: 'eastasia' } }] })
let callIdx = 0
const fetchImpl = vi.fn(async () => {
@@ -661,6 +702,7 @@ describe('createLlmRouterService', () => {
envelopeCrypto: crypto,
gatewayMetrics: metrics,
fetchImpl,
redis: makeRedisStub(),
})
const res = await router.routeTts({
@@ -679,7 +721,7 @@ describe('createLlmRouterService', () => {
// azure adapter throws `Error & { status: number }` on upstream non-2xx
// (see azure.ts:189-194). 401 is in fallbackHttpCodes so we must try
// the next key.
const { config, crypto } = makeTtsConfig({ upstreams: [{ baseURL: 'https://az.example', keyIds: ['kA1', 'kA2'] }] })
const { config, crypto } = makeTtsConfig({ upstreams: [{ baseURL: 'https://az.example', keyIds: ['kA1', 'kA2'], adapterParams: { region: 'eastasia' } }] })
let callIdx = 0
const fetchImpl = vi.fn(async () => {
@@ -695,6 +737,7 @@ describe('createLlmRouterService', () => {
envelopeCrypto: crypto,
gatewayMetrics: metrics,
fetchImpl,
redis: makeRedisStub(),
})
const res = await router.routeTts({
@@ -92,11 +92,11 @@ export const providerOfficialSpeech = defineProvider({
listModels: async (): Promise<ModelInfo[]> => {
const res = await globalThis.fetch(`${SERVER_URL}/api/v1/audio/models`, { headers: authHeaders() })
if (!res.ok)
return []
throw new Error(`audio models upstream ${res.status}: ${await res.text().catch(() => '')}`.slice(0, 256))
const data = await res.json() as { models?: { id: string, name: string }[] }
if (!Array.isArray(data.models))
return []
throw new Error('audio models upstream returned malformed body')
return data.models.map(m => ({
id: m.id,
@@ -104,10 +104,18 @@ export const providerOfficialSpeech = defineProvider({
provider: OFFICIAL_SPEECH_PROVIDER_ID,
}))
},
listVoices: async (): Promise<VoiceInfo[]> => {
const res = await globalThis.fetch(`${SERVER_URL}/api/v1/audio/voices`, { headers: authHeaders() })
listVoices: async (_config, _provider, model): Promise<VoiceInfo[]> => {
// Voice catalogs are model-scoped on the server side. Pass the active
// model through so Azure / cosyvoice / future provider voices route to
// the right adapter. `auto` defers to the server's DEFAULT_TTS_MODEL,
// but it MUST be sent explicitly — an absent `model` is treated as a
// client bug and returns 400.
const target = model && model.length > 0 ? model : 'auto'
const url = new URL(`${SERVER_URL}/api/v1/audio/voices`)
url.searchParams.set('model', target)
const res = await globalThis.fetch(url.toString(), { headers: authHeaders() })
if (!res.ok)
return []
throw new Error(`audio voices upstream ${res.status}: ${await res.text().catch(() => '')}`.slice(0, 256))
// Shape aligned with unspeech's types.ListVoicesResponse, plus the
// `recommended` field our server injects from configKV DEFAULT_TTS_VOICES.
@@ -132,7 +140,7 @@ export const providerOfficialSpeech = defineProvider({
recommendedVoicesByLocale = (data.recommended && typeof data.recommended === 'object') ? data.recommended : {}
if (!Array.isArray(data.voices))
return []
throw new Error('audio voices upstream returned malformed body')
return data.voices.map((v) => {
// unspeech surfaces gender inside labels rather than as a top-level field.
@@ -205,6 +213,11 @@ export const providerOfficialSpeechStreaming = defineProvider({
validationRequiredWhen: () => false,
extraMethods: {
listModels: async (): Promise<ModelInfo[]> => {
// Streaming TTS catalog is operator-controlled via configKV
// (`STREAMING_TTS_MODELS`). The wire `model` field uses the
// `<backend>/<api_resource_id>` shape unspeech expects (see
// `unspeech/docs/wire-protocols/audio-speech-stream-v1.md`); the server
// returns whatever the operator put there, no client-side defaults.
const res = await globalThis.fetch(`${SERVER_URL}/api/v1/audio/models/streaming`, { headers: authHeaders() })
if (!res.ok)
throw new Error(`streaming models upstream ${res.status}: ${await res.text().catch(() => '')}`.slice(0, 256))
@@ -240,7 +253,7 @@ export const providerOfficialSpeechStreaming = defineProvider({
{ headers: authHeaders() },
)
if (!res.ok)
return []
throw new Error(`streaming voices upstream ${res.status}: ${await res.text().catch(() => '')}`.slice(0, 256))
const data = await res.json() as {
voices?: {
@@ -253,7 +266,7 @@ export const providerOfficialSpeechStreaming = defineProvider({
}[]
}
if (!Array.isArray(data.voices))
return []
throw new Error('streaming voices upstream returned malformed body')
return data.voices.map((v) => {
const rawGender = typeof v.labels?.gender === 'string' ? (v.labels.gender as string) : undefined