diff --git a/packages/provider-inference/README.md b/packages/provider-inference/README.md new file mode 100644 index 000000000..d0db2472f --- /dev/null +++ b/packages/provider-inference/README.md @@ -0,0 +1,31 @@ +# Provider Inference + +`@proj-airi/provider-inference` owns runtime-neutral AIRI provider definitions. + +Use this package to list built-in providers, create provider configuration schemas, and create provider instances. The package runs in Node.js and Browser runtimes. + +Do not use this package for provider configuration persistence, Vue views, Pinia state, authentication, or Electron-native providers. Those concerns remain in `@proj-airi/stage-ui`. + +## Use + +```ts +import { getDefinedProvider, listProviders } from '@proj-airi/provider-inference' + +const provider = getDefinedProvider('openai') +const providers = listProviders() +``` + +Browser-only definitions, such as Web Speech API, load in Node.js. Their availability hook returns `false` when the required Browser capability is absent. + +Use `@proj-airi/stage-ui` for saved provider configuration, Vue settings views, Pinia state, authentication, and Electron-native providers. Do not use this package to manage those application concerns. + +## Verify + +Run the package checks from the workspace root: + +```text +pnpm -F @proj-airi/provider-inference typecheck +pnpm -F @proj-airi/provider-inference test:node +pnpm -F @proj-airi/provider-inference test:browser +pnpm -F @proj-airi/provider-inference build +``` diff --git a/packages/provider-inference/docs/provider-inference-extraction-phase-1-spec.md b/packages/provider-inference/docs/provider-inference-extraction-phase-1-spec.md new file mode 100644 index 000000000..d96ab2377 --- /dev/null +++ b/packages/provider-inference/docs/provider-inference-extraction-phase-1-spec.md @@ -0,0 +1,809 @@ +# Provider Inference Extraction Specification + +Status: Proposed for architecture review + +Phase: 1 of the provider extraction + +Target package: `@proj-airi/provider-inference` + +## 1. Purpose + +This specification defines the first provider extraction from `@proj-airi/stage-ui`. + +Phase 1 extracts provider definitions from `packages/stage-ui/src/libs/providers/providers`. + +The package must not depend on Vue, Vue Pinia, or Browser data persistence APIs. + +Browser-specific APIs are allowed when a provider needs them. + +The package must load in Node.js and Browser test environments. + +Each provider must declare or demonstrate the runtime that it can execute in. + +This phase keeps the existing provider definition design unless a change removes a forbidden dependency. + +## 2. Terms + +- A **provider definition** describes configuration, validation, capabilities, and provider creation. +- A **provider instance** performs an operation such as text generation or speech generation. +- A **cloud-api provider** calls a hosted remote service. +- A **local-api provider** calls a service that runs on the user's machine or network. +- A **local-run provider** runs a model or platform capability in the current application runtime. +- A **portable provider** passes tests in both Node.js and Browser runtimes. +- A **Browser-only provider** executes in Browser but not in Node.js. +- A **runtime adapter** supplies a capability that differs between runtimes. +- A **provider registry** stores provider definitions and returns them by identifier. + +## 3. Phase 1 Goals + +Phase 1 has these goals: + +1. Create `packages/provider-inference` as an independent workspace package. +2. Extract provider definitions that do not depend on Vue, Vue Pinia, or Browser persistence. +3. Allow Browser APIs inside provider implementations when the provider needs them. +4. Classify providers as `cloud-api`, `local-api`, or `local-run`. +5. Verify the runtime support of each provider with Node.js and Browser tests. +6. Use `vitest-plugin-fakemic` for ASR pipeline acceptance. +7. Keep the provider definition interface close to its current shape. +8. Keep UI, Pinia state, persistence, and application assembly in stage-ui. + +## 4. Scope + +### 4.1 Included + +Phase 1 includes: + +- Provider definitions under `packages/stage-ui/src/libs/providers/providers`. +- The provider registry in `providers/registry.ts`. +- Core provider types that portable or Browser provider definitions require. +- Provider validators and capability types that do not depend on Vue or Pinia. +- Provider tests that use Node.js, Browser, or explicit runtime fakes. +- Cross-runtime and Browser-specific test configuration. +- Package exports, TypeScript configuration, and the `tsdown` build configuration. + +The package can use Browser APIs such as `window`, `navigator`, `MediaStream`, `SpeechRecognition`, `FileReader`, `Worker`, and WebGPU. + +The package can use shared APIs such as `fetch`, `URL`, `AbortSignal`, `ReadableStream`, `Response`, `Blob`, and `ArrayBuffer`. + +### 4.2 Excluded + +Phase 1 excludes: + +- Vue components and Vue composables. +- Vue Pinia stores and Pinia synchronization. +- Browser persistence, including `localStorage`, IndexedDB, Cache API, and cookie access. +- Electron-only native implementations. +- Stage-ui stores, authentication, server configuration, and analytics. +- `packages/stage-ui/src/services/inference-service-providers.ts`. +- The runtime implementation under `packages/stage-ui/src/libs/inference`. +- The full `@proj-airi/testing-audio` package as a dependency of the core package. + +Phase 1 does not add a compatibility layer for the old provider import path. + +## 5. Runtime Policy + +### 5.1 Forbidden dependencies + +The new package must not import or use: + +- `vue` +- `vue-i18n` +- `pinia` +- `pinia-plugin-synced` +- `@proj-airi/stage-ui` +- Stage-ui stores or persistence modules +- `localStorage` +- `indexedDB` +- `caches` +- `document.cookie` +- Browser storage wrappers that use these APIs internally + +Type-only imports also count as dependencies when they enter the package type graph. + +### 5.2 Allowed Browser APIs + +Browser APIs are not a package-wide restriction. + +A provider can use a Browser API when its definition loads safely in Node.js and reports the correct availability there. + +A Browser-only provider must not access the Browser API during module import. + +A Browser-only provider must access the API inside a guarded availability function or inside provider execution. + +The Node.js test must load the provider definition without defining Browser globals. + +The Browser test must exercise the provider with a real or explicit fake Browser capability. + +### 5.3 Persistence rule + +Provider configuration remains in the host application. + +The provider package receives configuration values as function input. + +The provider package does not save credentials, model selection, validation state, or runtime state. + +## 6. Runtime Classification + +The classification describes provider behavior. + +It does not require a new field in `ProviderDefinition` during Phase 1. + +The classification uses the fixed `cloud` and `local` source folders. + +### 6.1 Cloud API + +Cloud API providers call hosted services. + +Expected cloud API providers include: + +- `302-ai` +- `aihubmix` +- `aliyun-nls` +- `amazon-bedrock` +- `anthropic` +- `atlascloud` +- `azure-ai-foundry` +- `azure-openai` +- `byteplus` +- `byteplus-coding-plan` +- `cerebras-ai` +- `cloudflare-workers-ai` +- `comet-api` +- `deepseek` +- `elevenlabs` +- `featherless-ai` +- `fireworks-ai` +- `google-gemini-audio-speech` +- `google-generative-ai` +- `groq` +- `mimo` +- `mimo-audio` +- `minimax` +- `minimax-speech` +- `mistral-ai` +- `modelscope` +- `moonshot-ai` +- `n1n` +- `novita-ai` +- `nvidia` +- `openai` +- `openai-audio` +- `openai-compatible` +- `openpaths` +- `openrouter-ai` +- `openrouter-audio-speech` +- `perplexity-ai` +- `together-ai` +- `unspeech` +- `volcengine-coding-plan` +- `xai` +- `zai` + +`official` is also a cloud API provider by behavior. + +Its current implementation remains excluded because it depends on Vue, Pinia, authentication, and stage-ui server state. + +Some cloud API definitions also support a user-selected local endpoint. + +`openai-compatible` is one example. + +`cloudflare-workers-ai` remains a cloud API provider. Its name does not mean that it uses a Browser Worker. + +The endpoint configuration, and not the folder name, decides the actual service location. + +### 6.2 Local API + +Local API providers call a local HTTP or WebSocket service. + +Expected local API providers include: + +- `index-tts-vllm` +- `lm-studio` +- `ollama` +- `player2-speech` +- `voicevox` + +A local API provider can run in Node.js and Browser when it uses a compatible transport. + +Browser tests must account for CORS and local network access. + +CORS is a deployment constraint, not a Vue or persistence dependency. + +The test suite must use a fake transport for deterministic provider contract tests. + +The suite can use a live local service in a separate opt-in test. + +### 6.3 Local run + +Local run providers execute in the current runtime. + +Expected local run providers include: + +- `browser-web-speech-api` +- `kokoro-local` +- `local-audio` +- `apple-speech` +- `speech-noop` + +`browser-web-speech-api` is Browser-only because Node.js does not provide Web Speech recognition. + +`kokoro-local` is Browser-oriented because it uses WebGPU and a Worker-based inference adapter. + +`local-audio` contains Browser and Electron local implementations. + +`apple-speech` uses a native Electron implementation. + +`speech-noop` has no external runtime requirement. + +The classification does not mean that every local run provider enters the portable package. + +Each implementation must pass the dependency and runtime audit before it moves. + +## 7. Folder Layout + +Phase 1 uses `cloud` and `local` as the top-level provider folders. + +This layout is a Phase 1 decision. + +### 7.1 Cloud and local folders + +```text +src/providers/ + cloud/ + local/ +``` + +The `cloud` folder contains cloud API providers. + +The `local` folder contains local API and local run providers. + +Local API and local run providers can share the `local` folder. + +The provider classification and test matrix must keep these two local categories distinct. + +Providers with configurable endpoints use their primary provider behavior for folder placement. + +For example, `openai-compatible` can call a cloud or local endpoint. + +Its folder path does not define the configured endpoint. + +Do not create `local-api` or `local-run` subfolders during Phase 1. + +Add more folders only in a later architecture-reviewed change. + +### 7.2 Layout rules + +The implementation must follow these rules: + +- The registry imports definitions without hidden side effects. +- Browser-only definitions load without breaking Node.js imports. +- The test matrix maps to each provider category. + +The folder name is an organization aid. + +The folder name is not a runtime capability contract. + +The provider definition must not gain a `kind` or `category` field only to support a folder layout. + +If a machine-readable category is required later, add an optional field in a separate architecture-reviewed change. + +## 8. Runtime Compatibility Verification + +The leader's statement is directionally correct for most cloud API and local API providers. + +The statement is not correct for every current provider. + +The source audit found these groups: + +- Most HTTP providers use `@xsai`, `fetch`, `URL`, `Response`, or `ReadableStream`. +- Most HTTP providers have no Vue or Pinia runtime import. +- Browser Web Speech requires a Browser capability. +- Kokoro local requires WebGPU and a Worker-based inference adapter. +- Apple Speech requires Electron IPC and a native plugin. +- Official providers require Vue, Pinia, authentication, and stage-ui server state. +- Aliyun NLS creates a WebSocket and needs a Node transport decision. +- Mimo audio uses `FileReader` and needs a small file-reading change for Node.js. +- Several definitions use a type-only vue-i18n import. + +The final runtime result must come from tests, not from the source classification alone. + +### 8.1 Node.js test requirements + +Run every moved provider definition in a Node.js Vitest project. + +The Node.js project must use no Browser environment. + +The Node.js project must not define `window`, `document`, `navigator`, `localStorage`, `indexedDB`, or `Worker`. + +For a dual-runtime provider, test configuration, provider creation, validators, and provider request behavior. + +For a Browser-only provider, test import safety and `isAvailableBy` in Node.js. + +For an Electron-only provider, keep the provider outside the portable package. + +### 8.2 Browser test requirements + +Run every moved provider definition in a Vitest Browser project. + +Use the repository Playwright Browser provider. + +For a dual-runtime provider, test the same public behavior as the Node.js test. + +For a Browser-only provider, test the Browser capability and provider execution. + +For Browser APIs that do not exist in headless Chromium, use an explicit fake. + +Do not use Browser persistence to prepare a unit test. + +### 8.3 Runtime result categories + +Each provider test report must use one of these results: + +- `node-and-browser`: provider execution passes in both runtimes. +- `browser-only`: provider loads in Node.js and executes in Browser. +- `node-only`: provider loads in Browser and executes in Node.js. +- `runtime-excluded`: provider remains outside the package because it needs Electron, Vue, Pinia, or persistence. + +Phase 1 accepts `browser-only` for providers such as Web Speech. + +Phase 1 expects `node-and-browser` for the HTTP cloud API and local API candidates. + +## 9. Minimal Provider Definition Changes + +The current provider definition remains the primary interface. + +Phase 1 keeps these fields and behaviors: + +- `id` +- `name` +- `description` +- `nameLocalize` +- `descriptionLocalize` +- `tasks` +- `createProviderConfig` +- `createProvider` +- `extraMethods` +- `validationRequiredWhen` +- `validators` +- `capabilities` +- `requiresCredentials` +- `configuredBy` +- `business` + +The following changes are required: + +1. Replace `ComposerTranslation` with a plain translator function type. +2. Keep the existing `{ t }` context shape for provider definitions. +3. Move the Vue `views` field to a stage-ui view registry. +4. Keep `isAvailableBy` as the existing runtime availability hook. +5. Keep Browser capability checks inside `isAvailableBy` or provider execution. + +The translator change removes the vue-i18n type dependency. + +The view change removes the Vue component dependency. + +The existing provider creation, validation, capability, and extra method contracts stay unchanged. + +Phase 1 does not add a provider category field. + +Phase 1 does not add a dependency container to every provider definition. + +An adapter is added only when a provider cannot run in both target runtimes without one. + +### 9.1 Mimo audio change + +`providers/mimo-audio/index.ts` uses `FileReader` for transcription input. + +Replace that helper with `Blob.arrayBuffer()` and a cross-runtime base64 conversion. + +This change removes a Node.js execution failure. + +It does not change the provider definition shape. + +### 9.2 Aliyun NLS change + +`providers/aliyun-nls/provider.ts` creates a global WebSocket. + +Browser WebSocket support is available in the target Browser runtime. + +Node.js needs an explicit WebSocket adapter or a Node transport. + +The Phase 1 decision is pending architecture review and the Node test result. + +The token and URL utility modules can move independently when their import graph stays separate. + +### 9.3 Official provider change + +The official provider definitions remain in stage-ui during Phase 1. + +The definitions use Vue state and Pinia chat session state. + +They also use stage-ui authentication and server configuration. + +Moving them requires a reviewed application adapter. + +That adapter is not part of the core provider definition redesign. + +## 10. Restricted Parts + +The following parts cannot enter the core package without a separate change. + +### 10.1 Provider barrel + +`providers/index.ts` imports every provider for side effects. + +It currently loads Vue, Electron, local inference, and Browser runtime modules together. + +Split it into a portable or approved-runtime barrel and a stage-ui runtime barrel. + +The root package entry point must not load excluded providers. + +### 10.2 Vue and Pinia dependencies + +The following files contain forbidden Vue or Pinia dependencies: + +- `providers/official/index.ts` +- `providers/official/shared.ts` +- `providers/apple-speech/hearing-settings.vue` +- `providers/provider-definitions.test.ts` +- `providers/voicevox/define.test.ts` +- `providers/elevenlabs/index.ts`, through type-only vue-i18n usage +- `providers/openai-audio/index.ts`, through type-only vue-i18n usage +- `providers/unspeech/index.ts`, through type-only vue-i18n usage +- `providers/local-audio/index.ts`, through type-only vue-i18n usage + +The type-only imports require replacement with the core translator type. + +The runtime Vue and Pinia imports require stage-ui ownership or a reviewed adapter. + +### 10.3 Browser persistence + +Provider definitions must not import provider stores or persistence helpers. + +`packages/stage-ui/src/stores/providers/config.ts` remains in stage-ui. + +It owns Pinia state and `useLocalStorage` persistence. + +The provider package receives the resulting configuration as input. + +The existing `packages/testing-audio` setup also uses `localStorage` for application preparation. + +That usage is valid in the integration harness but forbidden in the provider package. + +### 10.4 Apple Speech + +The following files remain in stage-ui: + +- `providers/apple-speech/index.ts` +- `providers/apple-speech/provider.ts` +- `providers/apple-speech/hearing-settings.vue` +- `providers/apple-speech/index.test.ts` + +The provider uses Electron renderer IPC and the Apple Speech Electron plugin. + +It accesses `window.electron.ipcRenderer` and native platform state. + +The view uses Vue and VueUse. + +This provider is an Electron local-run provider, not a Web and Node provider. + +### 10.5 Browser Web Speech + +`providers/browser-web-speech-api` can enter the package after the type graph is clean. + +Browser API use is allowed for this provider. + +The provider uses `window.SpeechRecognition` and Browser media behavior. + +Node.js does not provide Speech Recognition. + +The Node test must load the definition and report unavailable support. + +The Browser test must use a real or explicit fake Speech Recognition implementation. + +This provider is `browser-only`, not `runtime-excluded`. + +### 10.6 Kokoro local provider + +`providers/kokoro-local/index.ts` needs review before it moves. + +It uses `navigator.gpu` and stage-shared WebGPU state. + +It calls the stage-ui inference adapter. + +That adapter uses a Worker and local model runtime behavior. + +Browser API use is allowed, but the stage-ui inference import is not a core package dependency. + +The provider can move after the local inference capability has a package-owned or injected adapter. + +### 10.7 Local audio providers + +`providers/local-audio/index.ts` needs review before it moves. + +It contains Browser and Electron local implementations. + +It uses device memory and WebGPU checks. + +It uses stage-shared platform detection and local audio runtime modules. + +Its Browser branch can enter a later package revision after the runtime adapter is isolated. + +Its Electron branch remains in stage-ui. + +### 10.8 Mimo audio provider + +`providers/mimo-audio/index.ts` can enter the cloud API group after the `FileReader` helper changes. + +The current helper is not available in Node.js. + +The required change is local to audio input conversion. + +The provider definition interface does not need to change. + +### 10.9 Aliyun NLS provider + +The following files need a Node transport decision: + +- `providers/aliyun-nls/provider.ts` +- `providers/aliyun-nls/session.ts` +- `providers/aliyun-nls/provider.test.ts` + +The provider creates and manages WebSocket connections. + +Browser WebSocket is allowed. + +Node.js requires an injected adapter or a compatible Node implementation. + +Keep the provider outside the `node-and-browser` set until the adapter test passes. + +The following files can move as utility modules when their import graph stays independent: + +- `providers/aliyun-nls/token.ts` +- `providers/aliyun-nls/token.test.ts` +- `providers/aliyun-nls/utils.ts` + +### 10.10 NVIDIA availability dependency + +`providers/nvidia/index.ts` uses the stage-shared `isStageTamagotchi` helper. + +That helper reads Vite environment state. + +The new package cannot depend on application build environment state for Node.js execution. + +Move the availability decision to stage-ui or replace it with a reviewed runtime-neutral hook. + +The HTTP provider implementation itself remains a cloud API candidate. + +### 10.11 UI metadata + +The following files remain in stage-ui: + +- `packages/stage-ui/src/libs/providers/metadata.ts` +- `packages/stage-ui/src/libs/providers/hearing-view.ts` + +The metadata module uses Vue i18n and stage-ui schema helpers. + +The hearing view module uses Vue injection and computed references. + +These modules render or prepare UI behavior. + +They do not belong in the provider core. + +## 11. ASR Acceptance Feasibility + +Using `@proj-airi/vitest-plugin-fakemic` for ASR acceptance is feasible. + +The plugin starts a Playwright Web or Electron runtime. + +It gives Chromium a file-backed fake microphone input. + +It can therefore test the real audio path from microphone input to ASR output. + +The plugin does not replace provider unit tests. + +It does not directly test an isolated provider function. + +### 11.1 Test ownership + +Keep full audio pipeline acceptance in `@proj-airi/testing-audio`. + +That package already owns runtime startup, routes, selectors, application preparation, and audio observations. + +Update its provider setup to import definitions from `@proj-airi/provider-inference` after the extraction. + +Do not add `@proj-airi/testing-audio` as a dependency of the provider package. + +The provider package can use `@proj-airi/vitest-plugin-fakemic` as a development dependency only if it later owns a package-specific runtime harness. + +### 11.2 Audio fixtures + +The existing `testing-audio` package contains these usable WAV fixtures: + +- `cases/long-leading-silence/input.test.wav` +- `cases/single-utterance-pipeline/input.test.wav` +- `cases/two-utterance-streaming/input.test.wav` + +Copy the required WAV files into package-owned test fixtures when a package-local acceptance suite needs them. + +Copy the files without copying the `testing-audio` application harness. + +Record the source path and fixture purpose in the package test documentation. + +The current fixtures cover leading silence, one utterance, and streaming multiple utterances. + +### 11.3 ASR test layers + +Use three test layers: + +1. Node.js provider tests for request construction, response parsing, errors, and stream handling. +2. Browser provider tests for Browser APIs and Browser runtime behavior. +3. Fakemic pipeline acceptance for real microphone, VAD, ASR, and UI integration. + +Use `*.audio.test.ts` for cases that run in both configured audio runtimes. + +Use `*.audio.web.test.ts` for Browser-only cases. + +Use `*.audio.electron.test.ts` only for Electron native cases. + +Do not use `@proj-airi/testing-audio` persistence setup in Node.js provider tests. + +## 12. Test Plan + +### 12.1 Registry tests + +Test provider registration. + +Test lookup by provider identifier. + +Test deterministic provider listing. + +Test that the core entry point does not register excluded Electron or Vue providers. + +Test Browser-only definitions through the core registry without Browser globals in Node.js. + +### 12.2 Provider tests + +Test configuration creation for each moved provider. + +Test configuration validation. + +Test runtime validation with deterministic transports. + +Test provider instance creation. + +Test declared chat, transcription, and speech capabilities. + +Test model and voice discovery where the provider defines those methods. + +Use explicit fetch fakes for HTTP providers. + +Use explicit Browser capability fakes for Browser-only providers. + +Use an explicit WebSocket adapter for Aliyun NLS if it enters the package. + +### 12.3 Cross-runtime matrix + +Run Node.js and Browser projects for every moved provider. + +Record one runtime result for every provider. + +Require `node-and-browser` for cloud API and local API candidates unless the provider has a documented runtime exception. + +Require `browser-only` for Browser-specific local-run providers. + +Keep Electron-only providers outside the portable package. + +### 12.4 Import audit + +Scan source imports and declaration output for forbidden packages. + +Scan source code for Browser persistence APIs. + +The audit must reject: + +- `vue` +- `vue-i18n` +- `pinia` +- `pinia-plugin-synced` +- `localStorage` +- `indexedDB` +- `caches` +- `document.cookie` +- `@proj-airi/stage-ui` +- Stage-ui stores and persistence modules + +The audit must allow approved Browser APIs. + +The audit must also report Browser-only APIs so that the runtime test matrix includes them. + +### 12.5 Workspace checks + +Run the provider package typecheck. + +Run the provider package Node.js tests. + +Run the provider package Browser tests. + +Run the ASR acceptance project with `@proj-airi/vitest-plugin-fakemic`. + +Run the stage-ui typecheck. + +Run the repository typecheck. + +Run the repository lint command. + +## 13. Migration Rules + +The implementation must follow these rules: + +1. Move the core provider contract before moving provider definitions. +2. Replace Vue i18n types with the runtime-neutral translator type. +3. Keep the `{ t }` context shape to reduce provider changes. +4. Move Vue views to a stage-ui view registry. +5. Split the side-effect barrel into approved and runtime-specific barrels. +6. Move cloud API and local API candidates after the runtime audit. +7. Move Browser-only providers only when Node.js import safety passes. +8. Keep Electron providers in stage-ui. +9. Keep Pinia and persistence in stage-ui. +10. Update stage-ui imports in one migration. +11. Remove old provider imports after the migration. +12. Do not add a provider category field only for folder organization. +13. Do not hide a forbidden dependency behind a runtime check. + +The provider package owns provider definitions and their registry. + +Stage-ui owns UI, persistence, authentication, platform assembly, and Electron integration. + +## 14. Acceptance Criteria + +Phase 1 is complete when all conditions pass: + +- `packages/provider-inference` exists as a workspace package. +- The package root loads in Node.js without Browser globals. +- The package root loads in a Browser test project. +- The package has no Vue, Vue Pinia, or Browser persistence dependency. +- Browser-specific providers use guarded runtime access. +- Every moved provider has a Node.js and Browser test result. +- Cloud API and local API candidates pass both runtime tests, or have a documented exception. +- Browser-only local-run providers pass Browser behavior tests and Node.js import tests. +- Electron-only providers remain in stage-ui. +- `vitest-plugin-fakemic` validates the ASR pipeline. +- The ASR acceptance suite uses a fixture from the existing audio test set or a documented copy. +- The provider definition design changes only for Vue removal, view ownership, or a required runtime adapter. +- Stage-ui uses the new provider core interface. +- `inference-service-providers.ts` remains a stage-ui configuration service. +- Typecheck and lint pass for the repository. + +## 15. Later Phases + +Later phases can address: + +- Additional provider subfolders after an architecture review. +- A Node WebSocket adapter for Aliyun NLS. +- An injected local inference adapter for Kokoro. +- A split Browser and Electron implementation for local audio. +- A native speech adapter for Apple Speech. +- A reviewed application adapter for official providers. +- A separate provider configuration package for remote CRUD. +- Public package release and external package documentation. + +These changes require new interfaces or runtime adapters. + +They do not change the Phase 1 provider contract without architecture review. + +## 16. Fixed Decisions + +The following decisions are fixed for Phase 1: + +- The extraction source is `packages/stage-ui/src/libs/providers/providers`. +- The top-level provider folders are `cloud` and `local`. +- `local-api` and `local-run` share the `local` folder during Phase 1. +- Vue, Vue Pinia, and Browser persistence remain forbidden. +- Browser-specific APIs are allowed. +- `inference-service-providers.ts` does not move. +- Provider definitions keep their current shape as far as possible. +- Node.js and Browser tests both run for moved providers. +- `vitest-plugin-fakemic` validates ASR pipeline behavior. +- `testing-audio` WAV fixtures can be copied for package-owned acceptance tests. +- `testing-audio` application setup does not become a core package dependency. diff --git a/packages/provider-inference/package.json b/packages/provider-inference/package.json new file mode 100644 index 000000000..e49145fb8 --- /dev/null +++ b/packages/provider-inference/package.json @@ -0,0 +1,64 @@ +{ + "name": "@proj-airi/provider-inference", + "type": "module", + "version": "0.12.0-beta.5", + "private": true, + "description": "Runtime-neutral provider definitions for AIRI inference", + "author": { + "name": "Moeru AI Project AIRI Team", + "email": "airi@moeru.ai", + "url": "https://github.com/moeru-ai" + }, + "license": "MIT", + "repository": { + "type": "git", + "url": "https://github.com/moeru-ai/airi.git", + "directory": "packages/provider-inference" + }, + "exports": { + ".": { + "types": "./dist/index.d.mts", + "default": "./dist/index.mjs" + } + }, + "main": "./dist/index.mjs", + "types": "./dist/index.d.mts", + "files": [ + "README.md", + "dist", + "package.json" + ], + "scripts": { + "dev": "pnpm run build", + "build": "tsdown", + "typecheck": "tsc --noEmit", + "test:run": "vitest run", + "test:node": "vitest run --project node", + "test:browser": "vitest run --project browser" + }, + "dependencies": { + "@moeru/std": "catalog:", + "@proj-airi/audio": "workspace:^", + "@xsai-ext/providers": "catalog:", + "@xsai-transformers/shared": "catalog:", + "@xsai/generate-text": "catalog:", + "@xsai/model": "catalog:", + "@xsai/stream-transcription": "catalog:", + "@xsai/utils-chat": "catalog:", + "clustr": "catalog:", + "es-toolkit": "catalog:", + "is-network-error": "catalog:", + "unspeech": "catalog:xsai", + "xsschema": "catalog:", + "zod": "catalog:" + }, + "devDependencies": { + "@types/node": "catalog:", + "@vitest/browser-playwright": "catalog:vitest", + "@xsai/generate-speech": "catalog:", + "playwright": "catalog:", + "tsdown": "catalog:", + "typescript": "catalog:", + "vitest": "catalog:vitest" + } +} diff --git a/packages/provider-inference/src/index.ts b/packages/provider-inference/src/index.ts new file mode 100644 index 000000000..3b13eb969 --- /dev/null +++ b/packages/provider-inference/src/index.ts @@ -0,0 +1,23 @@ +import { portableProviderDefinitions } from './providers' +import { createProviderRegistry } from './providers/registry' + +const providerRegistry = createProviderRegistry(portableProviderDefinitions) + +/** IDs of the provider definitions included in the portable registry. */ +export type PortableProviderId = typeof portableProviderDefinitions[number]['id'] + +/** Returns the portable definition with this stable provider id. */ +export function getDefinedProvider(id: PortableProviderId) { + return providerRegistry.get(id) +} + +/** Returns portable definitions in their deterministic display order. */ +export function listProviders() { + return providerRegistry.list() +} + +export { portableProviderDefinitions } +export { createWebSpeechAPIProvider, streamWebSpeechAPITranscription } from './providers/local/browser-web-speech-api' +export * from './providers/registry' +export * from './types' +export * from './validators' diff --git a/packages/stage-ui/src/libs/providers/providers/302-ai/index.ts b/packages/provider-inference/src/providers/cloud/302-ai/index.ts similarity index 89% rename from packages/stage-ui/src/libs/providers/providers/302-ai/index.ts rename to packages/provider-inference/src/providers/cloud/302-ai/index.ts index 67a17c67a..df44ff14d 100644 --- a/packages/stage-ui/src/libs/providers/providers/302-ai/index.ts +++ b/packages/provider-inference/src/providers/cloud/302-ai/index.ts @@ -1,9 +1,9 @@ import { createChatProvider, createEmbedProvider, createModelProvider, merge } from '@xsai-ext/providers/utils' import { z } from 'zod' -import { ProviderValidationCheck } from '../../types' -import { createOpenAICompatibleValidators } from '../../validators' -import { defineProvider } from '../registry' +import { ProviderValidationCheck } from '../../../types' +import { createOpenAICompatibleValidators } from '../../../validators' +import { defineProvider } from '../../registry' const ai302ConfigSchema = z.object({ apiKey: z @@ -16,7 +16,7 @@ const ai302ConfigSchema = z.object({ type AI302Config = z.input -export const provider302AI = defineProvider({ +export const provider302AI = defineProvider({ id: '302-ai', order: 7, name: '302.AI', diff --git a/packages/stage-ui/src/libs/providers/providers/aihubmix/index.ts b/packages/provider-inference/src/providers/cloud/aihubmix/index.ts similarity index 89% rename from packages/stage-ui/src/libs/providers/providers/aihubmix/index.ts rename to packages/provider-inference/src/providers/cloud/aihubmix/index.ts index 54ab30a50..8f76c66ce 100644 --- a/packages/stage-ui/src/libs/providers/providers/aihubmix/index.ts +++ b/packages/provider-inference/src/providers/cloud/aihubmix/index.ts @@ -1,11 +1,11 @@ -import type { ChatRequestOptions } from '../../types' +import type { ChatRequestOptions } from '../../../types' import { createChatProvider, createEmbedProvider, createModelProvider, merge } from '@xsai-ext/providers/utils' import { z } from 'zod' -import { ProviderValidationCheck } from '../../types' -import { createOpenAICompatibleValidators } from '../../validators' -import { defineProvider } from '../registry' +import { ProviderValidationCheck } from '../../../types' +import { createOpenAICompatibleValidators } from '../../../validators' +import { defineProvider } from '../../registry' const aihubmixConfigSchema = z.object({ apiKey: z @@ -18,7 +18,7 @@ const aihubmixConfigSchema = z.object({ type AIHubMixConfig = z.input -export const providerAIHubMix = defineProvider({ +export const providerAIHubMix = defineProvider({ id: 'aihubmix', order: 1, name: 'AIHubMix', diff --git a/packages/stage-ui/src/libs/providers/providers/amazon-bedrock/index.test.ts b/packages/provider-inference/src/providers/cloud/amazon-bedrock/index.test.ts similarity index 100% rename from packages/stage-ui/src/libs/providers/providers/amazon-bedrock/index.test.ts rename to packages/provider-inference/src/providers/cloud/amazon-bedrock/index.test.ts diff --git a/packages/stage-ui/src/libs/providers/providers/amazon-bedrock/index.ts b/packages/provider-inference/src/providers/cloud/amazon-bedrock/index.ts similarity index 99% rename from packages/stage-ui/src/libs/providers/providers/amazon-bedrock/index.ts rename to packages/provider-inference/src/providers/cloud/amazon-bedrock/index.ts index aff7fe6f8..7ebfe4b69 100644 --- a/packages/stage-ui/src/libs/providers/providers/amazon-bedrock/index.ts +++ b/packages/provider-inference/src/providers/cloud/amazon-bedrock/index.ts @@ -1,9 +1,9 @@ -import type { ModelInfo } from '../../types' +import type { ModelInfo } from '../../../types' import { createModelProvider, merge } from '@xsai-ext/providers/utils' import { z } from 'zod' -import { defineProvider } from '../registry' +import { defineProvider } from '../../registry' const amazonBedrockConfigSchema = z.object({ apiKey: z @@ -182,7 +182,7 @@ function createBedrockConverseProvider(config: { } } -export const providerAmazonBedrock = defineProvider({ +export const providerAmazonBedrock = defineProvider({ id: 'amazon-bedrock', order: 18, name: 'Amazon Bedrock', diff --git a/packages/stage-ui/src/libs/providers/providers/anthropic/index.ts b/packages/provider-inference/src/providers/cloud/anthropic/index.ts similarity index 91% rename from packages/stage-ui/src/libs/providers/providers/anthropic/index.ts rename to packages/provider-inference/src/providers/cloud/anthropic/index.ts index 032f92b81..1b3743b01 100644 --- a/packages/stage-ui/src/libs/providers/providers/anthropic/index.ts +++ b/packages/provider-inference/src/providers/cloud/anthropic/index.ts @@ -1,11 +1,11 @@ -import type { ModelInfo } from '../../types' +import type { ModelInfo } from '../../../types' import { createChatProvider, createModelProvider, merge } from '@xsai-ext/providers/utils' import { z } from 'zod' -import { ProviderValidationCheck } from '../../types' -import { createOpenAICompatibleValidators } from '../../validators' -import { defineProvider } from '../registry' +import { ProviderValidationCheck } from '../../../types' +import { createOpenAICompatibleValidators } from '../../../validators' +import { defineProvider } from '../../registry' const anthropicConfigSchema = z.object({ apiKey: z @@ -37,7 +37,7 @@ function createAnthropic(apiKey: string, baseURL: string = 'https://api.anthropi ) } -export const providerAnthropic = defineProvider({ +export const providerAnthropic = defineProvider({ id: 'anthropic', order: 6, name: 'Anthropic', diff --git a/packages/stage-ui/src/libs/providers/providers/ark-providers.test.ts b/packages/provider-inference/src/providers/cloud/ark-providers.test.ts similarity index 86% rename from packages/stage-ui/src/libs/providers/providers/ark-providers.test.ts rename to packages/provider-inference/src/providers/cloud/ark-providers.test.ts index cdbd391a4..dc8b6af28 100644 --- a/packages/stage-ui/src/libs/providers/providers/ark-providers.test.ts +++ b/packages/provider-inference/src/providers/cloud/ark-providers.test.ts @@ -1,13 +1,16 @@ import { beforeEach, describe, expect, it, vi } from 'vitest' import { parse } from 'zod/v4/core' -const createOpenAIMock = vi.fn((apiKey: string, baseURL: string) => ({ - apiKey, - baseURL, - chat: vi.fn((model: string) => ({ +import { createProviderRegistry } from '../registry' +import { providerBytePlus } from './byteplus' +import { providerBytePlusCodingPlan } from './byteplus-coding-plan' +import { providerVolcengineCodingPlan } from './volcengine-coding-plan' + +const { createOpenAIMock } = vi.hoisted(() => ({ + createOpenAIMock: vi.fn((apiKey: string, baseURL: string) => ({ apiKey, baseURL, - model, + chat: vi.fn((model: string) => ({ apiKey, baseURL, model })), })), })) @@ -22,10 +25,7 @@ describe('ark chat provider definitions', () => { }) it('lists prefixed models and strips the prefix before chat requests', async () => { - const { getDefinedProvider } = await import('./registry') - await import('./volcengine-coding-plan') - - const provider = getDefinedProvider('volcengine-coding-plan') + const provider = createProviderRegistry([providerVolcengineCodingPlan]).get('volcengine-coding-plan') expect(provider).toBeDefined() const schema = await provider!.createProviderConfig({ t: input => input }) @@ -88,12 +88,9 @@ describe('ark chat provider definitions', () => { }) it('registers byteplus providers with the spec base urls', async () => { - const { getDefinedProvider } = await import('./registry') - await import('./byteplus') - await import('./byteplus-coding-plan') - - const byteplus = getDefinedProvider('byteplus') - const byteplusCodingPlan = getDefinedProvider('byteplus-coding-plan') + const registry = createProviderRegistry([providerBytePlus, providerBytePlusCodingPlan]) + const byteplus = registry.get('byteplus') + const byteplusCodingPlan = registry.get('byteplus-coding-plan') expect(byteplus).toBeDefined() expect(byteplusCodingPlan).toBeDefined() diff --git a/packages/stage-ui/src/libs/providers/providers/ark-shared.ts b/packages/provider-inference/src/providers/cloud/ark-shared.ts similarity index 89% rename from packages/stage-ui/src/libs/providers/providers/ark-shared.ts rename to packages/provider-inference/src/providers/cloud/ark-shared.ts index 79ab7373f..ef83facb1 100644 --- a/packages/stage-ui/src/libs/providers/providers/ark-shared.ts +++ b/packages/provider-inference/src/providers/cloud/ark-shared.ts @@ -1,11 +1,11 @@ -import type { ChatRequestOptions, ModelInfo } from '../types' +import type { ChatRequestOptions, ModelInfo } from '../../types' import { createOpenAI } from '@xsai-ext/providers/create' import { z } from 'zod' -import { ProviderValidationCheck } from '../types' -import { createOpenAICompatibleValidators } from '../validators' -import { defineProvider } from './registry' +import { ProviderValidationCheck } from '../../types' +import { createOpenAICompatibleValidators } from '../../validators' +import { defineProvider } from '../registry' const arkProviderConfigSchema = z.object({ apiKey: z @@ -21,8 +21,8 @@ interface ArkModelSpec { descriptionKey?: string } -interface ArkProviderDefinitionOptions { - id: string +interface ArkProviderDefinitionOptions { + id: TId order: number name: string nameKey: string @@ -41,7 +41,7 @@ function stripModelPrefix(modelId: string, modelPrefix: string) { : modelId } -export function createArkChatProviderDefinition(options: ArkProviderDefinitionOptions) { +export function createArkChatProviderDefinition(options: ArkProviderDefinitionOptions) { const { id, order, diff --git a/packages/stage-ui/src/libs/providers/providers/atlascloud/index.ts b/packages/provider-inference/src/providers/cloud/atlascloud/index.ts similarity index 91% rename from packages/stage-ui/src/libs/providers/providers/atlascloud/index.ts rename to packages/provider-inference/src/providers/cloud/atlascloud/index.ts index 8b750ff79..f31a38dcd 100644 --- a/packages/stage-ui/src/libs/providers/providers/atlascloud/index.ts +++ b/packages/provider-inference/src/providers/cloud/atlascloud/index.ts @@ -1,9 +1,9 @@ import { createOpenAI } from '@xsai-ext/providers/create' import { z } from 'zod' -import { ProviderValidationCheck } from '../../types' -import { createOpenAICompatibleValidators } from '../../validators' -import { defineProvider } from '../registry' +import { ProviderValidationCheck } from '../../../types' +import { createOpenAICompatibleValidators } from '../../../validators' +import { defineProvider } from '../../registry' export const ATLASCLOUD_DEFAULT_BASE_URL = 'https://api.atlascloud.ai/v1' @@ -18,7 +18,7 @@ const atlasCloudConfigSchema = z.object({ type AtlasCloudConfig = z.input -export const providerAtlasCloud = defineProvider({ +export const providerAtlasCloud = defineProvider({ id: 'atlascloud', order: 5, name: 'Atlas Cloud', diff --git a/packages/stage-ui/src/libs/providers/providers/azure-ai-foundry/index.ts b/packages/provider-inference/src/providers/cloud/azure-ai-foundry/index.ts similarity index 97% rename from packages/stage-ui/src/libs/providers/providers/azure-ai-foundry/index.ts rename to packages/provider-inference/src/providers/cloud/azure-ai-foundry/index.ts index e368dd043..ae0f5ba3c 100644 --- a/packages/stage-ui/src/libs/providers/providers/azure-ai-foundry/index.ts +++ b/packages/provider-inference/src/providers/cloud/azure-ai-foundry/index.ts @@ -1,9 +1,9 @@ -import type { ChatRequestOptions, ModelInfo, ProviderInstance } from '../../types' +import type { ChatRequestOptions, ModelInfo, ProviderInstance } from '../../../types' import { createAzure } from '@xsai-ext/providers/special/create' import { z } from 'zod' -import { defineProvider } from '../registry' +import { defineProvider } from '../../registry' const azureAIFoundryConfigSchema = z.object({ apiKey: z.string('API Key'), @@ -14,7 +14,7 @@ const azureAIFoundryConfigSchema = z.object({ type AzureAIFoundryConfig = z.input -export const providerAzureAIFoundry = defineProvider({ +export const providerAzureAIFoundry = defineProvider({ id: 'azure-ai-foundry', order: 17, name: 'Azure AI Foundry', diff --git a/packages/stage-ui/src/libs/providers/providers/azure-openai/index.ts b/packages/provider-inference/src/providers/cloud/azure-openai/index.ts similarity index 99% rename from packages/stage-ui/src/libs/providers/providers/azure-openai/index.ts rename to packages/provider-inference/src/providers/cloud/azure-openai/index.ts index 0dbecd457..80f698e4b 100644 --- a/packages/stage-ui/src/libs/providers/providers/azure-openai/index.ts +++ b/packages/provider-inference/src/providers/cloud/azure-openai/index.ts @@ -4,8 +4,8 @@ import { errorMessageFrom } from '@moeru/std' import { createOpenAI } from '@xsai-ext/providers/create' import { z } from 'zod' +import { defineProvider } from '../../registry' import { collapseToolSchemaPrimitiveAnyOf } from '../../tool-schema' -import { defineProvider } from '../registry' const AZURE_OPENAI_PROVIDER_ID = 'azure-openai' as const const DEFAULT_COMPLETIONS_API_VERSION = '2024-04-01-preview' @@ -196,7 +196,7 @@ function createAzureOpenAIFetch(config: AzureOpenAIConfig) { } } -export const providerAzureOpenAI = defineProvider({ +export const providerAzureOpenAI = defineProvider({ id: 'azure-openai', order: 2, name: 'Azure OpenAI', diff --git a/packages/stage-ui/src/libs/providers/providers/byteplus-coding-plan/index.ts b/packages/provider-inference/src/providers/cloud/byteplus-coding-plan/index.ts similarity index 100% rename from packages/stage-ui/src/libs/providers/providers/byteplus-coding-plan/index.ts rename to packages/provider-inference/src/providers/cloud/byteplus-coding-plan/index.ts diff --git a/packages/stage-ui/src/libs/providers/providers/byteplus/index.ts b/packages/provider-inference/src/providers/cloud/byteplus/index.ts similarity index 100% rename from packages/stage-ui/src/libs/providers/providers/byteplus/index.ts rename to packages/provider-inference/src/providers/cloud/byteplus/index.ts diff --git a/packages/stage-ui/src/libs/providers/providers/cerebras-ai/index.ts b/packages/provider-inference/src/providers/cloud/cerebras-ai/index.ts similarity index 88% rename from packages/stage-ui/src/libs/providers/providers/cerebras-ai/index.ts rename to packages/provider-inference/src/providers/cloud/cerebras-ai/index.ts index 2c05b8d40..3d42cdddd 100644 --- a/packages/stage-ui/src/libs/providers/providers/cerebras-ai/index.ts +++ b/packages/provider-inference/src/providers/cloud/cerebras-ai/index.ts @@ -1,11 +1,11 @@ -import type { ChatRequestOptions } from '../../types' +import type { ChatRequestOptions } from '../../../types' import { createCerebras } from '@xsai-ext/providers/create' import { z } from 'zod' -import { ProviderValidationCheck } from '../../types' -import { createOpenAICompatibleValidators } from '../../validators' -import { defineProvider } from '../registry' +import { ProviderValidationCheck } from '../../../types' +import { createOpenAICompatibleValidators } from '../../../validators' +import { defineProvider } from '../../registry' const cerebrasConfigSchema = z.object({ apiKey: z @@ -18,7 +18,7 @@ const cerebrasConfigSchema = z.object({ type CerebrasConfig = z.input -export const providerCerebrasAI = defineProvider({ +export const providerCerebrasAI = defineProvider({ id: 'cerebras-ai', name: 'Cerebras', nameLocalize: ({ t }) => t('settings.pages.providers.provider.cerebras.title'), diff --git a/packages/stage-ui/src/libs/providers/providers/cloudflare-workers-ai/index.ts b/packages/provider-inference/src/providers/cloud/cloudflare-workers-ai/index.ts similarity index 80% rename from packages/stage-ui/src/libs/providers/providers/cloudflare-workers-ai/index.ts rename to packages/provider-inference/src/providers/cloud/cloudflare-workers-ai/index.ts index 3ec0bf74e..2dbe11d69 100644 --- a/packages/stage-ui/src/libs/providers/providers/cloudflare-workers-ai/index.ts +++ b/packages/provider-inference/src/providers/cloud/cloudflare-workers-ai/index.ts @@ -1,9 +1,16 @@ import { createWorkersAI } from '@xsai-ext/providers/special/create' import { z } from 'zod' -import { defineProvider } from '../registry' +import { defineProvider } from '../../registry' -export const providerCloudflareWorkersAI = defineProvider<{ accountId: string, apiKey: string }>({ +const cloudflareWorkersAIConfigSchema = z.object({ + apiKey: z.string('API Key'), + accountId: z.string('Account ID'), +}) + +type CloudflareWorkersAIConfig = z.input + +export const providerCloudflareWorkersAI = defineProvider({ id: 'cloudflare-workers-ai', name: 'Cloudflare Workers AI', nameLocalize: ({ t }) => t('settings.pages.providers.provider.cloudflare-workers-ai.title'), @@ -13,14 +20,14 @@ export const providerCloudflareWorkersAI = defineProvider<{ accountId: string, a icon: 'i-simple-icons:cloudflare', iconColor: 'i-lobe-icons:cloudflare-color', - createProviderConfig: ({ t }) => z.object({ - apiKey: z.string().meta({ + createProviderConfig: ({ t }) => cloudflareWorkersAIConfigSchema.extend({ + apiKey: cloudflareWorkersAIConfigSchema.shape.apiKey.meta({ labelLocalized: t('settings.pages.providers.catalog.edit.config.common.fields.field.api-key.label'), descriptionLocalized: t('settings.pages.providers.catalog.edit.config.common.fields.field.api-key.description'), placeholderLocalized: t('settings.pages.providers.provider.cloudflare-workers-ai.fields.field.api-key.placeholder'), type: 'password', }), - accountId: z.string().meta({ + accountId: cloudflareWorkersAIConfigSchema.shape.accountId.meta({ labelLocalized: t('settings.pages.providers.provider.cloudflare-workers-ai.fields.field.account-id.label'), descriptionLocalized: t('settings.pages.providers.provider.cloudflare-workers-ai.fields.field.account-id.description'), placeholderLocalized: t('settings.pages.providers.provider.cloudflare-workers-ai.fields.field.account-id.placeholder'), diff --git a/packages/stage-ui/src/libs/providers/providers/comet-api/index.ts b/packages/provider-inference/src/providers/cloud/comet-api/index.ts similarity index 92% rename from packages/stage-ui/src/libs/providers/providers/comet-api/index.ts rename to packages/provider-inference/src/providers/cloud/comet-api/index.ts index f1c09e765..a17c6ba82 100644 --- a/packages/stage-ui/src/libs/providers/providers/comet-api/index.ts +++ b/packages/provider-inference/src/providers/cloud/comet-api/index.ts @@ -1,9 +1,9 @@ import { createChatProvider, createModelProvider, createSpeechProvider, createTranscriptionProvider, merge } from '@xsai-ext/providers/utils' import { z } from 'zod' -import { ProviderValidationCheck } from '../../types' -import { createOpenAICompatibleValidators } from '../../validators' -import { defineProvider } from '../registry' +import { ProviderValidationCheck } from '../../../types' +import { createOpenAICompatibleValidators } from '../../../validators' +import { defineProvider } from '../../registry' const cometApiConfigSchema = z.object({ apiKey: z @@ -16,7 +16,7 @@ const cometApiConfigSchema = z.object({ type CometApiConfig = z.input -export const providerCometAPI = defineProvider({ +export const providerCometAPI = defineProvider({ id: 'comet-api', name: 'CometAPI', nameLocalize: ({ t }) => t('settings.pages.providers.provider.comet-api.title'), @@ -56,7 +56,7 @@ export const providerCometAPI = defineProvider({ }, }) -export const providerCometAPISpeech = defineProvider({ +export const providerCometAPISpeech = defineProvider({ id: 'comet-api-speech', name: 'CometAPI Speech', nameLocalize: ({ t }) => t('settings.pages.providers.provider.comet-api.title'), @@ -75,7 +75,7 @@ export const providerCometAPISpeech = defineProvider({ validators: createOpenAICompatibleValidators({ checks: [ProviderValidationCheck.ModelList] }), }) -export const providerCometAPITranscription = defineProvider({ +export const providerCometAPITranscription = defineProvider({ id: 'comet-api-transcription', name: 'CometAPI Transcription', nameLocalize: ({ t }) => t('settings.pages.providers.provider.comet-api.title'), diff --git a/packages/stage-ui/src/libs/providers/providers/deepseek/index.test.ts b/packages/provider-inference/src/providers/cloud/deepseek/index.test.ts similarity index 99% rename from packages/stage-ui/src/libs/providers/providers/deepseek/index.test.ts rename to packages/provider-inference/src/providers/cloud/deepseek/index.test.ts index ee9732709..83d2d13c2 100644 --- a/packages/stage-ui/src/libs/providers/providers/deepseek/index.test.ts +++ b/packages/provider-inference/src/providers/cloud/deepseek/index.test.ts @@ -2,7 +2,7 @@ import type { ChatProviderWithExtraOptions, } from '@xsai-ext/providers/utils' -import type { ChatRequestOptions, ProviderInstance } from '../../types' +import type { ChatRequestOptions, ProviderInstance } from '../../../types' import { describe, expect, it } from 'vitest' diff --git a/packages/stage-ui/src/libs/providers/providers/deepseek/index.ts b/packages/provider-inference/src/providers/cloud/deepseek/index.ts similarity index 92% rename from packages/stage-ui/src/libs/providers/providers/deepseek/index.ts rename to packages/provider-inference/src/providers/cloud/deepseek/index.ts index 9b43b7c74..76b5577a2 100644 --- a/packages/stage-ui/src/libs/providers/providers/deepseek/index.ts +++ b/packages/provider-inference/src/providers/cloud/deepseek/index.ts @@ -1,11 +1,11 @@ -import type { ChatRequestOptions } from '../../types' +import type { ChatRequestOptions } from '../../../types' import { createDeepSeek } from '@xsai-ext/providers/create' import { z } from 'zod' -import { ProviderValidationCheck } from '../../types' -import { createOpenAICompatibleValidators } from '../../validators' -import { defineProvider } from '../registry' +import { ProviderValidationCheck } from '../../../types' +import { createOpenAICompatibleValidators } from '../../../validators' +import { defineProvider } from '../../registry' type DeepSeekThinkingMode = 'auto' | 'disable' | 'enable' @@ -48,7 +48,7 @@ function resolveDeepSeekThinking(modeRaw: unknown): { type: 'disabled' | 'enable } } -export const providerDeepSeek = defineProvider({ +export const providerDeepSeek = defineProvider({ id: 'deepseek', order: 4, name: 'DeepSeek', diff --git a/packages/stage-ui/src/libs/providers/providers/elevenlabs/index.ts b/packages/provider-inference/src/providers/cloud/elevenlabs/index.ts similarity index 97% rename from packages/stage-ui/src/libs/providers/providers/elevenlabs/index.ts rename to packages/provider-inference/src/providers/cloud/elevenlabs/index.ts index adec28884..214562e3b 100644 --- a/packages/stage-ui/src/libs/providers/providers/elevenlabs/index.ts +++ b/packages/provider-inference/src/providers/cloud/elevenlabs/index.ts @@ -1,11 +1,12 @@ import type { SpeechProviderWithExtraOptions } from '@xsai-ext/providers/utils' import type { ListVoicesOptions, UnElevenLabsOptions, VoiceProviderWithExtraOptions } from 'unspeech' -import type { ComposerTranslation } from 'vue-i18n' + +import type { ProviderContext } from '../../../types' import { createUnElevenLabs, listVoices } from 'unspeech' import { z } from 'zod' -import { defineProvider } from '../registry' +import { defineProvider } from '../../registry' import { models as elevenLabsModels } from './list-models' const elevenLabsConfigSchema = z.object({ @@ -27,7 +28,7 @@ function toListVoicesOptions(provider: VoiceProviderWithExtraOptions ({ + ({ t }: ProviderContext) => ({ id: 'elevenlabs:check-config', name: t('settings.pages.providers.catalog.edit.validators.openai-compatible.check-config.title'), validator: async (config: ElevenLabsConfig) => { @@ -65,7 +66,7 @@ function createElevenLabsValidators() { } } -export const providerElevenLabs = defineProvider({ +export const providerElevenLabs = defineProvider({ id: 'elevenlabs', name: 'ElevenLabs', nameLocalize: ({ t }) => t('settings.pages.providers.provider.elevenlabs.title'), diff --git a/packages/stage-ui/src/libs/providers/providers/elevenlabs/list-models.ts b/packages/provider-inference/src/providers/cloud/elevenlabs/list-models.ts similarity index 100% rename from packages/stage-ui/src/libs/providers/providers/elevenlabs/list-models.ts rename to packages/provider-inference/src/providers/cloud/elevenlabs/list-models.ts diff --git a/packages/stage-ui/src/libs/providers/providers/featherless-ai/index.ts b/packages/provider-inference/src/providers/cloud/featherless-ai/index.ts similarity index 90% rename from packages/stage-ui/src/libs/providers/providers/featherless-ai/index.ts rename to packages/provider-inference/src/providers/cloud/featherless-ai/index.ts index 34c182cdc..906eb9d39 100644 --- a/packages/stage-ui/src/libs/providers/providers/featherless-ai/index.ts +++ b/packages/provider-inference/src/providers/cloud/featherless-ai/index.ts @@ -1,11 +1,11 @@ -import type { ChatRequestOptions } from '../../types' +import type { ChatRequestOptions } from '../../../types' import { createOpenAI } from '@xsai-ext/providers/create' import { z } from 'zod' -import { ProviderValidationCheck } from '../../types' -import { createOpenAICompatibleValidators } from '../../validators' -import { defineProvider } from '../registry' +import { ProviderValidationCheck } from '../../../types' +import { createOpenAICompatibleValidators } from '../../../validators' +import { defineProvider } from '../../registry' const featherlessConfigSchema = z.object({ apiKey: z @@ -18,7 +18,7 @@ const featherlessConfigSchema = z.object({ type FeatherlessConfig = z.input -export const providerFeatherlessAI = defineProvider({ +export const providerFeatherlessAI = defineProvider({ id: 'featherless-ai', name: 'Featherless.ai', nameLocalize: ({ t }) => t('settings.pages.providers.provider.featherless.title'), diff --git a/packages/stage-ui/src/libs/providers/providers/fireworks-ai/index.ts b/packages/provider-inference/src/providers/cloud/fireworks-ai/index.ts similarity index 90% rename from packages/stage-ui/src/libs/providers/providers/fireworks-ai/index.ts rename to packages/provider-inference/src/providers/cloud/fireworks-ai/index.ts index 9c6abbeec..5c7e6daf6 100644 --- a/packages/stage-ui/src/libs/providers/providers/fireworks-ai/index.ts +++ b/packages/provider-inference/src/providers/cloud/fireworks-ai/index.ts @@ -1,11 +1,11 @@ -import type { ChatRequestOptions } from '../../types' +import type { ChatRequestOptions } from '../../../types' import { createFireworks } from '@xsai-ext/providers/create' import { z } from 'zod' -import { ProviderValidationCheck } from '../../types' -import { createOpenAICompatibleValidators } from '../../validators' -import { defineProvider } from '../registry' +import { ProviderValidationCheck } from '../../../types' +import { createOpenAICompatibleValidators } from '../../../validators' +import { defineProvider } from '../../registry' const fireworksConfigSchema = z.object({ apiKey: z @@ -18,7 +18,7 @@ const fireworksConfigSchema = z.object({ type FireworksConfig = z.input -export const providerFireworksAI = defineProvider({ +export const providerFireworksAI = defineProvider({ id: 'fireworks-ai', name: 'Fireworks.ai', nameLocalize: ({ t }) => t('settings.pages.providers.provider.fireworks.title'), diff --git a/packages/stage-ui/src/libs/providers/providers/google-gemini-audio-speech/index.ts b/packages/provider-inference/src/providers/cloud/google-gemini-audio-speech/index.ts similarity index 98% rename from packages/stage-ui/src/libs/providers/providers/google-gemini-audio-speech/index.ts rename to packages/provider-inference/src/providers/cloud/google-gemini-audio-speech/index.ts index c2bb13260..1848d6ef6 100644 --- a/packages/stage-ui/src/libs/providers/providers/google-gemini-audio-speech/index.ts +++ b/packages/provider-inference/src/providers/cloud/google-gemini-audio-speech/index.ts @@ -1,7 +1,7 @@ import { toWavFromPCM16 } from '@proj-airi/audio/encoding' import { z } from 'zod' -import { defineProvider } from '../registry' +import { defineProvider } from '../../registry' const DEFAULT_BASE_URL = 'https://generativelanguage.googleapis.com/v1beta/' const DEFAULT_MODEL = 'gemini-2.5-flash-preview-tts' @@ -112,7 +112,7 @@ function createAudioFetch(apiKey: string, baseUrl: string) { } } -export const providerGoogleGeminiAudioSpeech = defineProvider({ +export const providerGoogleGeminiAudioSpeech = defineProvider({ id: 'google-gemini-audio-speech', name: 'Google Gemini', nameLocalize: ({ t }) => t('settings.pages.providers.provider.google-gemini-audio-speech.title'), diff --git a/packages/stage-ui/src/libs/providers/providers/google-generative-ai/index.ts b/packages/provider-inference/src/providers/cloud/google-generative-ai/index.ts similarity index 90% rename from packages/stage-ui/src/libs/providers/providers/google-generative-ai/index.ts rename to packages/provider-inference/src/providers/cloud/google-generative-ai/index.ts index 6f5ca046d..3f61631d1 100644 --- a/packages/stage-ui/src/libs/providers/providers/google-generative-ai/index.ts +++ b/packages/provider-inference/src/providers/cloud/google-generative-ai/index.ts @@ -1,11 +1,11 @@ -import type { ChatRequestOptions } from '../../types' +import type { ChatRequestOptions } from '../../../types' import { createGoogleGenerativeAI } from '@xsai-ext/providers/create' import { z } from 'zod' -import { ProviderValidationCheck } from '../../types' -import { createOpenAICompatibleValidators } from '../../validators' -import { defineProvider } from '../registry' +import { ProviderValidationCheck } from '../../../types' +import { createOpenAICompatibleValidators } from '../../../validators' +import { defineProvider } from '../../registry' const googleGenerativeConfigSchema = z.object({ apiKey: z @@ -18,7 +18,7 @@ const googleGenerativeConfigSchema = z.object({ type GoogleGenerativeConfig = z.input -export const providerGoogleGenerativeAI = defineProvider({ +export const providerGoogleGenerativeAI = defineProvider({ id: 'google-generative-ai', order: 8, name: 'Google Gemini', diff --git a/packages/stage-ui/src/libs/providers/providers/groq/index.ts b/packages/provider-inference/src/providers/cloud/groq/index.ts similarity index 88% rename from packages/stage-ui/src/libs/providers/providers/groq/index.ts rename to packages/provider-inference/src/providers/cloud/groq/index.ts index 221542ecb..276f0e846 100644 --- a/packages/stage-ui/src/libs/providers/providers/groq/index.ts +++ b/packages/provider-inference/src/providers/cloud/groq/index.ts @@ -1,11 +1,11 @@ -import type { ChatRequestOptions } from '../../types' +import type { ChatRequestOptions } from '../../../types' import { createOpenAI } from '@xsai-ext/providers/create' import { z } from 'zod' -import { ProviderValidationCheck } from '../../types' -import { createOpenAICompatibleValidators } from '../../validators' -import { defineProvider } from '../registry' +import { ProviderValidationCheck } from '../../../types' +import { createOpenAICompatibleValidators } from '../../../validators' +import { defineProvider } from '../../registry' const groqConfigSchema = z.object({ apiKey: z @@ -18,7 +18,7 @@ const groqConfigSchema = z.object({ type GroqConfig = z.input -export const providerGroq = defineProvider({ +export const providerGroq = defineProvider({ id: 'groq', name: 'Groq', nameLocalize: ({ t }) => t('settings.pages.providers.provider.groq.title'), diff --git a/packages/stage-ui/src/libs/providers/providers/mimo-audio/index.ts b/packages/provider-inference/src/providers/cloud/mimo-audio/index.ts similarity index 95% rename from packages/stage-ui/src/libs/providers/providers/mimo-audio/index.ts rename to packages/provider-inference/src/providers/cloud/mimo-audio/index.ts index b7b0b0685..0510029ba 100644 --- a/packages/stage-ui/src/libs/providers/providers/mimo-audio/index.ts +++ b/packages/provider-inference/src/providers/cloud/mimo-audio/index.ts @@ -1,6 +1,8 @@ +import type { ProviderContext } from '../../../types' + import { z } from 'zod' -import { defineProvider } from '../registry' +import { defineProvider } from '../../registry' const mimoSpeechConfigSchema = z.object({ apiKey: z.string(), @@ -29,7 +31,7 @@ function normalizeBaseUrl(baseUrl: string | undefined) { function createMimoValidators(id: string) { return { validateConfig: [ - ({ t }: { t: (key: string) => string }) => ({ + ({ t }: ProviderContext) => ({ id: `${id}:check-config`, name: t('settings.pages.providers.catalog.edit.validators.openai-compatible.check-config.title'), validator: async (config: TConfig) => { @@ -147,13 +149,14 @@ function audioFormatFromDataUri(dataUri: string) { return 'wav' } -function readBlobAsDataUri(file: Blob) { - return new Promise((resolve, reject) => { - const reader = new FileReader() - reader.onload = () => resolve(String(reader.result)) - reader.onerror = () => reject(new Error('Failed to read audio file')) - reader.readAsDataURL(file) - }) +async function readBlobAsDataUri(file: Blob): Promise { + const bytes = new Uint8Array(await file.arrayBuffer()) + let binary = '' + + for (const byte of bytes) + binary += String.fromCharCode(byte) + + return `data:${file.type || 'audio/wav'};base64,${btoa(binary)}` } function createMimoTranscriptionProvider(config: MimoTranscriptionConfig) { @@ -206,7 +209,7 @@ function createMimoTranscriptionProvider(config: MimoTranscriptionConfig) { } } -export const providerMimoAudioSpeech = defineProvider({ +export const providerMimoAudioSpeech = defineProvider({ id: 'mimo-audio-speech', name: 'Xiaomi MiMo', nameLocalize: ({ t }) => t('settings.pages.providers.provider.mimo.title'), @@ -238,7 +241,7 @@ export const providerMimoAudioSpeech = defineProvider({ }, }) -export const providerMimoAudioTranscription = defineProvider({ +export const providerMimoAudioTranscription = defineProvider({ id: 'mimo-audio-transcription', name: 'Xiaomi MiMo', nameLocalize: ({ t }) => t('settings.pages.providers.provider.mimo.title'), diff --git a/packages/stage-ui/src/libs/providers/providers/mimo/index.ts b/packages/provider-inference/src/providers/cloud/mimo/index.ts similarity index 88% rename from packages/stage-ui/src/libs/providers/providers/mimo/index.ts rename to packages/provider-inference/src/providers/cloud/mimo/index.ts index a0baa1dd5..647d89e1f 100644 --- a/packages/stage-ui/src/libs/providers/providers/mimo/index.ts +++ b/packages/provider-inference/src/providers/cloud/mimo/index.ts @@ -1,11 +1,11 @@ -import type { ChatRequestOptions } from '../../types' +import type { ChatRequestOptions } from '../../../types' import { createXiaomi } from '@xsai-ext/providers/create' import { z } from 'zod' -import { ProviderValidationCheck } from '../../types' -import { createOpenAICompatibleValidators } from '../../validators' -import { defineProvider } from '../registry' +import { ProviderValidationCheck } from '../../../types' +import { createOpenAICompatibleValidators } from '../../../validators' +import { defineProvider } from '../../registry' const mimoConfigSchema = z.object({ apiKey: z @@ -18,7 +18,7 @@ const mimoConfigSchema = z.object({ type MimoConfig = z.input -export const providerMimo = defineProvider({ +export const providerMimo = defineProvider({ id: 'mimo', order: 4, name: 'Xiaomi MiMo', diff --git a/packages/stage-ui/src/libs/providers/providers/minimax-speech/index.ts b/packages/provider-inference/src/providers/cloud/minimax-speech/index.ts similarity index 98% rename from packages/stage-ui/src/libs/providers/providers/minimax-speech/index.ts rename to packages/provider-inference/src/providers/cloud/minimax-speech/index.ts index f15062c4f..9f827e1db 100644 --- a/packages/stage-ui/src/libs/providers/providers/minimax-speech/index.ts +++ b/packages/provider-inference/src/providers/cloud/minimax-speech/index.ts @@ -1,6 +1,6 @@ import { z } from 'zod' -import { defineProvider } from '../registry' +import { defineProvider } from '../../registry' const minimaxSpeechConfigSchema = z.object({ apiKey: z.string(), @@ -9,7 +9,7 @@ const minimaxSpeechConfigSchema = z.object({ type MinimaxSpeechConfig = z.input -export const providerMinimaxSpeech = defineProvider({ +export const providerMinimaxSpeech = defineProvider({ id: 'minimax-speech', name: 'MiniMax Speech', nameLocalize: ({ t }) => t('settings.pages.providers.provider.minimax-speech.title'), diff --git a/packages/stage-ui/src/libs/providers/providers/minimax/index.ts b/packages/provider-inference/src/providers/cloud/minimax/index.ts similarity index 93% rename from packages/stage-ui/src/libs/providers/providers/minimax/index.ts rename to packages/provider-inference/src/providers/cloud/minimax/index.ts index d7e6d9d07..a50021888 100644 --- a/packages/stage-ui/src/libs/providers/providers/minimax/index.ts +++ b/packages/provider-inference/src/providers/cloud/minimax/index.ts @@ -1,9 +1,9 @@ import { createMinimax, createMinimaxCn } from '@xsai-ext/providers/create' import { z } from 'zod' -import { ProviderValidationCheck } from '../../types' -import { createOpenAICompatibleValidators } from '../../validators' -import { defineProvider } from '../registry' +import { ProviderValidationCheck } from '../../../types' +import { createOpenAICompatibleValidators } from '../../../validators' +import { defineProvider } from '../../registry' const minimaxCnConfigSchema = z.object({ apiKey: z @@ -27,7 +27,7 @@ const minimaxGlobalConfigSchema = z.object({ type MinimaxGlobalConfig = z.input -export const providerMinimax = defineProvider({ +export const providerMinimax = defineProvider({ id: 'minimax', name: 'MiniMax', nameLocalize: ({ t }) => t('settings.pages.providers.provider.minimax.title'), @@ -64,7 +64,7 @@ export const providerMinimax = defineProvider({ }, }) -export const providerMinimaxGlobal = defineProvider({ +export const providerMinimaxGlobal = defineProvider({ id: 'minimax-global', name: 'MiniMax Global', nameLocalize: ({ t }) => t('settings.pages.providers.provider.minimax-global.title'), diff --git a/packages/stage-ui/src/libs/providers/providers/mistral-ai/index.ts b/packages/provider-inference/src/providers/cloud/mistral-ai/index.ts similarity index 88% rename from packages/stage-ui/src/libs/providers/providers/mistral-ai/index.ts rename to packages/provider-inference/src/providers/cloud/mistral-ai/index.ts index 80893e661..5c8580986 100644 --- a/packages/stage-ui/src/libs/providers/providers/mistral-ai/index.ts +++ b/packages/provider-inference/src/providers/cloud/mistral-ai/index.ts @@ -1,9 +1,9 @@ import { createMistral } from '@xsai-ext/providers/create' import { z } from 'zod' -import { ProviderValidationCheck } from '../../types' -import { createOpenAICompatibleValidators } from '../../validators' -import { defineProvider } from '../registry' +import { ProviderValidationCheck } from '../../../types' +import { createOpenAICompatibleValidators } from '../../../validators' +import { defineProvider } from '../../registry' const mistralConfigSchema = z.object({ apiKey: z @@ -16,7 +16,7 @@ const mistralConfigSchema = z.object({ type MistralConfig = z.input -export const providerMistralAI = defineProvider({ +export const providerMistralAI = defineProvider({ id: 'mistral-ai', name: 'Mistral', nameLocalize: ({ t }) => t('settings.pages.providers.provider.mistral.title'), diff --git a/packages/stage-ui/src/libs/providers/providers/modelscope/index.ts b/packages/provider-inference/src/providers/cloud/modelscope/index.ts similarity index 91% rename from packages/stage-ui/src/libs/providers/providers/modelscope/index.ts rename to packages/provider-inference/src/providers/cloud/modelscope/index.ts index 4703e1cfa..963f1e5ef 100644 --- a/packages/stage-ui/src/libs/providers/providers/modelscope/index.ts +++ b/packages/provider-inference/src/providers/cloud/modelscope/index.ts @@ -1,9 +1,9 @@ import { createOpenAI } from '@xsai-ext/providers/create' import { z } from 'zod' -import { ProviderValidationCheck } from '../../types' -import { createOpenAICompatibleValidators } from '../../validators' -import { defineProvider } from '../registry' +import { ProviderValidationCheck } from '../../../types' +import { createOpenAICompatibleValidators } from '../../../validators' +import { defineProvider } from '../../registry' const modelscopeConfigSchema = z.object({ apiKey: z @@ -16,7 +16,7 @@ const modelscopeConfigSchema = z.object({ type ModelscopeConfig = z.input -export const providerModelScope = defineProvider({ +export const providerModelScope = defineProvider({ id: 'modelscope', name: 'ModelScope', nameLocalize: ({ t }) => t('settings.pages.providers.provider.modelscope.title'), diff --git a/packages/stage-ui/src/libs/providers/providers/moonshot-ai/index.ts b/packages/provider-inference/src/providers/cloud/moonshot-ai/index.ts similarity index 87% rename from packages/stage-ui/src/libs/providers/providers/moonshot-ai/index.ts rename to packages/provider-inference/src/providers/cloud/moonshot-ai/index.ts index 344ce3641..3e491df6f 100644 --- a/packages/stage-ui/src/libs/providers/providers/moonshot-ai/index.ts +++ b/packages/provider-inference/src/providers/cloud/moonshot-ai/index.ts @@ -1,11 +1,11 @@ -import type { ChatRequestOptions } from '../../types' +import type { ChatRequestOptions } from '../../../types' import { createMoonshotai } from '@xsai-ext/providers/create' import { z } from 'zod' -import { ProviderValidationCheck } from '../../types' -import { createOpenAICompatibleValidators } from '../../validators' -import { defineProvider } from '../registry' +import { ProviderValidationCheck } from '../../../types' +import { createOpenAICompatibleValidators } from '../../../validators' +import { defineProvider } from '../../registry' const moonshotConfigSchema = z.object({ apiKey: z @@ -18,7 +18,7 @@ const moonshotConfigSchema = z.object({ type MoonshotConfig = z.input -export const providerMoonshotAI = defineProvider({ +export const providerMoonshotAI = defineProvider({ id: 'moonshot-ai', name: 'Moonshot AI', nameLocalize: ({ t }) => t('settings.pages.providers.provider.moonshot.title'), diff --git a/packages/stage-ui/src/libs/providers/providers/n1n/index.ts b/packages/provider-inference/src/providers/cloud/n1n/index.ts similarity index 88% rename from packages/stage-ui/src/libs/providers/providers/n1n/index.ts rename to packages/provider-inference/src/providers/cloud/n1n/index.ts index 3d0d6b7cd..3f5cab554 100644 --- a/packages/stage-ui/src/libs/providers/providers/n1n/index.ts +++ b/packages/provider-inference/src/providers/cloud/n1n/index.ts @@ -1,9 +1,9 @@ import { createOpenAI } from '@xsai-ext/providers/create' import { z } from 'zod' -import { ProviderValidationCheck } from '../../types' -import { createOpenAICompatibleValidators } from '../../validators' -import { defineProvider } from '../registry' +import { ProviderValidationCheck } from '../../../types' +import { createOpenAICompatibleValidators } from '../../../validators' +import { defineProvider } from '../../registry' const n1nConfigSchema = z.object({ apiKey: z @@ -17,7 +17,7 @@ const n1nConfigSchema = z.object({ type N1NConfig = z.input -export const providerN1N = defineProvider({ +export const providerN1N = defineProvider({ id: 'n1n', order: 9, name: 'n1n', diff --git a/packages/stage-ui/src/libs/providers/providers/novita-ai/index.ts b/packages/provider-inference/src/providers/cloud/novita-ai/index.ts similarity index 88% rename from packages/stage-ui/src/libs/providers/providers/novita-ai/index.ts rename to packages/provider-inference/src/providers/cloud/novita-ai/index.ts index e186ac46e..fca60e3f5 100644 --- a/packages/stage-ui/src/libs/providers/providers/novita-ai/index.ts +++ b/packages/provider-inference/src/providers/cloud/novita-ai/index.ts @@ -1,11 +1,11 @@ -import type { ChatRequestOptions } from '../../types' +import type { ChatRequestOptions } from '../../../types' import { createNovita } from '@xsai-ext/providers/create' import { z } from 'zod' -import { ProviderValidationCheck } from '../../types' -import { createOpenAICompatibleValidators } from '../../validators' -import { defineProvider } from '../registry' +import { ProviderValidationCheck } from '../../../types' +import { createOpenAICompatibleValidators } from '../../../validators' +import { defineProvider } from '../../registry' const novitaConfigSchema = z.object({ apiKey: z @@ -18,7 +18,7 @@ const novitaConfigSchema = z.object({ type NovitaConfig = z.input -export const providerNovitaAI = defineProvider({ +export const providerNovitaAI = defineProvider({ id: 'novita-ai', name: 'Novita', nameLocalize: ({ t }) => t('settings.pages.providers.provider.novita.title'), diff --git a/packages/stage-ui/src/libs/providers/providers/openai-audio/index.ts b/packages/provider-inference/src/providers/cloud/openai-audio/index.ts similarity index 96% rename from packages/stage-ui/src/libs/providers/providers/openai-audio/index.ts rename to packages/provider-inference/src/providers/cloud/openai-audio/index.ts index c5ec9a76d..cac2d6a24 100644 --- a/packages/stage-ui/src/libs/providers/providers/openai-audio/index.ts +++ b/packages/provider-inference/src/providers/cloud/openai-audio/index.ts @@ -1,10 +1,10 @@ -import type { ComposerTranslation } from 'vue-i18n' +import type { ProviderContext, ProviderTranslator } from '../../../types' import { createOpenAI } from '@xsai-ext/providers/create' import { listModels } from '@xsai/model' import { z } from 'zod' -import { defineProvider } from '../registry' +import { defineProvider } from '../../registry' const OPENAI_BASE_URL = 'https://api.openai.com/v1/' @@ -22,7 +22,7 @@ type OpenAIAudioConfig = z.input type OpenAICompatibleAudioConfig = z.input type AudioConfig = OpenAIAudioConfig | OpenAICompatibleAudioConfig -function createAudioConfigSchema(schema: T, t: ComposerTranslation) { +function createAudioConfigSchema(schema: T, t: ProviderTranslator) { return schema.extend({ apiKey: schema.shape.apiKey.meta({ labelLocalized: t('settings.pages.providers.catalog.edit.config.common.fields.field.api-key.label'), @@ -60,7 +60,7 @@ function createTranscriptionProvider(config: AudioConfig) { function createAudioValidators() { return { validateConfig: [ - ({ t }: { t: ComposerTranslation }) => ({ + ({ t }: ProviderContext) => ({ id: 'openai-audio:check-config', name: t('settings.pages.providers.catalog.edit.validators.openai-compatible.check-config.title'), validator: async (config: TConfig) => { @@ -173,7 +173,7 @@ const openAITranscriptionModels = [ deprecated: false, })) -export const providerOpenAIAudioSpeech = defineProvider({ +export const providerOpenAIAudioSpeech = defineProvider({ id: 'openai-audio-speech', name: 'OpenAI', nameLocalize: ({ t }) => t('settings.pages.providers.provider.openai.title'), @@ -191,7 +191,7 @@ export const providerOpenAIAudioSpeech = defineProvider({ }, }) -export const providerOpenAICompatibleAudioSpeech = defineProvider({ +export const providerOpenAICompatibleAudioSpeech = defineProvider({ id: 'openai-compatible-audio-speech', name: 'OpenAI Compatible', nameLocalize: ({ t }) => t('settings.pages.providers.provider.openai-compatible.title'), @@ -226,7 +226,7 @@ export const providerOpenAICompatibleAudioSpeech = defineProvider({ +export const providerOpenAIAudioTranscription = defineProvider({ id: 'openai-audio-transcription', name: 'OpenAI', nameLocalize: ({ t }) => t('settings.pages.providers.provider.openai.title'), @@ -246,7 +246,7 @@ export const providerOpenAIAudioTranscription = defineProvider({ +export const providerOpenAICompatibleAudioTranscription = defineProvider({ id: 'openai-compatible-audio-transcription', name: 'OpenAI Compatible', nameLocalize: ({ t }) => t('settings.pages.providers.provider.openai-compatible.title'), diff --git a/packages/stage-ui/src/libs/providers/providers/openai-compatible/index.ts b/packages/provider-inference/src/providers/cloud/openai-compatible/index.ts similarity index 90% rename from packages/stage-ui/src/libs/providers/providers/openai-compatible/index.ts rename to packages/provider-inference/src/providers/cloud/openai-compatible/index.ts index a5b68493c..ec5a6608b 100644 --- a/packages/stage-ui/src/libs/providers/providers/openai-compatible/index.ts +++ b/packages/provider-inference/src/providers/cloud/openai-compatible/index.ts @@ -1,9 +1,9 @@ import { createOpenAI } from '@xsai-ext/providers/create' import { z } from 'zod' -import { ProviderValidationCheck } from '../../types' -import { createOpenAICompatibleValidators } from '../../validators' -import { defineProvider } from '../registry' +import { ProviderValidationCheck } from '../../../types' +import { createOpenAICompatibleValidators } from '../../../validators' +import { defineProvider } from '../../registry' const openAICompatibleConfigSchema = z.object({ apiKey: z @@ -17,7 +17,7 @@ const openAICompatibleConfigSchema = z.object({ type OpenAICompatibleConfig = z.input -export const providerOpenAICompatible = defineProvider({ +export const providerOpenAICompatible = defineProvider({ id: 'openai-compatible', order: 4, name: 'OpenAI Compatible', diff --git a/packages/stage-ui/src/libs/providers/providers/openai/index.ts b/packages/provider-inference/src/providers/cloud/openai/index.ts similarity index 90% rename from packages/stage-ui/src/libs/providers/providers/openai/index.ts rename to packages/provider-inference/src/providers/cloud/openai/index.ts index 73ae6ea61..b77c491c1 100644 --- a/packages/stage-ui/src/libs/providers/providers/openai/index.ts +++ b/packages/provider-inference/src/providers/cloud/openai/index.ts @@ -1,11 +1,11 @@ -import type { ChatRequestOptions } from '../../types' +import type { ChatRequestOptions } from '../../../types' import { createOpenAI } from '@xsai-ext/providers/create' import { z } from 'zod' -import { ProviderValidationCheck } from '../../types' -import { createOpenAICompatibleValidators } from '../../validators' -import { defineProvider } from '../registry' +import { ProviderValidationCheck } from '../../../types' +import { createOpenAICompatibleValidators } from '../../../validators' +import { defineProvider } from '../../registry' const openAICompatibleConfigSchema = z.object({ apiKey: z @@ -18,7 +18,7 @@ const openAICompatibleConfigSchema = z.object({ type OpenAICompatibleConfig = z.input -export const providerOpenAI = defineProvider({ +export const providerOpenAI = defineProvider({ id: 'openai', order: 5, name: 'OpenAI', diff --git a/packages/stage-ui/src/libs/providers/providers/openpaths/index.ts b/packages/provider-inference/src/providers/cloud/openpaths/index.ts similarity index 88% rename from packages/stage-ui/src/libs/providers/providers/openpaths/index.ts rename to packages/provider-inference/src/providers/cloud/openpaths/index.ts index 1990b1a0f..2fd2e97fe 100644 --- a/packages/stage-ui/src/libs/providers/providers/openpaths/index.ts +++ b/packages/provider-inference/src/providers/cloud/openpaths/index.ts @@ -1,9 +1,9 @@ import { createOpenAI } from '@xsai-ext/providers/create' import { z } from 'zod' -import { ProviderValidationCheck } from '../../types' -import { createOpenAICompatibleValidators } from '../../validators' -import { defineProvider } from '../registry' +import { ProviderValidationCheck } from '../../../types' +import { createOpenAICompatibleValidators } from '../../../validators' +import { defineProvider } from '../../registry' const openPathsConfigSchema = z.object({ apiKey: z @@ -16,7 +16,7 @@ const openPathsConfigSchema = z.object({ type OpenPathsConfig = z.input -export const providerOpenPaths = defineProvider({ +export const providerOpenPaths = defineProvider({ id: 'openpaths', name: 'OpenPaths', nameLocalize: ({ t }) => t('settings.pages.providers.provider.openpaths.title'), diff --git a/packages/stage-ui/src/libs/providers/providers/openrouter-ai/index.ts b/packages/provider-inference/src/providers/cloud/openrouter-ai/index.ts similarity index 92% rename from packages/stage-ui/src/libs/providers/providers/openrouter-ai/index.ts rename to packages/provider-inference/src/providers/cloud/openrouter-ai/index.ts index 39aa1095d..f7f34f5d2 100644 --- a/packages/stage-ui/src/libs/providers/providers/openrouter-ai/index.ts +++ b/packages/provider-inference/src/providers/cloud/openrouter-ai/index.ts @@ -1,11 +1,11 @@ -import type { ChatRequestOptions } from '../../types' +import type { ChatRequestOptions } from '../../../types' import { createOpenRouter } from '@xsai-ext/providers/create' import { z } from 'zod' -import { ProviderValidationCheck } from '../../types' -import { createOpenAICompatibleValidators } from '../../validators' -import { defineProvider } from '../registry' +import { ProviderValidationCheck } from '../../../types' +import { createOpenAICompatibleValidators } from '../../../validators' +import { defineProvider } from '../../registry' export const OPENROUTER_ATTRIBUTION_HEADERS: Record = { 'HTTP-Referer': 'https://airi.moeru.ai/', @@ -23,7 +23,7 @@ const openRouterConfigSchema = z.object({ type OpenRouterConfig = z.input -export const providerOpenRouterAI = defineProvider({ +export const providerOpenRouterAI = defineProvider({ id: 'openrouter-ai', order: 0, name: 'OpenRouter', diff --git a/packages/stage-ui/src/libs/providers/providers/openrouter-audio-speech/index.ts b/packages/provider-inference/src/providers/cloud/openrouter-audio-speech/index.ts similarity index 98% rename from packages/stage-ui/src/libs/providers/providers/openrouter-audio-speech/index.ts rename to packages/provider-inference/src/providers/cloud/openrouter-audio-speech/index.ts index e0fad4796..73cc98795 100644 --- a/packages/stage-ui/src/libs/providers/providers/openrouter-audio-speech/index.ts +++ b/packages/provider-inference/src/providers/cloud/openrouter-audio-speech/index.ts @@ -1,8 +1,8 @@ import { toWavFromPCM16 } from '@proj-airi/audio/encoding' import { z } from 'zod' +import { defineProvider } from '../../registry' import { OPENROUTER_ATTRIBUTION_HEADERS } from '../openrouter-ai' -import { defineProvider } from '../registry' const DEFAULT_BASE_URL = 'https://openrouter.ai/api/v1/' const DEFAULT_MODEL = 'openai/gpt-audio-mini' @@ -123,7 +123,7 @@ function createAudioFetch(apiKey: string, baseUrl: string, model: string) { } } -export const providerOpenRouterAudioSpeech = defineProvider({ +export const providerOpenRouterAudioSpeech = defineProvider({ id: 'openrouter-audio-speech', name: 'OpenRouter', nameLocalize: ({ t }) => t('settings.pages.providers.provider.openrouter-audio-speech.title'), diff --git a/packages/stage-ui/src/libs/providers/providers/perplexity-ai/index.ts b/packages/provider-inference/src/providers/cloud/perplexity-ai/index.ts similarity index 92% rename from packages/stage-ui/src/libs/providers/providers/perplexity-ai/index.ts rename to packages/provider-inference/src/providers/cloud/perplexity-ai/index.ts index 3bc1c4a0a..3afa2e6b8 100644 --- a/packages/stage-ui/src/libs/providers/providers/perplexity-ai/index.ts +++ b/packages/provider-inference/src/providers/cloud/perplexity-ai/index.ts @@ -1,11 +1,11 @@ -import type { ModelInfo } from '../../types' +import type { ModelInfo } from '../../../types' import { createPerplexity } from '@xsai-ext/providers/create' import { z } from 'zod' -import { ProviderValidationCheck } from '../../types' -import { createOpenAICompatibleValidators } from '../../validators' -import { defineProvider } from '../registry' +import { ProviderValidationCheck } from '../../../types' +import { createOpenAICompatibleValidators } from '../../../validators' +import { defineProvider } from '../../registry' const perplexityConfigSchema = z.object({ apiKey: z @@ -18,7 +18,7 @@ const perplexityConfigSchema = z.object({ type PerplexityConfig = z.input -export const providerPerplexityAI = defineProvider({ +export const providerPerplexityAI = defineProvider({ id: 'perplexity-ai', name: 'Perplexity', nameLocalize: ({ t }) => t('settings.pages.providers.provider.perplexity.title'), diff --git a/packages/stage-ui/src/libs/providers/providers/together-ai/index.ts b/packages/provider-inference/src/providers/cloud/together-ai/index.ts similarity index 88% rename from packages/stage-ui/src/libs/providers/providers/together-ai/index.ts rename to packages/provider-inference/src/providers/cloud/together-ai/index.ts index f03f9a22f..ab733fecc 100644 --- a/packages/stage-ui/src/libs/providers/providers/together-ai/index.ts +++ b/packages/provider-inference/src/providers/cloud/together-ai/index.ts @@ -1,11 +1,11 @@ -import type { ChatRequestOptions } from '../../types' +import type { ChatRequestOptions } from '../../../types' import { createTogetherAI } from '@xsai-ext/providers/create' import { z } from 'zod' -import { ProviderValidationCheck } from '../../types' -import { createOpenAICompatibleValidators } from '../../validators' -import { defineProvider } from '../registry' +import { ProviderValidationCheck } from '../../../types' +import { createOpenAICompatibleValidators } from '../../../validators' +import { defineProvider } from '../../registry' const togetherConfigSchema = z.object({ apiKey: z @@ -18,7 +18,7 @@ const togetherConfigSchema = z.object({ type TogetherConfig = z.input -export const providerTogetherAI = defineProvider({ +export const providerTogetherAI = defineProvider({ id: 'together-ai', name: 'Together.ai', nameLocalize: ({ t }) => t('settings.pages.providers.provider.together.title'), diff --git a/packages/stage-ui/src/libs/providers/providers/unspeech/index.ts b/packages/provider-inference/src/providers/cloud/unspeech/index.ts similarity index 95% rename from packages/stage-ui/src/libs/providers/providers/unspeech/index.ts rename to packages/provider-inference/src/providers/cloud/unspeech/index.ts index 16c17d026..c0dbe3ad7 100644 --- a/packages/stage-ui/src/libs/providers/providers/unspeech/index.ts +++ b/packages/provider-inference/src/providers/cloud/unspeech/index.ts @@ -6,7 +6,8 @@ import type { UnVolcengineOptions, VoiceProviderWithExtraOptions, } from 'unspeech' -import type { ComposerTranslation } from 'vue-i18n' + +import type { ProviderContext, ProviderTranslator } from '../../../types' import { createUnAlibabaCloud, @@ -17,7 +18,7 @@ import { } from 'unspeech' import { z } from 'zod' -import { defineProvider } from '../registry' +import { defineProvider } from '../../registry' const unspeechConfigSchema = z.object({ apiKey: z.string(), @@ -36,12 +37,12 @@ type UnspeechConfig = z.input type MicrosoftSpeechConfig = z.input type VolcengineSpeechConfig = z.input -function createUnspeechConfigSchema(schema: typeof volcengineSpeechConfigSchema, t: ComposerTranslation): typeof volcengineSpeechConfigSchema -function createUnspeechConfigSchema(schema: typeof microsoftSpeechConfigSchema, t: ComposerTranslation): typeof microsoftSpeechConfigSchema -function createUnspeechConfigSchema(schema: typeof unspeechConfigSchema, t: ComposerTranslation): typeof unspeechConfigSchema +function createUnspeechConfigSchema(schema: typeof volcengineSpeechConfigSchema, t: ProviderTranslator): typeof volcengineSpeechConfigSchema +function createUnspeechConfigSchema(schema: typeof microsoftSpeechConfigSchema, t: ProviderTranslator): typeof microsoftSpeechConfigSchema +function createUnspeechConfigSchema(schema: typeof unspeechConfigSchema, t: ProviderTranslator): typeof unspeechConfigSchema function createUnspeechConfigSchema( schema: typeof unspeechConfigSchema | typeof microsoftSpeechConfigSchema | typeof volcengineSpeechConfigSchema, - t: ComposerTranslation, + t: ProviderTranslator, ) { return schema.extend({ apiKey: schema.shape.apiKey.meta({ @@ -105,7 +106,7 @@ function validateUnspeechConfig(config: UnspeechConfig, requireAppId = false) { function createUnspeechValidators(id: string, requireAppId = false) { return { validateConfig: [ - ({ t }: { t: ComposerTranslation }) => ({ + ({ t }: ProviderContext) => ({ id: `${id}:check-config`, name: t('settings.pages.providers.catalog.edit.validators.openai-compatible.check-config.title'), validator: async (config: TConfig) => validateUnspeechConfig(config, requireAppId), @@ -114,7 +115,7 @@ function createUnspeechValidators(id: string, re } } -export const providerDeepgramTts = defineProvider({ +export const providerDeepgramTts = defineProvider({ id: 'deepgram-tts', name: 'Deepgram', nameLocalize: ({ t }) => t('settings.pages.providers.provider.deepgram-tts.title'), @@ -147,7 +148,7 @@ export const providerDeepgramTts = defineProvider({ }, }) -export const providerMicrosoftSpeech = defineProvider({ +export const providerMicrosoftSpeech = defineProvider({ id: 'microsoft-speech', name: 'Microsoft / Azure Speech', nameLocalize: ({ t }) => t('settings.pages.providers.provider.microsoft-speech.title'), @@ -176,7 +177,7 @@ export const providerMicrosoftSpeech = defineProvider({ }, }) -export const providerAlibabaCloudModelStudio = defineProvider({ +export const providerAlibabaCloudModelStudio = defineProvider({ id: 'alibaba-cloud-model-studio', name: 'Alibaba Cloud Model Studio', nameLocalize: ({ t }) => t('settings.pages.providers.provider.alibaba-cloud-model-studio.title'), @@ -209,7 +210,7 @@ export const providerAlibabaCloudModelStudio = defineProvider({ }, }) -export const providerVolcengineSpeech = defineProvider({ +export const providerVolcengineSpeech = defineProvider({ id: 'volcengine', name: 'Volcengine', nameLocalize: ({ t }) => t('settings.pages.providers.provider.volcengine.title'), diff --git a/packages/stage-ui/src/libs/providers/providers/volcengine-coding-plan/index.ts b/packages/provider-inference/src/providers/cloud/volcengine-coding-plan/index.ts similarity index 100% rename from packages/stage-ui/src/libs/providers/providers/volcengine-coding-plan/index.ts rename to packages/provider-inference/src/providers/cloud/volcengine-coding-plan/index.ts diff --git a/packages/stage-ui/src/libs/providers/providers/xai/index.ts b/packages/provider-inference/src/providers/cloud/xai/index.ts similarity index 88% rename from packages/stage-ui/src/libs/providers/providers/xai/index.ts rename to packages/provider-inference/src/providers/cloud/xai/index.ts index 93717c1f7..9d9329fff 100644 --- a/packages/stage-ui/src/libs/providers/providers/xai/index.ts +++ b/packages/provider-inference/src/providers/cloud/xai/index.ts @@ -1,9 +1,9 @@ import { createXai } from '@xsai-ext/providers/create' import { z } from 'zod' -import { ProviderValidationCheck } from '../../types' -import { createOpenAICompatibleValidators } from '../../validators' -import { defineProvider } from '../registry' +import { ProviderValidationCheck } from '../../../types' +import { createOpenAICompatibleValidators } from '../../../validators' +import { defineProvider } from '../../registry' const xaiConfigSchema = z.object({ apiKey: z @@ -16,7 +16,7 @@ const xaiConfigSchema = z.object({ type XAIConfig = z.input -export const providerXAI = defineProvider({ +export const providerXAI = defineProvider({ id: 'xai', name: 'xAI', nameLocalize: ({ t }) => t('settings.pages.providers.provider.xai.title'), diff --git a/packages/stage-ui/src/libs/providers/providers/zai/index.ts b/packages/provider-inference/src/providers/cloud/zai/index.ts similarity index 88% rename from packages/stage-ui/src/libs/providers/providers/zai/index.ts rename to packages/provider-inference/src/providers/cloud/zai/index.ts index 80b548b51..9143a2490 100644 --- a/packages/stage-ui/src/libs/providers/providers/zai/index.ts +++ b/packages/provider-inference/src/providers/cloud/zai/index.ts @@ -1,11 +1,11 @@ -import type { ChatRequestOptions } from '../../types' +import type { ChatRequestOptions } from '../../../types' import { createZai } from '@xsai-ext/providers/create' import { z } from 'zod' -import { ProviderValidationCheck } from '../../types' -import { createOpenAICompatibleValidators } from '../../validators' -import { defineProvider } from '../registry' +import { ProviderValidationCheck } from '../../../types' +import { createOpenAICompatibleValidators } from '../../../validators' +import { defineProvider } from '../../registry' const zaiConfigSchema = z.object({ apiKey: z @@ -18,7 +18,7 @@ const zaiConfigSchema = z.object({ type ZaiConfig = z.input -export const providerZai = defineProvider({ +export const providerZai = defineProvider({ id: 'zai', name: 'Z.ai', nameLocalize: ({ t }) => t('settings.pages.providers.provider.zai.title'), diff --git a/packages/provider-inference/src/providers/index.ts b/packages/provider-inference/src/providers/index.ts new file mode 100644 index 000000000..f7991fae9 --- /dev/null +++ b/packages/provider-inference/src/providers/index.ts @@ -0,0 +1,148 @@ +import type { ProviderDefinition } from '../types' + +import { provider302AI } from './cloud/302-ai' +import { providerAIHubMix } from './cloud/aihubmix' +import { providerAmazonBedrock } from './cloud/amazon-bedrock' +import { providerAnthropic } from './cloud/anthropic' +import { providerAtlasCloud } from './cloud/atlascloud' +import { providerAzureAIFoundry } from './cloud/azure-ai-foundry' +import { providerAzureOpenAI } from './cloud/azure-openai' +import { providerBytePlus } from './cloud/byteplus' +import { providerBytePlusCodingPlan } from './cloud/byteplus-coding-plan' +import { providerCerebrasAI } from './cloud/cerebras-ai' +import { providerCloudflareWorkersAI } from './cloud/cloudflare-workers-ai' +import { providerCometAPI, providerCometAPISpeech, providerCometAPITranscription } from './cloud/comet-api' +import { providerDeepSeek } from './cloud/deepseek' +import { providerElevenLabs } from './cloud/elevenlabs' +import { providerFeatherlessAI } from './cloud/featherless-ai' +import { providerFireworksAI } from './cloud/fireworks-ai' +import { providerGoogleGeminiAudioSpeech } from './cloud/google-gemini-audio-speech' +import { providerGoogleGenerativeAI } from './cloud/google-generative-ai' +import { providerGroq } from './cloud/groq' +import { providerMimo } from './cloud/mimo' +import { providerMimoAudioSpeech, providerMimoAudioTranscription } from './cloud/mimo-audio' +import { providerMinimax, providerMinimaxGlobal } from './cloud/minimax' +import { providerMinimaxSpeech } from './cloud/minimax-speech' +import { providerMistralAI } from './cloud/mistral-ai' +import { providerModelScope } from './cloud/modelscope' +import { providerMoonshotAI } from './cloud/moonshot-ai' +import { providerN1N } from './cloud/n1n' +import { providerNovitaAI } from './cloud/novita-ai' +import { providerOpenAI } from './cloud/openai' +import { + providerOpenAIAudioSpeech, + providerOpenAIAudioTranscription, + providerOpenAICompatibleAudioSpeech, + providerOpenAICompatibleAudioTranscription, +} from './cloud/openai-audio' +import { providerOpenAICompatible } from './cloud/openai-compatible' +import { providerOpenPaths } from './cloud/openpaths' +import { providerOpenRouterAI } from './cloud/openrouter-ai' +import { providerOpenRouterAudioSpeech } from './cloud/openrouter-audio-speech' +import { providerPerplexityAI } from './cloud/perplexity-ai' +import { providerTogetherAI } from './cloud/together-ai' +import { + providerAlibabaCloudModelStudio, + providerDeepgramTts, + providerMicrosoftSpeech, + providerVolcengineSpeech, +} from './cloud/unspeech' +import { providerVolcengineCodingPlan } from './cloud/volcengine-coding-plan' +import { providerXAI } from './cloud/xai' +import { providerZai } from './cloud/zai' +import { providerBrowserWebSpeechApi } from './local/browser-web-speech-api' +import { providerIndexTtsVllm } from './local/index-tts-vllm' +import { providerLmStudio } from './local/lm-studio' +import { providerOllama } from './local/ollama' +import { providerPlayer2Speech } from './local/player2-speech' +import { providerSpeechNoop } from './local/speech-noop' +import { providerAivisSpeech, providerVoicevox } from './local/voicevox' + +/** + * Erases configuration types at the registry boundary. + * + * A registry selects definitions by a runtime id. It cannot know the selected + * configuration type. Individual provider exports preserve their exact type. + */ +type ProviderDefinitionRegistration = Pick & { id: TId } + +type ErasedProviderDefinitions = { + [K in keyof TDefinitions]: TDefinitions[K] extends ProviderDefinitionRegistration + ? ProviderDefinition, TId> + : never +} + +function eraseProviderDefinitions( + ...definitions: TDefinitions +): ErasedProviderDefinitions { + return definitions as unknown as ErasedProviderDefinitions +} + +/** + * Definitions that can load without Vue, Pinia, persistence, or Electron. + * + * This list includes Browser-only definitions. Hosts must call + * `isAvailableBy` before they offer one to a user in the current runtime. + */ +export const portableProviderDefinitions = eraseProviderDefinitions( + provider302AI, + providerAIHubMix, + providerAmazonBedrock, + providerAnthropic, + providerAtlasCloud, + providerAzureAIFoundry, + providerAzureOpenAI, + providerBytePlus, + providerBytePlusCodingPlan, + providerCerebrasAI, + providerCloudflareWorkersAI, + providerCometAPI, + providerCometAPISpeech, + providerCometAPITranscription, + providerDeepSeek, + providerElevenLabs, + providerFeatherlessAI, + providerFireworksAI, + providerGoogleGeminiAudioSpeech, + providerGoogleGenerativeAI, + providerGroq, + providerMinimax, + providerMinimaxGlobal, + providerMinimaxSpeech, + providerMimo, + providerMimoAudioSpeech, + providerMimoAudioTranscription, + providerMistralAI, + providerModelScope, + providerMoonshotAI, + providerN1N, + providerNovitaAI, + providerOpenAI, + providerOpenAIAudioSpeech, + providerOpenAIAudioTranscription, + providerOpenAICompatibleAudioSpeech, + providerOpenAICompatibleAudioTranscription, + providerOpenAICompatible, + providerOpenPaths, + providerOpenRouterAI, + providerOpenRouterAudioSpeech, + providerPerplexityAI, + providerTogetherAI, + providerAlibabaCloudModelStudio, + providerDeepgramTts, + providerMicrosoftSpeech, + providerVolcengineSpeech, + providerVolcengineCodingPlan, + providerXAI, + providerZai, + providerBrowserWebSpeechApi, + providerIndexTtsVllm, + providerLmStudio, + providerOllama, + providerPlayer2Speech, + providerSpeechNoop, + providerAivisSpeech, + providerVoicevox, +) + +export { createProviderRegistry, defineProvider } from './registry' diff --git a/packages/provider-inference/src/providers/local/browser-web-speech-api/index.browser.test.ts b/packages/provider-inference/src/providers/local/browser-web-speech-api/index.browser.test.ts new file mode 100644 index 000000000..fb6eed0c0 --- /dev/null +++ b/packages/provider-inference/src/providers/local/browser-web-speech-api/index.browser.test.ts @@ -0,0 +1,47 @@ +import { afterEach, describe, expect, it, vi } from 'vitest' + +import { providerBrowserWebSpeechApi } from '.' + +class FakeSpeechRecognition { + continuous = false + interimResults = false + lang = '' + maxAlternatives = 1 + onend: (() => void) | undefined + + start(): void { + this.onend?.() + } + + stop(): void {} +} + +afterEach(() => { + vi.unstubAllGlobals() +}) + +describe('browser Web Speech provider', () => { + it('uses an explicit Browser capability fake', async () => { + vi.stubGlobal('SpeechRecognition', FakeSpeechRecognition) + + expect(await providerBrowserWebSpeechApi.isAvailableBy?.()).toBe(true) + + const provider = await providerBrowserWebSpeechApi.createProvider({ + continuous: true, + interimResults: true, + language: 'en-US', + maxAlternatives: 1, + }) + + if (!('transcription' in provider)) + throw new Error('Web Speech API did not create a transcription provider.') + + const request = provider.transcription?.('web-speech-api') + if (!request?.fetch) + throw new Error('Web Speech API did not create a transcription request.') + + const response = await request.fetch(new URL('https://provider.test/transcription'), {}) + + expect(response).toBeInstanceOf(ReadableStream) + }) +}) diff --git a/packages/stage-ui/src/libs/providers/providers/browser-web-speech-api/index.ts b/packages/provider-inference/src/providers/local/browser-web-speech-api/index.ts similarity index 98% rename from packages/stage-ui/src/libs/providers/providers/browser-web-speech-api/index.ts rename to packages/provider-inference/src/providers/local/browser-web-speech-api/index.ts index ce10a4f5a..d7bed1166 100644 --- a/packages/stage-ui/src/libs/providers/providers/browser-web-speech-api/index.ts +++ b/packages/provider-inference/src/providers/local/browser-web-speech-api/index.ts @@ -1,6 +1,6 @@ import { z } from 'zod' -import { defineProvider } from '../registry' +import { defineProvider } from '../../registry' import { createWebSpeechAPIProvider } from './provider' const webSpeechApiConfigSchema = z.object({ diff --git a/packages/stage-ui/src/libs/providers/providers/browser-web-speech-api/provider.ts b/packages/provider-inference/src/providers/local/browser-web-speech-api/provider.ts similarity index 99% rename from packages/stage-ui/src/libs/providers/providers/browser-web-speech-api/provider.ts rename to packages/provider-inference/src/providers/local/browser-web-speech-api/provider.ts index b1939a779..e72c5faee 100644 --- a/packages/stage-ui/src/libs/providers/providers/browser-web-speech-api/provider.ts +++ b/packages/provider-inference/src/providers/local/browser-web-speech-api/provider.ts @@ -1,7 +1,7 @@ import type { TranscriptionProviderWithExtraOptions } from '@xsai-ext/providers/utils' import type { StreamTranscriptionDelta, StreamTranscriptionResult } from '@xsai/stream-transcription' -import { errorMessageFromValue } from '@proj-airi/stage-shared' +import { errorMessageFrom } from '@moeru/std' // NOTICE: Copied/adapted from @xsai/stream-transcription delayed promise helper. // Ref: @xsai/stream-transcription@0.4.0-beta.8 (dist/index.js DelayedPromise usage). @@ -308,7 +308,7 @@ export function streamWebSpeechAPITranscription( } catch (newErr) { console.error('Web Speech API failed to create new instance:', newErr) - const error = new Error(`Failed to restart recognition: ${errorMessageFromValue(newErr)}`) + const error = new Error(`Failed to restart recognition: ${errorMessageFrom(newErr) ?? 'Unknown error'}`) fullStreamCtrl?.error(error) textStreamCtrl?.error(error) deferredText.reject(error) diff --git a/packages/stage-ui/src/libs/providers/providers/index-tts-vllm/index.ts b/packages/provider-inference/src/providers/local/index-tts-vllm/index.ts similarity index 97% rename from packages/stage-ui/src/libs/providers/providers/index-tts-vllm/index.ts rename to packages/provider-inference/src/providers/local/index-tts-vllm/index.ts index 2cdccc203..eca2ac447 100644 --- a/packages/stage-ui/src/libs/providers/providers/index-tts-vllm/index.ts +++ b/packages/provider-inference/src/providers/local/index-tts-vllm/index.ts @@ -1,7 +1,7 @@ import { errorMessageFrom } from '@moeru/std' import { z } from 'zod' -import { defineProvider } from '../registry' +import { defineProvider } from '../../registry' const indexTtsConfigSchema = z.object({ baseUrl: z.string().default('http://localhost:11996/tts/'), @@ -14,7 +14,7 @@ function voicesUrl(config: IndexTtsConfig) { return `${config.baseUrl ?? 'http://localhost:11996/tts/'}audio/voices` } -export const providerIndexTtsVllm = defineProvider({ +export const providerIndexTtsVllm = defineProvider({ id: 'index-tts-vllm', name: 'Index-TTS by Bilibili', nameLocalize: ({ t }) => t('settings.pages.providers.provider.index-tts-vllm.title'), diff --git a/packages/stage-ui/src/libs/providers/providers/lm-studio/index.ts b/packages/provider-inference/src/providers/local/lm-studio/index.ts similarity index 92% rename from packages/stage-ui/src/libs/providers/providers/lm-studio/index.ts rename to packages/provider-inference/src/providers/local/lm-studio/index.ts index b7180d466..a5a41f829 100644 --- a/packages/stage-ui/src/libs/providers/providers/lm-studio/index.ts +++ b/packages/provider-inference/src/providers/local/lm-studio/index.ts @@ -1,9 +1,9 @@ import { createChatProvider, createEmbedProvider, createModelProvider, merge } from '@xsai-ext/providers/utils' import { z } from 'zod' -import { ProviderValidationCheck } from '../../types' -import { createOpenAICompatibleValidators } from '../../validators' -import { defineProvider } from '../registry' +import { ProviderValidationCheck } from '../../../types' +import { createOpenAICompatibleValidators } from '../../../validators' +import { defineProvider } from '../../registry' const lmStudioConfigSchema = z.object({ apiKey: z @@ -17,7 +17,7 @@ const lmStudioConfigSchema = z.object({ type LMStudioConfig = z.input -export const providerLmStudio = defineProvider({ +export const providerLmStudio = defineProvider({ id: 'lm-studio', order: 3, name: 'LM Studio', diff --git a/packages/stage-ui/src/libs/providers/providers/ollama/index.test.ts b/packages/provider-inference/src/providers/local/ollama/index.test.ts similarity index 99% rename from packages/stage-ui/src/libs/providers/providers/ollama/index.test.ts rename to packages/provider-inference/src/providers/local/ollama/index.test.ts index a7bdb1bfd..cafbe074e 100644 --- a/packages/stage-ui/src/libs/providers/providers/ollama/index.test.ts +++ b/packages/provider-inference/src/providers/local/ollama/index.test.ts @@ -1,6 +1,6 @@ import type { ChatProviderWithExtraOptions } from '@xsai-ext/providers/utils' -import type { ChatRequestOptions, ProviderInstance } from '../../types' +import type { ChatRequestOptions, ProviderInstance } from '../../../types' import { describe, expect, it } from 'vitest' diff --git a/packages/stage-ui/src/libs/providers/providers/ollama/index.ts b/packages/provider-inference/src/providers/local/ollama/index.ts similarity index 95% rename from packages/stage-ui/src/libs/providers/providers/ollama/index.ts rename to packages/provider-inference/src/providers/local/ollama/index.ts index 6cfac97c1..1cc068edf 100644 --- a/packages/stage-ui/src/libs/providers/providers/ollama/index.ts +++ b/packages/provider-inference/src/providers/local/ollama/index.ts @@ -1,11 +1,11 @@ -import type { ChatRequestOptions } from '../../types' +import type { ChatRequestOptions } from '../../../types' import { createOllama } from '@xsai-ext/providers/create' import { z } from 'zod' -import { ProviderValidationCheck } from '../../types' -import { createOpenAICompatibleValidators } from '../../validators' -import { defineProvider } from '../registry' +import { ProviderValidationCheck } from '../../../types' +import { createOpenAICompatibleValidators } from '../../../validators' +import { defineProvider } from '../../registry' type OllamaReasoningEffort = 'high' | 'low' | 'medium' | 'none' type OllamaThinkingMode = 'auto' | 'disable' | 'enable' | 'high' | 'low' | 'medium' @@ -61,7 +61,7 @@ export function resolveOllamaReasoningEffort(modeRaw: unknown): OllamaReasoningE } } -export const providerOllama = defineProvider({ +export const providerOllama = defineProvider({ id: 'ollama', order: 2, name: 'Ollama', diff --git a/packages/stage-ui/src/libs/providers/providers/player2-speech/index.ts b/packages/provider-inference/src/providers/local/player2-speech/index.ts similarity index 98% rename from packages/stage-ui/src/libs/providers/providers/player2-speech/index.ts rename to packages/provider-inference/src/providers/local/player2-speech/index.ts index 3973cd370..322e6849a 100644 --- a/packages/stage-ui/src/libs/providers/providers/player2-speech/index.ts +++ b/packages/provider-inference/src/providers/local/player2-speech/index.ts @@ -2,7 +2,7 @@ import { errorMessageFrom } from '@moeru/std' import { createPlayer2 } from '@xsai-ext/providers/special/create' import { z } from 'zod' -import { defineProvider } from '../registry' +import { defineProvider } from '../../registry' const player2ConfigSchema = z.object({ baseUrl: z.string().default('http://localhost:4315/v1/'), @@ -28,7 +28,7 @@ function normalizeBaseUrl(baseUrl: string | undefined) { return value.endsWith('/') ? value : `${value}/` } -export const providerPlayer2Speech = defineProvider({ +export const providerPlayer2Speech = defineProvider({ id: 'player2-speech', name: 'Player2 Speech', nameLocalize: ({ t }) => t('settings.pages.providers.provider.player2.title'), diff --git a/packages/stage-ui/src/libs/providers/providers/speech-noop/index.ts b/packages/provider-inference/src/providers/local/speech-noop/index.ts similarity index 94% rename from packages/stage-ui/src/libs/providers/providers/speech-noop/index.ts rename to packages/provider-inference/src/providers/local/speech-noop/index.ts index 7cf7c2506..edf6e251c 100644 --- a/packages/stage-ui/src/libs/providers/providers/speech-noop/index.ts +++ b/packages/provider-inference/src/providers/local/speech-noop/index.ts @@ -1,6 +1,6 @@ import { z } from 'zod' -import { defineProvider } from '../registry' +import { defineProvider } from '../../registry' const speechNoopConfigSchema = z.object({}) diff --git a/packages/stage-ui/src/libs/providers/providers/voicevox/define.test.ts b/packages/provider-inference/src/providers/local/voicevox/define.test.ts similarity index 97% rename from packages/stage-ui/src/libs/providers/providers/voicevox/define.test.ts rename to packages/provider-inference/src/providers/local/voicevox/define.test.ts index 83d4ccb2c..055943670 100644 --- a/packages/stage-ui/src/libs/providers/providers/voicevox/define.test.ts +++ b/packages/provider-inference/src/providers/local/voicevox/define.test.ts @@ -1,14 +1,15 @@ import type { SpeechProvider } from '@xsai-ext/providers/utils' -import type { ComposerTranslation } from 'vue-i18n' + +import type { ProviderTranslator } from '../../../types' import { generateSpeech } from '@xsai/generate-speech' import { afterEach, describe, expect, it, vi } from 'vitest' import { z } from 'zod' import { providerAivisSpeech, providerVoicevox } from '.' -import { getProviderValidationIntervalMs } from '../../validators/run' +import { getProviderValidationIntervalMs } from '../../../validators/run' -const translate = ((key: string) => key) as unknown as ComposerTranslation +const translate = ((key: string) => key) as unknown as ProviderTranslator interface EngineCall { init: RequestInit diff --git a/packages/stage-ui/src/libs/providers/providers/voicevox/define.ts b/packages/provider-inference/src/providers/local/voicevox/define.ts similarity index 94% rename from packages/stage-ui/src/libs/providers/providers/voicevox/define.ts rename to packages/provider-inference/src/providers/local/voicevox/define.ts index d19cbc46a..31a339484 100644 --- a/packages/stage-ui/src/libs/providers/providers/voicevox/define.ts +++ b/packages/provider-inference/src/providers/local/voicevox/define.ts @@ -1,10 +1,10 @@ -import type { ProviderDefinition, VoiceInfo } from '../../types' +import type { ProviderDefinition, VoiceInfo } from '../../../types' import type { VoicevoxSynthesisParameters } from './engine' import { errorMessageFrom } from '@moeru/std' import { z } from 'zod' -import { defineProvider } from '../registry' +import { defineProvider } from '../../registry' import { fetchEngineVersion, fetchSpeakers, synthesizeSpeech } from './engine' /** @@ -50,12 +50,12 @@ const voicevoxVoiceSettingsSchema = z.object({ export type VoicevoxFamilyConfig = z.input> -export interface VoicevoxFamilyProviderOptions { +export interface VoicevoxFamilyProviderOptions { /** Prefilled Base URL, and the address the validator names when the field is empty. */ defaultBaseUrl: string /** Fallback description, shown when the locale has no entry. */ description: string - id: string + id: TId /** Fallback name, shown when the locale has no entry. */ name: string } @@ -67,10 +67,12 @@ export interface VoicevoxFamilyProviderOptions { * OpenAI-shaped request that `generateSpeech` builds. {@link synthesizeSpeech} * makes the two engine requests instead. */ -export function defineVoicevoxFamilyProvider(options: VoicevoxFamilyProviderOptions): ProviderDefinition { +export function defineVoicevoxFamilyProvider( + options: VoicevoxFamilyProviderOptions, +): ProviderDefinition { const configSchema = createVoicevoxConfigSchema(options.defaultBaseUrl) - return defineProvider({ + return defineProvider({ createProvider(config) { return { speech: () => ({ diff --git a/packages/stage-ui/src/libs/providers/providers/voicevox/engine.test.ts b/packages/provider-inference/src/providers/local/voicevox/engine.test.ts similarity index 100% rename from packages/stage-ui/src/libs/providers/providers/voicevox/engine.test.ts rename to packages/provider-inference/src/providers/local/voicevox/engine.test.ts diff --git a/packages/stage-ui/src/libs/providers/providers/voicevox/engine.ts b/packages/provider-inference/src/providers/local/voicevox/engine.ts similarity index 100% rename from packages/stage-ui/src/libs/providers/providers/voicevox/engine.ts rename to packages/provider-inference/src/providers/local/voicevox/engine.ts diff --git a/packages/stage-ui/src/libs/providers/providers/voicevox/index.ts b/packages/provider-inference/src/providers/local/voicevox/index.ts similarity index 100% rename from packages/stage-ui/src/libs/providers/providers/voicevox/index.ts rename to packages/provider-inference/src/providers/local/voicevox/index.ts diff --git a/packages/provider-inference/src/providers/registry.test.ts b/packages/provider-inference/src/providers/registry.test.ts new file mode 100644 index 000000000..7f041605d --- /dev/null +++ b/packages/provider-inference/src/providers/registry.test.ts @@ -0,0 +1,48 @@ +import type { PortableProviderId } from '../index' + +import { describe, expect, expectTypeOf, it } from 'vitest' + +import { createProviderRegistry, getDefinedProvider, listProviders, portableProviderDefinitions } from '../index' + +describe('portable provider registry', () => { + it('exports the portable provider id union', () => { + expectTypeOf<'openai'>().toExtend() + expectTypeOf().not.toExtend() + + expect(getDefinedProvider('openai')).toBeDefined() + }) + + it('registers each portable definition by id', () => { + for (const definition of portableProviderDefinitions) + expect(getDefinedProvider(definition.id)).toBe(definition) + }) + + it('lists definitions in deterministic display order', () => { + expect(listProviders()).toEqual(listProviders()) + }) + + it('places definitions without an order after all ordered definitions', () => { + const [template] = portableProviderDefinitions + const registry = createProviderRegistry([ + { ...template, id: 'unordered-z', name: 'Zeta', order: undefined }, + { ...template, id: 'high-order', name: 'High order', order: 100_000 }, + { ...template, id: 'unordered-a', name: 'Alpha', order: undefined }, + ]) + + expect(registry.list().map(provider => provider.id)).toEqual([ + 'high-order', + 'unordered-a', + 'unordered-z', + ]) + }) + + it('does not include providers that require application runtime adapters', () => { + const providerIds = new Set(listProviders().map(provider => provider.id)) + + expect(providerIds).not.toContain('official-provider') + expect(providerIds).not.toContain('apple-speech-transcription') + expect(providerIds).not.toContain('kokoro-local') + expect(providerIds).not.toContain('browser-local-audio-speech') + expect(providerIds).not.toContain('aliyun-nls-transcription') + }) +}) diff --git a/packages/provider-inference/src/providers/registry.ts b/packages/provider-inference/src/providers/registry.ts new file mode 100644 index 000000000..f3aa2a658 --- /dev/null +++ b/packages/provider-inference/src/providers/registry.ts @@ -0,0 +1,58 @@ +import type { ProviderDefinition } from '../types' + +/** + * A read-only lookup surface for provider definitions. + * + * The caller supplies the definitions. Importing a provider module does not + * change a shared registry. + */ +export interface ProviderRegistry { + get: (id: string) => ProviderDefinition | undefined + list: () => ProviderDefinition[] +} + +/** + * Keeps the existing declaration style while making provider modules pure. + * + * The application registry receives the returned definitions explicitly. + */ +export function defineProvider( + definition: ProviderDefinition, +): ProviderDefinition { + return definition +} + +/** + * Creates a deterministic provider registry from explicit definitions. + * + * @throws {Error} When more than one definition declares the same id. + */ +export function createProviderRegistry(definitions: readonly ProviderDefinition[]): ProviderRegistry { + const definitionsById = new Map() + + for (const definition of definitions) { + if (definitionsById.has(definition.id)) + throw new Error(`Provider definition "${definition.id}" is registered more than once.`) + + definitionsById.set(definition.id, definition) + } + + const sortedDefinitions = [...definitionsById.values()].toSorted((left, right) => { + if (left.order === undefined && right.order !== undefined) + return 1 + if (left.order !== undefined && right.order === undefined) + return -1 + if (left.order !== undefined && right.order !== undefined && left.order !== right.order) + return left.order - right.order + if (left.name < right.name) + return -1 + if (left.name > right.name) + return 1 + return 0 + }) + + return { + get: id => definitionsById.get(id), + list: () => [...sortedDefinitions], + } +} diff --git a/packages/provider-inference/src/providers/runtime-matrix.browser.test.ts b/packages/provider-inference/src/providers/runtime-matrix.browser.test.ts new file mode 100644 index 000000000..ddb920574 --- /dev/null +++ b/packages/provider-inference/src/providers/runtime-matrix.browser.test.ts @@ -0,0 +1,45 @@ +import type { ProviderTranslator } from '../types' + +import { describe, expect, it } from 'vitest' + +import { portableProviderDefinitions } from '.' + +const translate: ProviderTranslator = key => key +const browserOnlyProviderIds = new Set(['browser-web-speech-api']) +const localApiProviderIds = new Set([ + 'aivis-speech', + 'index-tts-vllm', + 'lm-studio', + 'ollama', + 'player2-speech', + 'voicevox', +]) +const localRunProviderIds = new Set([ + 'browser-web-speech-api', + 'speech-noop', +]) + +function testCategory(providerId: string): 'cloud-api' | 'local-api' | 'local-run' { + if (localApiProviderIds.has(providerId)) + return 'local-api' + if (localRunProviderIds.has(providerId)) + return 'local-run' + return 'cloud-api' +} + +function testRuntimeResult(providerId: string): 'browser-only' | 'node-and-browser' { + return browserOnlyProviderIds.has(providerId) ? 'browser-only' : 'node-and-browser' +} + +describe('portable provider runtime matrix in Browser', () => { + for (const definition of portableProviderDefinitions) { + const category = testCategory(definition.id) + const result = testRuntimeResult(definition.id) + + it(`[${category}/${result}] loads ${definition.id}`, async () => { + const schema = await definition.createProviderConfig({ t: translate }) + + expect(schema).toBeDefined() + }) + } +}) diff --git a/packages/provider-inference/src/providers/runtime-matrix.test.ts b/packages/provider-inference/src/providers/runtime-matrix.test.ts new file mode 100644 index 000000000..774f11239 --- /dev/null +++ b/packages/provider-inference/src/providers/runtime-matrix.test.ts @@ -0,0 +1,48 @@ +import type { ProviderTranslator } from '../types' + +import { describe, expect, it } from 'vitest' + +import { portableProviderDefinitions } from '.' + +const translate: ProviderTranslator = key => key +const browserOnlyProviderIds = new Set(['browser-web-speech-api']) +const localApiProviderIds = new Set([ + 'aivis-speech', + 'index-tts-vllm', + 'lm-studio', + 'ollama', + 'player2-speech', + 'voicevox', +]) +const localRunProviderIds = new Set([ + 'browser-web-speech-api', + 'speech-noop', +]) + +function testCategory(providerId: string): 'cloud-api' | 'local-api' | 'local-run' { + if (localApiProviderIds.has(providerId)) + return 'local-api' + if (localRunProviderIds.has(providerId)) + return 'local-run' + return 'cloud-api' +} + +function testRuntimeResult(providerId: string): 'browser-only' | 'node-and-browser' { + return browserOnlyProviderIds.has(providerId) ? 'browser-only' : 'node-and-browser' +} + +describe('portable provider runtime matrix in Node.js', () => { + for (const definition of portableProviderDefinitions) { + const category = testCategory(definition.id) + const result = testRuntimeResult(definition.id) + + it(`[${category}/${result}] loads ${definition.id}`, async () => { + const schema = await definition.createProviderConfig({ t: translate }) + + expect(schema).toBeDefined() + + if (browserOnlyProviderIds.has(definition.id)) + expect(await definition.isAvailableBy?.()).toBe(false) + }) + } +}) diff --git a/packages/stage-ui/src/libs/providers/tool-schema.test.ts b/packages/provider-inference/src/providers/tool-schema.test.ts similarity index 100% rename from packages/stage-ui/src/libs/providers/tool-schema.test.ts rename to packages/provider-inference/src/providers/tool-schema.test.ts diff --git a/packages/stage-ui/src/libs/providers/tool-schema.ts b/packages/provider-inference/src/providers/tool-schema.ts similarity index 100% rename from packages/stage-ui/src/libs/providers/tool-schema.ts rename to packages/provider-inference/src/providers/tool-schema.ts diff --git a/packages/provider-inference/src/types.ts b/packages/provider-inference/src/types.ts new file mode 100644 index 000000000..9ccda53e8 --- /dev/null +++ b/packages/provider-inference/src/types.ts @@ -0,0 +1,317 @@ +import type { + ChatProvider, + ChatProviderWithExtraOptions, + EmbedProvider, + EmbedProviderWithExtraOptions, + ModelProvider, + ModelProviderWithExtraOptions, + SpeechProvider, + SpeechProviderWithExtraOptions, + TranscriptionProvider, + TranscriptionProviderWithExtraOptions, +} from '@xsai-ext/providers/utils' +import type { ProgressInfo } from '@xsai-transformers/shared/types' +import type { MaybePromise } from 'clustr' +import type { $ZodType } from 'zod/v4/core' + +/** Translates a provider label or description for the active interface locale. */ +export type ProviderTranslator = (input: string) => string + +/** Context shared by provider callbacks that need interface translation. */ +export interface ProviderContext { + /** Translates labels and descriptions for the active interface locale. */ + t: ProviderTranslator +} + +export type ProviderInstance + = | ChatProvider + | ChatProviderWithExtraOptions + | EmbedProvider + | EmbedProviderWithExtraOptions + | SpeechProvider + | SpeechProviderWithExtraOptions + | TranscriptionProvider + | TranscriptionProviderWithExtraOptions + | ModelProvider + | ModelProviderWithExtraOptions + +/** Validation lifecycle for one serializable provider configuration. */ +export type ProviderValidationStatus = 'unconfigured' | 'validating' | 'configured' | 'invalid' | 'bypassed' +export type ProviderConfiguredBy = 'user' | 'authentication' + +/** Serializable configuration for one provider instance. */ +export interface InferenceServiceProvider { + /** Stable provider instance id. */ + id: string + /** Provider definition id from the built-in provider registry. */ + definitionId: string + /** Provider-specific configuration values. */ + config: Record + /** Current validation state for this provider configuration. */ + status: ProviderValidationStatus + /** Lifecycle owner that creates and revokes this provider configuration. */ + configuredBy: ProviderConfiguredBy +} + +export function isModelProvider(providerInstance: ProviderInstance): providerInstance is ModelProvider | ModelProviderWithExtraOptions { + if ('model' in providerInstance && typeof providerInstance.model === 'function') { + return true + } + + return false +} + +export interface ProviderOnboardingField { + key: string + type: 'text' | 'password' + label: string + description?: string + placeholder?: string + required?: boolean + defaultValue?: string +} + +/** Inputs available while a Provider builds its configuration schema. */ +export interface ProviderConfigContext extends ProviderContext { + /** Cancels runtime discovery that contributes schema metadata. */ + abortSignal?: AbortSignal + /** Current draft values. Providers can use them to resolve dependent fields. */ + config?: Partial +} + +/** Serializable model discovery result returned across renderer boundaries. */ +export interface ProviderModelCatalog { + /** Models discovered for this provider. */ + models: ModelInfo[] + /** Whether the server exposes this catalog. Absent when discovery did not return an authoritative state. */ + available?: boolean + /** Server-selected model id, or null when the server has no default. */ + defaultModel?: string | null +} + +export interface ProviderExtraMethods { + listModelCatalog?: (config: TConfig, provider: ProviderInstance, contextOptions?: ProviderContext) => Promise + listModels?: (config: TConfig, provider: ProviderInstance, contextOptions?: ProviderContext) => Promise + /** + * Returns the voice catalogue. `model` lets providers whose voices vary by + * model variant (Volcengine streaming TTS 1.0 vs 2.0 differ in catalogue) + * narrow the result. Providers with a single catalogue ignore it. + */ + listVoices?: (config: TConfig, provider: ProviderInstance, model?: string) => Promise + loadModel?: (config: TConfig, provider: ProviderInstance, hooks?: { onProgress?: (progress: ProgressInfo) => Promise | void }) => Promise +} + +export interface ProviderValidationResult { + errors: Array<{ error: unknown, errorKey?: string }> + reason: string + reasonKey: string + valid: boolean +} + +/** + * Validator ID fragment for the chat completions probe. + * Matched via `.includes()` against validator instance ids + * (e.g. `openai-compatible:check-chat-completions`). + */ +export const CHAT_COMPLETIONS_VALIDATOR_ID = 'check-chat-completions' + +export enum ProviderValidationCheck { + /** Lightweight GET to /models endpoint to check reachability (definition system) */ + Connectivity = 'connectivity', + /** Fetch model list and verify non-empty */ + ModelList = 'model_list', + /** Send generateText ping with fine-grained error handling and caching (definition system) */ + ChatCompletions = 'chat_completions', + /** + * @deprecated + * Being used in builder system (a deprecated provider creation protocol), + * currently used by only OpenAI TTS && OpenAI Transcription. + * Send generateText ping with simple pass/fail, fallback to 'test' model (builder system) + */ + Health = 'health', +} + +export interface ProviderValidatorSchedule { + mode: 'once' | 'interval' + intervalMs?: number +} + +export interface ProviderConfigValidator { + id: string + name: string + validator: (config: TConfig, contextOptions: ProviderContext) => MaybePromise + schedule?: ProviderValidatorSchedule +} + +export interface ProviderRuntimeValidator { + id: string + name: string + validator: (config: TConfig, provider: ProviderInstance, providerExtra: ProviderExtraMethods, contextOptions: ProviderContext) => MaybePromise + schedule?: ProviderValidatorSchedule +} + +export interface ModelInfo { + id: string + name: string + provider: string + description?: string + capabilities?: string[] + contextLength?: number + deprecated?: boolean +} + +export interface VoiceInfo { + id: string + name: string + provider: string + compatibleModels?: string[] + description?: string + gender?: string + deprecated?: boolean + previewURL?: string + languages: { + code: string + title: string + }[] +} + +/** A provider definition and its configuration contract. */ +export interface ProviderDefinition, TId extends string = string> { + /** Stable provider definition id. */ + id: TId + order?: number + tasks: string[] + nameLocalize: (ctx: ProviderContext) => string // i18n key for provider name + name: string // Default name (fallback) + descriptionLocalize: (ctx: ProviderContext) => string // i18n key for provider description + description: string // Default description (fallback) + /** + * Iconify JSON icon name for the provider. + * + * Icons are available for most of the AI provides under @proj-airi/lobe-icons. + */ + icon?: string + iconColor?: string + /** + * In case of having image instead of icon, you can specify the image URL here. + */ + iconImage?: string + + /** + * Indicates whether the provider is available. + * If not specified, the provider is always available. + * + * May be specified when any of the following criteria is required: + * + * Platform requirements: + * + * - app-* providers are only available on desktop, this is responsible for Tauri runtime checks + * - web-* providers are only available on web, this means Node.js and Tauri should not be imported or used + * + * System spec requirements: + * + * - may requires WebGPU / NVIDIA / other types of GPU, + * on Web, WebGPU will automatically compiled to use targeting GPU hardware + * - may requires significant amount of GPU memory to run, especially for + * using of small language models within browser or Tauri app + * - may requires significant amount of memory to run, especially for those + * non-WebGPU supported environments. + */ + isAvailableBy?: () => MaybePromise + + /** + * If false, the provider does not require user-provided credentials (e.g. API keys). + * Used for built-in providers that authenticate via JWT Bearer tokens. + */ + requiresCredentials?: boolean + + /** + * Lifecycle owner for provider configurations created from this definition. + * + * @default 'user' + */ + configuredBy?: ProviderConfiguredBy + + /** Builds the validation schema and its UI metadata for the current draft. */ + createProviderConfig: (contextOptions: ProviderConfigContext) => MaybePromise<$ZodType> + onboardingFields?: (ctx: ProviderContext) => MaybePromise + createProvider: (config: TConfig) => MaybePromise + extraMethods?: ProviderExtraMethods + /** + * Returns true when the configuration has enough input for automatic validation. + * Provider settings keep the status unconfigured while this function returns false. + * + * @default false + */ + validationRequiredWhen?: (config: TConfig) => MaybePromise + validators?: { + validateConfig?: Array<(contextOptions: ProviderContext) => MaybePromise>> + validateProvider?: Array<(contextOptions: ProviderContext) => MaybePromise>> + } + capabilities?: { + chat?: { + reasoning?: ChatReasoningCapability + } + transcription?: { + protocol: 'websocket' | 'http' | 'native' + generateOutput: boolean + streamOutput: boolean + streamInput: boolean + } + /** + * Declares the TTS transport this provider speaks. The host uses it to + * select its TTS session adapter: + * + * - `rest` (default when this whole block is absent): the host opens + * a `pipelines-audio` IntentHandle and the provider's `speech()` is + * called per-segment by the speech-pipeline `tts()` callback. This + * matches every OpenAI-shaped HTTP TTS provider. + * - `bidirectional-ws`: the host opens one streaming TTS WebSocket + * for the whole LLM intent and forwards raw token chunks without + * client-side segmentation. The provider's `speech()` is unused + * for synthesis on this path (kept only for legacy fallback). + * + * Designed so a future provider (ElevenLabs streaming, OpenAI Realtime + * Voice, etc.) only needs to set this flag — Stage and the session + * factory do not need to know each provider's id. + */ + speech?: { + transport: 'rest' | 'bidirectional-ws' + } + } + /** + * When true, hides the "skip chat ping check" checkbox in the UI even + * when the provider defines a ChatCompletions validator. + * + * By default, the checkbox is shown automatically whenever a provider + * includes a ChatCompletions runtime validator. Set this to `true` for + * providers where skipping that check is not meaningful or has not been + * verified yet. + */ + disableChatPingCheckUI?: boolean + business?: (contextOptions: ProviderContext) => { + troubleshooting?: { + validators?: { + openaiCompatibleCheckConnectivity?: { + label?: string + content?: string + } + } + } + } +} + +/** Reasoning modes that AIRI can request from a chat provider. */ +export type ChatReasoningMode = 'disabled' | 'enabled' + +/** User-selected options that a provider applies to one chat request. */ +export interface ChatRequestOptions { + /** Requested reasoning mode. */ + reasoning: ChatReasoningMode +} + +/** Describes the reasoning controls that AIRI implements for a provider. */ +export interface ChatReasoningCapability { + /** Modes that AIRI can pass to the provider. */ + modes: readonly ChatReasoningMode[] +} diff --git a/packages/provider-inference/src/validators/index.ts b/packages/provider-inference/src/validators/index.ts new file mode 100644 index 000000000..ed93fcedd --- /dev/null +++ b/packages/provider-inference/src/validators/index.ts @@ -0,0 +1,2 @@ +export * from './openai-compatible' +export * from './run' diff --git a/packages/stage-ui/src/libs/providers/validators/openai-compatible.test.ts b/packages/provider-inference/src/validators/openai-compatible.test.ts similarity index 96% rename from packages/stage-ui/src/libs/providers/validators/openai-compatible.test.ts rename to packages/provider-inference/src/validators/openai-compatible.test.ts index ede01d781..26f429490 100644 --- a/packages/stage-ui/src/libs/providers/validators/openai-compatible.test.ts +++ b/packages/provider-inference/src/validators/openai-compatible.test.ts @@ -1,10 +1,8 @@ -import type { ComposerTranslation } from 'vue-i18n' - -import type { ProviderExtraMethods, ProviderInstance } from '../types' +import type { ProviderExtraMethods, ProviderInstance, ProviderTranslator } from '../types' import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' -import { providerOpenAI } from '../providers/openai' +import { providerOpenAI } from '../providers/cloud/openai' import { ProviderValidationCheck } from '../types' import { createOpenAICompatibleValidators } from './openai-compatible' @@ -24,7 +22,7 @@ vi.mock('@xsai/model', () => ({ listModels: listModelsMock, })) -const mockT = vi.fn((key: string) => key) as unknown as ComposerTranslation +const mockT = vi.fn((key: string) => key) as ProviderTranslator async function getProviderValidators(options?: Parameters[0]) { const validators = createOpenAICompatibleValidators(options) diff --git a/packages/stage-ui/src/libs/providers/validators/openai-compatible.ts b/packages/provider-inference/src/validators/openai-compatible.ts similarity index 100% rename from packages/stage-ui/src/libs/providers/validators/openai-compatible.ts rename to packages/provider-inference/src/validators/openai-compatible.ts diff --git a/packages/stage-ui/src/libs/providers/validators/run.test.ts b/packages/provider-inference/src/validators/run.test.ts similarity index 93% rename from packages/stage-ui/src/libs/providers/validators/run.test.ts rename to packages/provider-inference/src/validators/run.test.ts index 482f9687f..543706e89 100644 --- a/packages/stage-ui/src/libs/providers/validators/run.test.ts +++ b/packages/provider-inference/src/validators/run.test.ts @@ -1,13 +1,11 @@ -import type { ComposerTranslation } from 'vue-i18n' - -import type { ProviderDefinition } from '../types' +import type { ProviderDefinition, ProviderTranslator } from '../types' import { createChatProvider } from '@xsai-ext/providers/utils' import { describe, expect, it, vi } from 'vitest' import { getValidatorsOfProvider, validateProvider } from './run' -const mockT = ((key: string) => key) as unknown as ComposerTranslation +const mockT = ((key: string) => key) as ProviderTranslator describe('validateProvider', () => { it('resolves async validator factories and validation requirements', async () => { diff --git a/packages/provider-inference/src/validators/run.ts b/packages/provider-inference/src/validators/run.ts new file mode 100644 index 000000000..fd1934bd3 --- /dev/null +++ b/packages/provider-inference/src/validators/run.ts @@ -0,0 +1,213 @@ +import type { + ProviderConfigValidator, + ProviderContext, + ProviderDefinition, + ProviderExtraMethods, + ProviderInstance, + ProviderRuntimeValidator, +} from '../types' + +import { errorMessageFrom, merge } from '@moeru/std' + +export type ProviderValidationStepStatus = 'idle' | 'validating' | 'valid' | 'invalid' +export type ProviderValidationStepKind = 'config' | 'provider' +export interface ProviderValidationStep { + id: string + label: string + status: ProviderValidationStepStatus + reason: string + kind: ProviderValidationStepKind +} + +export interface ProviderValidationPlan { + steps: ProviderValidationStep[] + config: Record + definition: ProviderDefinition + configValidators: ProviderConfigValidator>[] + providerValidators: ProviderRuntimeValidator>[] + providerExtra: ProviderExtraMethods> | undefined + shouldValidate: boolean +} + +interface ProviderValidationValidatorInfo { + kind: ProviderValidationStepKind + index: number + step: ProviderValidationStep +} + +interface ProviderValidationValidatorResult { + reason: string + valid: boolean +} + +interface ProviderValidationValidatorSuccessInfo extends ProviderValidationValidatorInfo { + result: ProviderValidationValidatorResult +} + +interface ProviderValidationValidatorErrorInfo extends ProviderValidationValidatorInfo { + error: unknown +} + +export interface ProviderValidationCallbacks { + onValidatorStart?: (info: ProviderValidationValidatorInfo) => void + onValidatorSuccess?: (info: ProviderValidationValidatorSuccessInfo) => void + onValidatorError?: (info: ProviderValidationValidatorErrorInfo) => void +} + +interface GetProviderValidationIntervalOptions { + definition: ProviderDefinition + contextOptions: ProviderContext + defaultIntervalMs?: number +} + +interface GetValidatorsOfProviderOptions { + definition: ProviderDefinition + config: Record + schemaDefaults: Record + contextOptions: ProviderContext +} + +export function createConfigValidationSteps(configValidators: ProviderConfigValidator>[]): ProviderValidationStep[] { + return configValidators.map(validator => ({ + id: validator.id, + label: validator.name, + status: 'idle' as ProviderValidationStepStatus, + reason: '', + kind: 'config' as ProviderValidationStepKind, + })) +} + +export function createProviderValidationSteps(providerValidators: ProviderRuntimeValidator>[]): ProviderValidationStep[] { + return providerValidators.map(validator => ({ + id: validator.id, + label: validator.name, + status: 'idle' as ProviderValidationStepStatus, + reason: '', + kind: 'provider' as ProviderValidationStepKind, + })) +} + +export async function getProviderValidationIntervalMs(options: GetProviderValidationIntervalOptions) { + const validators = await Promise.all((options.definition.validators?.validateProvider || []).map(creator => creator(options.contextOptions))) + const defaultIntervalMs = options.defaultIntervalMs ?? 15_000 + const intervals = validators + .filter(validator => validator.schedule?.mode === 'interval') + .map(validator => validator.schedule?.intervalMs || defaultIntervalMs) + + if (intervals.length === 0) { + return undefined + } + + return Math.min(...intervals) +} + +export async function getValidatorsOfProvider(options: GetValidatorsOfProviderOptions): Promise { + const { definition } = options + + const configValidators = await Promise.all((definition.validators?.validateConfig || []).map(creator => creator(options.contextOptions))) + const allProviderValidators = await Promise.all((definition.validators?.validateProvider || []).map(creator => creator(options.contextOptions))) + + const providerValidators = allProviderValidators + + const steps: ProviderValidationStep[] = [ + ...createConfigValidationSteps(configValidators), + ...createProviderValidationSteps(providerValidators), + ] + + const normalizedConfig = merge(options.schemaDefaults, options.config) + const validationRequired = definition.validationRequiredWhen ?? (() => false) + const shouldValidate = await validationRequired(normalizedConfig) + + return { + steps, + config: normalizedConfig, + definition, + configValidators: configValidators as ProviderValidationPlan['configValidators'], + providerValidators: providerValidators as ProviderValidationPlan['providerValidators'], + providerExtra: definition.extraMethods as ProviderValidationPlan['providerExtra'], + shouldValidate, + } +} + +export async function validateProvider( + plan: ProviderValidationPlan, + contextOptions: ProviderContext, + callbacks: ProviderValidationCallbacks = {}, +) { + const { configValidators, providerValidators, steps, config, definition, providerExtra } = plan + const runContext = { + ...contextOptions, + validationCache: new Map(), + } + const { onValidatorError, onValidatorStart, onValidatorSuccess } = callbacks + + const configResults = await Promise.all(configValidators.map(async (validatorDefinition, index) => { + const step = steps[index] + step.status = 'validating' + step.reason = '' + onValidatorStart?.({ kind: 'config', index, step }) + try { + const result = await validatorDefinition.validator(config, runContext) + step.status = result.valid ? 'valid' : 'invalid' + step.reason = result.valid ? '' : result.reason + onValidatorSuccess?.({ kind: 'config', index, step, result }) + return result + } + catch (error) { + step.status = 'invalid' + step.reason = errorMessageFrom(error) ?? 'Unknown error' + onValidatorError?.({ kind: 'config', index, step, error }) + return { valid: false, reason: step.reason } + } + })) + + const configIsValid = configResults.every(result => result.valid) + + const providerStepOffset = configValidators.length + if (!configIsValid) { + for (let i = 0; i < providerValidators.length; i++) { + const step = steps[providerStepOffset + i] + step.status = 'invalid' + step.reason = 'Fix configuration checks first.' + } + return steps + } + + let providerInstance: ProviderInstance + try { + providerInstance = await definition.createProvider(config) + } + catch (error) { + for (let i = 0; i < providerValidators.length; i++) { + const step = steps[providerStepOffset + i] + step.status = 'invalid' + step.reason = errorMessageFrom(error) ?? 'Unknown error' + } + return steps + } + + try { + await Promise.all(providerValidators.map(async (validatorDefinition, index) => { + const step = steps[providerStepOffset + index] + step.status = 'validating' + step.reason = '' + onValidatorStart?.({ kind: 'provider', index, step }) + try { + const result = await validatorDefinition.validator(config, providerInstance, providerExtra ?? {}, runContext) + step.status = result.valid ? 'valid' : 'invalid' + step.reason = result.valid ? '' : result.reason + onValidatorSuccess?.({ kind: 'provider', index, step, result }) + } + catch (error) { + step.status = 'invalid' + step.reason = errorMessageFrom(error) ?? 'Unknown error' + onValidatorError?.({ kind: 'provider', index, step, error }) + } + })) + } + finally { + await (providerInstance as ProviderInstance & { dispose?: () => Promise | void }).dispose?.() + } + + return steps +} diff --git a/packages/provider-inference/tsconfig.json b/packages/provider-inference/tsconfig.json new file mode 100644 index 000000000..b73e1f940 --- /dev/null +++ b/packages/provider-inference/tsconfig.json @@ -0,0 +1,34 @@ +{ + "compilerOptions": { + "target": "ESNext", + "lib": [ + "DOM", + "DOM.AsyncIterable", + "ESNext" + ], + "module": "ESNext", + "moduleResolution": "bundler", + "types": [ + "node", + "vitest" + ], + "strict": true, + "strictNullChecks": true, + "noUnusedLocals": true, + "noEmit": true, + "esModuleInterop": true, + "forceConsistentCasingInFileNames": true, + "isolatedModules": true, + "verbatimModuleSyntax": true, + "skipLibCheck": true + }, + "include": [ + "src/**/*.ts", + "tsdown.config.ts", + "vitest.config.ts" + ], + "exclude": [ + "dist", + "node_modules" + ] +} diff --git a/packages/provider-inference/tsdown.config.ts b/packages/provider-inference/tsdown.config.ts new file mode 100644 index 000000000..bfa3793e9 --- /dev/null +++ b/packages/provider-inference/tsdown.config.ts @@ -0,0 +1,7 @@ +import { defineConfig } from 'tsdown' + +export default defineConfig({ + entry: ['./src/index.ts'], + sourcemap: true, + unused: true, +}) diff --git a/packages/provider-inference/vitest.config.ts b/packages/provider-inference/vitest.config.ts new file mode 100644 index 000000000..2df4cdeaf --- /dev/null +++ b/packages/provider-inference/vitest.config.ts @@ -0,0 +1,31 @@ +import { playwright } from '@vitest/browser-playwright' +import { defineConfig } from 'vitest/config' + +export default defineConfig({ + root: import.meta.dirname, + test: { + projects: [ + { + extends: true, + test: { + name: 'node', + include: ['src/**/*.test.ts'], + exclude: ['src/**/*.browser.test.ts'], + }, + }, + { + extends: true, + test: { + name: 'browser', + include: ['src/**/*.browser.test.ts'], + browser: { + enabled: true, + headless: true, + provider: playwright(), + instances: [{ browser: 'chromium' }], + }, + }, + }, + ], + }, +}) diff --git a/packages/stage-pages/package.json b/packages/stage-pages/package.json index 08b5f673e..c3ad943ff 100644 --- a/packages/stage-pages/package.json +++ b/packages/stage-pages/package.json @@ -27,6 +27,7 @@ "@proj-airi/audio": "workspace:^", "@proj-airi/ccc": "workspace:*", "@proj-airi/i18n": "workspace:*", + "@proj-airi/provider-inference": "workspace:^", "@proj-airi/server-sdk": "workspace:*", "@proj-airi/stage-ui": "workspace:*", "@proj-airi/stage-ui-live2d": "workspace:*", diff --git a/packages/stage-pages/src/pages/settings/providers/transcription/browser-web-speech-api.vue b/packages/stage-pages/src/pages/settings/providers/transcription/browser-web-speech-api.vue index be13a9f41..c931a266f 100644 --- a/packages/stage-pages/src/pages/settings/providers/transcription/browser-web-speech-api.vue +++ b/packages/stage-pages/src/pages/settings/providers/transcription/browser-web-speech-api.vue @@ -1,6 +1,7 @@