アイリ VTuber

[Try it] [简体中文文档]

Heavily inspired by Neuro-sama

Current progress

Capable of

• Brain
  • Play Minecraft
  • Play Factorio (WIP, but PoC and demo available)
  • Chat in Telegram
  • Chat in Discord
• Ears
  • Audio input from browser
  • Audio input from Discord
  • Client side speech recognition
  • Client side talking detection
• Mouth
  • ElevenLabs voice synthesis
• Body
  • VRM support
    • Control VRM model
  • VRM model animations
    • Auto blink
    • Auto look at
    • Idle eye movement
  • Live2D support
    • Control Live2D model
  • Live2D model animations
    • Auto blink
    • Auto look at
    • Idle eye movement

Development

pnpm i

pnpm dev

Sub-projects born from this project

• unspeech: Universal endpoint proxy server for /audio/transcriptions and /audio/speech, like LiteLLM but for any ASR and TTS
• hfup: tools to help on deploying, bundling to HuggingFace Spaces
• @proj-airi/drizzle-duckdb-wasm: Drizzle ORM driver for DuckDB WASM
• @proj-airi/duckdb-wasm: Easy to use wrapper for @duckdb/duckdb-wasm
• @proj-airi/lobe-icons: Iconify JSON bundle for amazing AI & LLM icons from lobe-icons, support Tailwind and UnoCSS
• @proj-airi/elevenlabs: TypeScript definitions for ElevenLabs API
• Airi Factorio: Allow Airi to play Factorio
• Factorio RCON API: RESTful API wrapper for Factorio headless server console
• autorio: Factorio automation library
• `tstl-plugin-reload-factorio-mod: Reload Factorio mod when developing
• 🥺 SAD: Documentation and notes for self-host and browser running LLMs

Models used

• onnx-community/whisper-large-v3-turbo · Hugging Face

Similar Projects

• SugarcaneDefender/z-waif: Great at gaming, autonomous, and prompt engineering
• semperai/amica: Great at VRM, WebXR
• elizaOS/eliza: Great examples and software engineering on how to integrate agent into various of systems and APIs
• ardha27/AI-Waifu-Vtuber: Great about Twitch API integrations
• InsanityLabs/AIVTuber: Nice UI and UX
• IRedDragonICY/vixevia
• t41372/Open-LLM-VTuber
• PeterH0323/Streamer-Sales

Acknowledgements

• pixiv/ChatVRM
• josephrocca/ChatVRM-js: A JS conversion/adaptation of parts of the ChatVRM (TypeScript) code for standalone use in OpenCharacters and elsewhere
• Design of UI and style was inspired by Cookard, UNBEATABLE, and Sensei! I like you so much!, and artworks of Ayame by Mercedes Bazan with Wish by Mercedes Bazan
• mallorbc/whisper_mic
• xsai: Implemented a decent amount of packages to interact with LLMs and models, like Vercel AI SDK but a lot more smaller.