docs(devlog): added new DevLog for 2025.05.16

2025-05-28 18:00:39 +08:00
parent cb5ce84f50
commit 367cb8e7d1
16 changed files with 318 additions and 1 deletions
@@ -0,0 +1,313 @@
+---
+title: DevLog @ 2025.05.16
+---
+
+import { Image } from 'astro:assets';
+
+import VelinLight from '../../../assets/images/blog/DevLog-2025.05.16/velin-light.png'
+import VelinDark from '../../../assets/images/blog/DevLog-2025.05.16/velin-dark.png'
+
+import CharacterCardMenuLight from '../../../assets/images/blog/DevLog-2025.05.16/character-card-menu-light.png'
+import CharacterCardMenuDark from '../../../assets/images/blog/DevLog-2025.05.16/character-card-menu-dark.png'
+
+import CharacterCardSettingsLight from '../../../assets/images/blog/DevLog-2025.05.16/character-card-settings-light.png'
+import CharacterCardSettingsDark from '../../../assets/images/blog/DevLog-2025.05.16/character-card-settings-dark.png'
+
+import CharacterCardShowcaseLight from '../../../assets/images/blog/DevLog-2025.05.16/character-card-showcase-light.png'
+import CharacterCardShowcaseDark from '../../../assets/images/blog/DevLog-2025.05.16/character-card-showcase-dark.png'
+
+import VelinPlaygroundLight from '../../../assets/images/blog/DevLog-2025.05.16/velin-playground-light.png'
+import VelinPlaygroundDark from '../../../assets/images/blog/DevLog-2025.05.16/velin-playground-dark.png'
+
+import DemoDayHangzhou1 from '../../../assets/images/blog/DevLog-2025.05.16/demo-day-hangzhou-1.jpeg'
+import DemoDayHangzhou2 from '../../../assets/images/blog/DevLog-2025.05.16/demo-day-hangzhou-2.jpeg'
+import DemoDayHangzhou3 from '../../../assets/images/blog/DevLog-2025.05.16/demo-day-hangzhou-3.jpeg'
+
+Hello again! Here's [Neko](https://github.com/nekomeowww), the girl who started
+the [Project AIRI](https://github.com/moeru-ai/airi)!
+
+Sorry for being late for new update in
+Project AIRI through the posted DevLog, please forgive us
+for the delay.
+
+> We wrote many fantastic DevLogs about our development
+> progress once a while for the past months for AIRI, where
+> we share out thoughts, ideas, explaining in technologies
+> we use, artworks inspired from... everything.
+>
+> - [v0.4.0 UI update](./DevLog-2025.03.20.mdx)
+> - [v0.4.0 release & memory introduced](./DevLog-2025.04.06.mdx)
+>
+> I wrote these two amazing and beloved DevLogs too! Hope you
+> enjoy reading them.
+
+# Dejavu
+
+For the past few weeks, the major quests of Project AIRI itself
+haven't progressed for a while, perhaps I was quite burned
+out from the huge UI refactoring and release since March
+2025. Most of the work was done by community maintainers,
+
+Most profoundly appreciated to [@LemonNekoGH](https://github.com/LemonNekoGH) and
+[@RainbowBird](https://github.com/luoling8192),
+[@LittleSound](https://github.com/LittleSound) for their work done in the field of
+
+- Character Card support
+
+:::note[What is Character Card?]
+Well, local-first chat applications like [SillyTavern](https://github.com/SillyTavern/SillyTavern)
+, [RisuAI](https://risuai.net/) or online services like [JanitorAI](https://janitorai.com/) uses a
+file contains character's background, personality, and other role-playing essential context for
+each individual characters.
+
+- https://realm.risuai.net/
+- https://aicharactercards.com/
+- https://chub.ai/
+
+Character cards aren't the only thing for storing sharing LLM driven role-playing
+characters, [Lorebook](https://docs.novelai.net/text/lorebook.html) plays another key role
+in this field, but this is totally another story worth to write a entire series of
+documentation to share, for now, try read [Void's Lorebook Types](https://rentry.co/lorebooks-and-you) and
+[AI Dynamic Storytelling Wiki](https://aids.miraheze.org/wiki/Main_Page).
+
+> I personally love this wiki for learning these concepts:
+> [AI Dynamic Storytelling Wiki](https://aids.miraheze.org/wiki/Main_Page),
+> worth reading if you are interested in AI role-playing.
+:::
+
+> To use Character Card, navigate to Settings page (top right
+> corner of the app, or hovering Gear icon in desktop app),
+> Find and click the "Airi Card" button.
+
+<Image className="light" src={CharacterCardMenuLight} alt="screenshot of a menu offers Airi Card menu button" />
+<Image className="dark" src={CharacterCardMenuDark} alt="screenshot of a menu offers Airi Card menu button" />
+
+> This will bring you to the "Airi Card editor screen", where you can upload and
+> edit your character card for persona customization.
+
+<Image className="light" src={CharacterCardSettingsLight} alt="screenshot of a menu offers Airi Card menu button" />
+<Image className="dark" src={CharacterCardSettingsDark} alt="screenshot of a menu offers Airi Card menu button" />
+
+For Character Card showcases, we tried some approaches too...
+
+<Image className="light" src={CharacterCardShowcaseLight} alt="a card like user interface design for a blue hair character called ReLU" />
+<Image className="dark" src={CharacterCardShowcaseDark} alt="a card like user interface design for a blue hair character called ReLU" />
+
+It's live in our UI component library, you can play around with it: https://airi.moeru.ai/ui/#/story/src-components-menu-charactercard-story-vue .
+
+> Pure CSS and JavaScript controlled, layout works so we don't need to worry about the canvas calculation.
+>
+> Oh and most of the work for the character card showcase
+> was done and instructed by [@LittleSound](https://github.com/LittleSound),
+> much appreciated.
+
+- Tauri MCP support
+- Connects AIRI to Android devices
+
+These two was major update and try-out, she wrote another two DevLogs about
+these things and shared the technical details behind the scene.
+(Valuable for Tauri developers and users I guess.) You can read them here:
+
+- [Controls Android](./DevLog-2025.04.22.mdx)
+- [MCP in Tauri](./DevLog-2025.04.28.md)
+
+## Project AIRI major quests
+
+### Ears listening, and mouth speaking
+
+From April 15, I found both VAD (voice activation detection),
+[ASR (a.k.a. automatic speech recognition)](https://huggingface.co/tasks/automatic-speech-recognition),
+and [TTS (text to speech)](https://huggingface.co/tasks/text-to-speech) in AIRI
+are very complex and hard to use and understand, for that time, I was cooperating
+with [@himself65](https://github.com/himself65) to improve and test the use cases
+for the new project from [Llama Index](https://www.llamaindex.ai/), a library
+to help to process the event based stream of LLM streaming tokens, and audio
+bytes, called [`llama-flow`](https://github.com/run-llama/llama-flow).
+
+[`llama-flow`](https://github.com/run-llama/llama-flow) is really small,
+and type-safe to use. In the old days without it, I have to manually wrap
+another **queue** structured, and Vue's reactivity powered workflow system
+to chain many asynchronous tasks together to be able to process data to power AIRI.
+
+That was the time I started to experiment more examples, demos on simplifying VAD,
+ASR, TTS workflow.
+
+Eventually, I got this:
+[WebAI Realtime Voice Chat Examples](https://github.com/proj-airi/webai-example-realtime-voice-chat),
+which I managed to proof the work can be done on Web browser within one single
+300 ~ 500 lines of TypeScript code to achieve ChatGPT voice chat system.
+
+<video controls muted style={{ height: '640px' }}>
+  <source src="../../static/blog/DevLog-2025.05.16/webai-examples-demo.MP4"/>
+</video>
+
+I tried my best to split all the possible steps into small and reusable pieces
+to help demonstrate how you can construct a real-time voice chat system
+from ground up and scratch:
+
+- [VAD](https://github.com/proj-airi/webai-example-realtime-voice-chat/tree/8462ff6bcb83bb278bce5388d588d2e3e3dd6dae/apps/vad)
+- [VAD + ASR](https://github.com/proj-airi/webai-example-realtime-voice-chat/tree/8462ff6bcb83bb278bce5388d588d2e3e3dd6dae/apps/vad-asr)
+- [VAD + ASR + LLM Chat](https://github.com/proj-airi/webai-example-realtime-voice-chat/tree/8462ff6bcb83bb278bce5388d588d2e3e3dd6dae/apps/vad-asr-chat)
+- [VAD + ASR + LLM Chat + TTS](https://github.com/proj-airi/webai-example-realtime-voice-chat/tree/8462ff6bcb83bb278bce5388d588d2e3e3dd6dae/apps/vad-asr-chat-tts)
+
+We discovered a interesting and powerful repository though, called [k2-fsa/sherpa-onnx](https://github.com/k2-fsa/sherpa-onnx), it
+supports the 18 tasks of speech processing across macOS, Windows, Linux, Android,
+iOS, etc. over 12 languages. Fascinating!
+
+So [@luoling](https://github.com/luoling8192) made another small demo for this too:
+[Sherpa ONNX powered VAD + ASR + LLM Chat + TTS](https://github.com/proj-airi/webai-example-realtime-voice-chat/tree/main/apps/sherpa-onnx-demo)
+
+#### Birth of xsAI 🤗 Transformers.js
+
+This give birth to a new side project called
+[xsAI 🤗 Transformers.js](https://github.com/proj-airi/xsai-transformers)
+to be able to call the WebGPU powered model inference and serving with workers
+while still keep the API compatible to our prior succeeded project called
+[xsAI](https://github.com/moeru-ai/xsai).
+
+We got a playground for that too... play it on https://xsai-transformers.netlify.app
+
+You can install it via npm today!
+
+```bash
+npm install xsai-transformers
+```
+
+:::note[What does this mean?]
+This means you can swap between cloud LLM and speech providers and local WebGPU
+powered models with one if switch.
+
+This brought us a new possibility to be able to experiment and even achieve
+simple RAG and re-ranking system right in the browser, without the need of
+any server side code, or even a backend server.
+:::
+
+### Telegram Bot
+
+I added the support of Telegram bot to be able to process animated stickers,
+powered by `ffmpeg` (what else, obviously).
+It now can read and understand the animated stickers and even videos sent
+by users.
+
+The system prompt was way too huge, I managed to reduce the size of the
+system prompt drastically to save more than **80%** of the token usage.
+
+### Character Card showcase
+
+Many image assets requires me to manually find a suitable and easy to use
+online solutions to remove backgrounds, but I decided to make my on based
+on the work that [Xenova](https://github.com/xenova) have done... to make
+one for my own.
+
+I did some small experiments on integrating a WebGPU powered background
+remover right in the system, you can play around with it here in
+https://airi.moeru.ai/devtools/background-remove .
+
+### xsAI & unSpeech
+
+We added support for Alibaba Cloud Model Studio and Volcano Engine as speech
+provider, quite useful I guess?
+
+### UI
+
+- New [Tutorial stepper](https://airi.moeru.ai/ui/#/story/src-components-misc-steppers-steppers-story-vue?variantId=src-components-misc-steppers-steppers-story-vue-0), [File upload](https://airi.moeru.ai/ui/#/story/src-components-form-input-inputfile-story-vue?variantId=default), and [Textarea](https://airi.moeru.ai/ui/#/story/src-components-form-textarea-textarea-story-vue?variantId=default) component
+- Color issues
+- [Typography improved](https://airi.moeru.ai/ui/#/story/stories-typographysans-story-vue?)
+
+More of the stories can be found at [Roadmap v0.5](https://github.com/moeru-ai/airi/issues/113)
+
+## Side-quests
+
+### [Velin](https://github.com/luoling8192/velin)
+
+Since we got the Character Card supported, the feeling wasn't so good and
+smooth when dealing with template variable rendering, and component reusing...
+
+What if...
+
+- We could maintain a component prompt library that can be used for other agent or role-playing applications, or even character cards?
+  - For example:
+    - having a medieval fantasy background settings for magic and dragons
+    - the only thing we need to do is to focus our writing on our new character when wrap the world settings outside of it
+    - perhaps, only when the time goes to night, special prompts will get injected through `if` and `if-else` control flow
+  - We can do more things around it...
+    - with Vue SFC or React JSX, we can parse the template and identify the props, render a form panel for debugging and testing while writing prompts
+    - visualize the entire lorebook and character card in a single interactive page
+
+So why don't we make a tool to write LLM prompts with frontend frameworks like Vue or React, and maybe extend this beyond to other frameworks and platforms?
+
+This is what we got: [**Velin**](https://github.com/luoling8192/velin).
+
+<Image className="light" src={VelinLight} alt="tool to write LLM prompts with Vue.js" />
+<Image className="dark" src={VelinDark} alt="tool to write LLM prompts with Vue.js" />
+
+We even made a playground for editing it and render it on-the-fly,
+while enjoying the ecosystem of npm packages (yes you can import any!).
+
+<Image className="light" src={VelinPlaygroundLight} alt="tool to write LLM prompts with Vue.js" />
+<Image className="dark" src={VelinPlaygroundDark} alt="tool to write LLM prompts with Vue.js" />
+
+Try it here: https://velin-dev.netlify.app
+
+Programmatic API supported too, Markdown (MDX WIP, MDC supported),
+you can install it via npm today!
+
+```bash
+npm install @velin-dev/core
+```
+
+Well... that's all for today, I hope you enjoy reading this DevLog.
+
+Let's end with DevLog with more images from the recent event we attended in Hangzhou, China: The **Demo Day @ Hangzhou**.
+
+<Image src={DemoDayHangzhou1} alt="Demo Day @ Hangzhou" />
+
+This is me, I shared the AIRI project with other attendees, and we had a great time
+there! Meet so many of the talented developers, product designers, and entrepreneurs.
+
+Introduced almost everything I shared today in this DevLog, and also the beloved AI VTuber Neuro-sama.
+
+The slides I used to share was this:
+
+<Image src={DemoDayHangzhou2} alt="Demo Day @ Hangzhou" />
+<Image src={DemoDayHangzhou3} alt="Demo Day @ Hangzhou" />
+
+The slides itself is purely open sourced, you can play around
+it here too: https://talks.ayaka.io/nekoayaka/2025-05-10-airi-how-we-recreated-it/#/1
+
+## Milestones
+
+Oh... and since this DevLog also indicates the release of v0.5.0,
+I would love to mention some of the milestone we reached in the past few weeks:
+
+- We reached the 700 stars!
+- 4+ new fresh contributors in issues!
+- 72+ new fresh group members in Discord server!
+- ReLU character design finished!
+- ReLU character modeling finished!
+- Negotiated with a few companies for sponsorships, and cooperation!
+- 92 tasks finished for [Roadmap v0.5](https://github.com/moeru-ai/airi/issues/113)
+  - UI
+    - Loading screen and tutorial modules
+    - Multiple bug fixes including loading states and Firefox compatibility issues
+  - Body
+    - Motion embedding and RAG from semantic, developed in private repo "moeru-ai/motion-gen"
+    - Vector storage and retrieval using embedding providers and DuckDB WASM
+  - Inputs
+    - Fixed Discord Voice Channel speech recognition
+  - Outputs
+    - Experimental singing capabilities
+  - Engineering
+    - Shared UnoCSS configuration across projects
+    - Model catalog in "moeru-ai/inventory"
+    - Package reorganization across organizations
+  - Assets
+    - New character assets including stickers, UI elements, VTuber logos
+    - Voice line selection functionality
+    - Live2D modeling for characters "Me" and "ReLU"
+  - Community Support & Marketing
+    - Japanese README
+    - Plausible analytics integration
+    - Comprehensive documentation
+
+See you!
@@ -48,7 +48,11 @@ starlight-menu-button button svg {
 .content-panel .sl-container h6 {
  margin-top: 2em;
  color: #000;
-  font-size: 1em;
+  font-size: 0.8em;
+}
+
+.sl-container h2 {
+  font-size: 1.5em;
 }

@media (max-width: 72rem) {