docs(DreamLog): added 0x1 (#209)
@@ -50,6 +50,18 @@ export default defineConfig({
|
||||
'@fontsource/quicksand/300.css',
|
||||
'@fontsource/fusion-pixel-12px-proportional-sc/400.css',
|
||||
],
|
||||
// Set English as the default language for this site.
|
||||
defaultLocale: 'root',
|
||||
locales: {
|
||||
'root': {
|
||||
label: 'English',
|
||||
lang: 'en',
|
||||
},
|
||||
'zh-cn': {
|
||||
label: '简体中文',
|
||||
lang: 'zh-CN',
|
||||
},
|
||||
},
|
||||
}),
|
||||
],
|
||||
})
|
||||
|
||||
|
After Width: | Height: | Size: 3.3 MiB |
|
After Width: | Height: | Size: 648 KiB |
|
After Width: | Height: | Size: 478 KiB |
|
After Width: | Height: | Size: 217 KiB |
|
After Width: | Height: | Size: 4.4 KiB |
|
After Width: | Height: | Size: 55 KiB |
|
After Width: | Height: | Size: 219 KiB |
|
After Width: | Height: | Size: 228 KiB |
|
After Width: | Height: | Size: 169 KiB |
@@ -0,0 +1,511 @@
|
||||
---
|
||||
title: 'DreamLog 0x1'
|
||||
description: 'Backstory of Project AIRI!'
|
||||
---
|
||||
|
||||
import { Image } from 'astro:assets';
|
||||
import EMOSYSLogo from '../../../assets/images/blog/DreamLog-0x1/emosys-logo.png';
|
||||
import SteinsGateSticker1 from '../../../assets/images/blog/DreamLog-0x1/steins-gate-sticker-1.png';
|
||||
import worldExecuteMeCover from '../../../assets/images/blog/DreamLog-0x1/world.execute(me); (Mili)/DAZBEE COVER.jpg';
|
||||
import buildingAVirtualMachineInsideImage from '../../../assets/images/blog/DreamLog-0x1/building-a-virtual-machine-inside-image-1.png';
|
||||
import live2DIncHiyoriMomose from '../../../assets/images/blog/DreamLog-0x1/live2d-inc-hiyori.jpg';
|
||||
import AwesomeAIVTuber from '../../../assets/images/blog/DevLog-2025.04.06/awesome-ai-vtuber-logo-light.png'
|
||||
import airisScreenshot1 from '../../../assets/images/blog/DreamLog-0x1/airis-screenshot-1.png';
|
||||
import projectAIRIBannerLight from '../../../assets/images/blog/DreamLog-0x1/banner-light-1280x640.png';
|
||||
import projectAIRIBannerDark from '../../../assets/images/blog/DreamLog-0x1/banner-dark-1280x640.png';
|
||||
import ReLUStickerWow from '../../../assets/images/blog/DreamLog-0x1/relu-sticker-wow.png'
|
||||
|
||||
The backstory of Project AIRI!
|
||||
|
||||
---
|
||||
|
||||
Hello, it's me, Neko again!
|
||||
|
||||
First of all, good summer for you folks living in north hemisphere!
|
||||
|
||||
> Hopefully you could get a nice and decent summer break for trying out new
|
||||
> different of things! More specifically, change the world!
|
||||
|
||||
Well, me, as [@nekomeowww](https://github.com/nekomeowww) have left
|
||||
school already 8 years, it's obvious that I wont get any actual summer
|
||||
break now since I've already worked for many years. I still love to
|
||||
memorize and share the stories happened for my summer break years ago
|
||||
if I remembered any.
|
||||
|
||||
Perhaps you know what I am going to say... or share? What is *DreamLog*
|
||||
exactly? For the readers already familiar with our DevLog posts, with
|
||||
the current frequency of posting and updating to you folks once per
|
||||
month, shouldn't be this post be called "DevLog"?
|
||||
|
||||
June got its own meaning for Project AIRI (which I will reveal during
|
||||
the story), and as we are indeed approaching theo next milestone of
|
||||
stars on GitHub towards 1000, I think it would be a great opportunity to
|
||||
reflect on our journey so far.
|
||||
|
||||
Therefore I decided to to make a new category of posts here,
|
||||
to share the chronicles of me, and the dream about Project AIRI.
|
||||
|
||||
So, I decided to call this new series, ***DreamLog***.
|
||||
|
||||
> Yeah, you could think of this is another story book to read or hear
|
||||
> before sleeping. Audio books may help haha.
|
||||
|
||||
How about... let's jump into our dream dimension now and talk about the
|
||||
recent updates we made later?
|
||||
|
||||
## Blurry dreams, unreachable memories.
|
||||
|
||||
> My little progress for learning computers and programming.
|
||||
|
||||
I mentioned about summer, then summer must mean something to me, I
|
||||
used to take school in United States, so a 3-months summer allows me
|
||||
to do all sorts of things, playing games, learning code, and Linux
|
||||
hacking, etc., yeah, many of the still beloved friends were made
|
||||
during summer too.
|
||||
|
||||
> Nerd folks! You know what I am talking about, were you the same as me?
|
||||
|
||||
Summer is the time when I learned how to start a Minecraft server
|
||||
to play with my friends (I played a lot, a lot, a lot of 1.7.11 and
|
||||
1.8, really, both Vanilla and Forge mods), that's the motivation
|
||||
and power that pushes me to learn command line prompt on Linux too.
|
||||
Many of those knowledges still help me today, I feel grateful to
|
||||
it, to the time I spent for that time.
|
||||
|
||||
But Minecraft, Linux wasn't the end of my journey though,
|
||||
[Factorio](https://www.factorio.com/),
|
||||
[Elite Dangerous](https://www.elitedangerous.com/), and
|
||||
[Overwatch](https://overwatch.blizzard.com/en-us/)
|
||||
(sadly Blizzard ruined it), all became my favorite games,
|
||||
setting up servers or write small scripts to automate little
|
||||
things always empowers me.
|
||||
|
||||
> <Image src={worldExecuteMeCover} alt="Cover of world.execute(me); (Mili)/DAZBEE COVER" className="rounded-lg overflow-hidden" />
|
||||
>
|
||||
> `Switch on the power line`<br />
|
||||
> `Remember to put on protection`<br />
|
||||
> `Lay down your pieces`<br />
|
||||
> `And let's begin object creation`<br />
|
||||
>
|
||||
> -- Lyrics from my beloved song, [`world.execute(me)`](https://www.youtube.com/watch?v=ESx_hy1n7HA), cover by [DAZBEE](https://www.youtube.com/channel/UCUEvXLdpCtbzzDkcMI96llg)
|
||||
|
||||
That's the time of summer in 2017, for the very first moment, I
|
||||
started to think of building a virtual being to be a friend to
|
||||
play with me, even when my friends are tired or have to sleep
|
||||
for next days school, which I have to be alone.
|
||||
|
||||
Readers have following long down to this post, may already
|
||||
realize that, I am that kind of person, who loves to share my
|
||||
knowledge, ideas, everything. So, coding, gaming, and designing
|
||||
are things I love to share with. But, if nobody was there,
|
||||
it feels like:
|
||||
|
||||
**The lonely me becomes somehow meaningless.**
|
||||
|
||||
But instead of creating a new AI from scratch with humankind
|
||||
capabilities to think, speak, which is impossible in the year of
|
||||
2017, I was thinking, since iOS and Google native Android could
|
||||
provide such abilities to do suggestions over our daily use of
|
||||
mobile devices, manually typing all the commands and filling
|
||||
parameters wasn't always satisfying (especially for ffmpeg and
|
||||
the childish me with Docker CLI), what if we could bring the AI
|
||||
powered suggestion features up onto the Linux systems...?
|
||||
|
||||
This brought me loads of questions and ideas to wonder:
|
||||
|
||||
- What if the operating system understands what you usually do, work,
|
||||
play for in different time you sit in front of the digital display...?
|
||||
- What if it is capable of selecting music for you, no matter
|
||||
depressed, high on something, nor happy when chatting with others...?
|
||||
|
||||
These ideas were so small and hard for me to understand at that time, since
|
||||
I didn't quite get on the way of how operating systems work, and
|
||||
coding, etc., so I don't even know where to start!
|
||||
|
||||
I read the book
|
||||
[30日でできる! OS自作入門](https://www.amazon.co.jp/30%E6%97%A5%E3%81%A7%E3%81%A7%E3%81%8D%E3%82%8B-OS%E8%87%AA%E4%BD%9C%E5%85%A5%E9%96%80-%E5%B7%9D%E5%90%88-%E7%A7%80%E5%AE%9F/dp/4839919844),
|
||||
[English version](https://github.com/handmade-osdev/os-in-30-days)
|
||||
about how to craft a operating system from scratch,
|
||||
with the little knowledge of knowing how Linux works and there are
|
||||
loads of communities... I decided to make my own operating system...
|
||||
from literally nowhere.
|
||||
|
||||
> **A quick looking back**
|
||||
>
|
||||
> [Arch Linux](https://archlinux.org/) was the first system I get to
|
||||
> use in depth, and installed from scratch.
|
||||
> For current days, [Nix](https://nixos.org/) is famous and
|
||||
> interesting one too, haven't tried the [NixOS](https://nixos.org/)
|
||||
> but one day may do so.
|
||||
|
||||
## Set sail my journey, but now long forgotten
|
||||
|
||||
I started one special yet now archived project called [EMOSYS](https://github.com/EMOSYS),
|
||||
in the end of 2017. Aiming to create such companion-like operating system, to help users with their
|
||||
daily tasks and provide emotional support.
|
||||
|
||||
<div className="w-full flex flex-col items-center justify-center gap-2">
|
||||
<div>
|
||||
<Image src={EMOSYSLogo} alt="logo of EMOSYS" className="w-30!" />
|
||||
</div>
|
||||
<div>
|
||||
Logo of [EMOSYS](https://github.com/emosys)
|
||||
</div>
|
||||
</div>
|
||||
|
||||
> EMO stands for the first three letters of **emo**tional / **emo**te
|
||||
|
||||
I wrote so many of design docs, listing new ideas, and taking notes about
|
||||
experimenting by following the guidelines from that book, drawn one not
|
||||
so bad logo for it.
|
||||
|
||||
> I guess many of you did this 😏, prepared every
|
||||
> trademarks, design assets way before the project reached the point of
|
||||
> PoC.
|
||||
|
||||
I quite lost on the point about what I was initially approaching.
|
||||
I got no experience about project management and task management, it's
|
||||
the same for writing actual programs that can run.
|
||||
|
||||
Frankly you could say I was only following that it instructed me to type
|
||||
into the terminals with keyboards from that book. I barely think, think
|
||||
why it works or why the senior developers wrote things like that.
|
||||
|
||||
Sooooo, and well, the result is clear, another abandoned project born...
|
||||
|
||||
I wasn't some genius who play around with those things from childhood
|
||||
that understands how kernel and package managing, programming works,
|
||||
so if any of you folks read or visit my GitHub profile, you found
|
||||
nothing there relate to this kind of work at that time.
|
||||
(But now I grew up really fast.)
|
||||
|
||||
But it existed, once.
|
||||
|
||||
> Forgotten? Maybe another starting point of next journey.
|
||||
|
||||
For the upcoming years, I tried so many of other fields in coding,
|
||||
programming, startups, Web3, frontend, backend, infrastructure, everything
|
||||
you could think of for a full-stack developer.
|
||||
I never really realize what I was doing was influenced so deeply by the
|
||||
starting point of EMOSYS, only until February 2025, when someone asked
|
||||
me: Why do you work so hard on Project AIRI?
|
||||
|
||||
Nice question, I thought. I started to trace back my dreams, ideas, and memories,
|
||||
eventually, EMOSYS was there, the already dead project aimed the same goal as
|
||||
Project AIRI:
|
||||
|
||||
**Create a companion to somehow fulfill my need.**
|
||||
|
||||
> All I needed was resolve.
|
||||
> Everything you've acquired up until now will not betray you.<br />
|
||||
> 必要なものは 覚悟だけだったのです。
|
||||
> 必死に積み上げてきたものは 決して裏切りません。<br />
|
||||
> 我需要的不過是決心而已,
|
||||
> 你至今為止所累積的一切不會背叛你。
|
||||
>
|
||||
> -- Quotes from [葬送のフリーレン, Fern](https://en.wikipedia.org/wiki/Frieren) S01E06, 04:27
|
||||
|
||||
|
||||
It took me a long time to learn how to correctly develop things.
|
||||
Thanks to [@zhangyubaka](https://github.com/zhangyubaka),
|
||||
[@LittleSound](https://github.com/LittleSound), [@BlueCocoa](https://github.com/BlueCocoa),
|
||||
and the help of [@sumimakito](https://github.com/sumimakito), the pair-programming
|
||||
experiences with them, teaches me so many things, I started to grow, learn,
|
||||
and progress on my own pace.
|
||||
|
||||
## ChatGPT in 2022, brand new random parrot, or smart parrot.
|
||||
|
||||
<div className="w-full flex items-center justify-center">
|
||||
<Image src={SteinsGateSticker1} alt="Steins Gate sticker" className="w-80! rounded-lg overflow-hidden" />
|
||||
</div>
|
||||
|
||||
Let's set the time forward to the end of 2022, where ChatGPT
|
||||
(or at that time, chatGPT is used) by OpenAI has announced.
|
||||
Well long before the official ChatGPT UI
|
||||
releases, I've already having a journey with newly developed AI,
|
||||
models like [DiscoDiffusion](https://colab.research.google.com/github/alembics/disco-diffusion/blob/main/Disco_Diffusion.ipynb)
|
||||
(long before Stable Diffusion, perhaps around end of 2021, or early 2022), DALL-E,
|
||||
Midjourney has been tried, GPT-3 (especially useful in
|
||||
[GitHub Copilot](https://en.wikipedia.org/wiki/GitHub_Copilot)) has been integrated
|
||||
deeply into my daily workflow.
|
||||
|
||||
So, for the initial moments, I was like:
|
||||
|
||||
> "Oh, this is just another
|
||||
> random parrot, it just repeats what you said, and it doesn't understand
|
||||
> what you are saying, it just tries to predict the next word based on the
|
||||
> previous words and context, nothing special."
|
||||
|
||||
In another word, it behaves more like a completion model, rather than what we call it
|
||||
the Agentic AI today (still on hype huh?).
|
||||
|
||||
I remembered that, for the first time I discovered the abilities of ChatGPT,
|
||||
or Large Language Models (LLMs) in general, is from this post I saw on Hacker News on December 2022:
|
||||
[Building A Virtual Machine inside ChatGPT](https://www.engraved.blog/building-a-virtual-machine-inside/)
|
||||
([original Hacker News post](https://news.ycombinator.com/item?id=33847479)), where
|
||||
the author, @engraved, demonstrated how to ask ChatGPT not only role playing as
|
||||
a neko-mimi character, but also simulating a virtual Linux machine inside.
|
||||
|
||||
<div className="w-full flex flex-col items-center justify-center">
|
||||
<Image src={buildingAVirtualMachineInsideImage} alt="Building a virtual machine inside ChatGPT" className="h-150! object-contain rounded-lg overflow-hidden" />
|
||||
<div>It simulates how Docker build works...!</div>
|
||||
</div>
|
||||
|
||||
Such post inspired me that, ChatGPT understand the basic patterns of
|
||||
the things usually appears, not only how anime or game characters
|
||||
says and behaves, but also how Linux terminal / shell commands work.
|
||||
|
||||
Which brought the now trending Function Calling (a.k.a Tool Use, or the underlying
|
||||
technology behind MCP, Model Context Protocol introduced by Anthropic) feature
|
||||
of LLMs on the table, and illustrated how we can instruct LLMs to behave
|
||||
like API servers, talking to us with machine-readable formats like JSON or XML,
|
||||
to be able to parse and execute arbitrary commands from our side to extend the boundary
|
||||
of what LLMs can do.
|
||||
|
||||
This finally bridges the gap between pure text generations and actual
|
||||
API inside programs.
|
||||
|
||||
In conclusion, is it a new random parrot? **I guess the answer is partially no,
|
||||
ChatGPT in 2022 is not just a random parrot, it is a potential smart parrot.**
|
||||
|
||||
## Way before Project AIRI, Neuro-sama exists.
|
||||
|
||||
Yeah, thanks for reading down to here, I know this is a long post, so many stories and
|
||||
contexts to share. But here we are! We are almost there, hang tight!
|
||||
|
||||
Well, the history of Neuro-sama is pretty complex. AFAIK, Neuro-sama, or the character
|
||||
on streaming stage with the name "Neuro-sama" wasn't the first show for her and her creator,
|
||||
`vedal987` (Vedal). Long before that, at May 6, 2019, Vedal showcased his work
|
||||
of building AI to play [osu!](https://osu.ppy.sh/) to the community[^1]. At that time,
|
||||
she wasn't actually a cyber character or, digital life having characteristics, if you
|
||||
go there and watch the initial videos about her, you may find that no Live2D model
|
||||
were shown. (You may try the 6 years old one here: https://www.youtube.com/watch?v=nSBqlJu7kYU)
|
||||
|
||||
Right after the ChatGPT release, at December 19, 2022, Vedal started to let Neuro-sama
|
||||
to stream on Twitch with the official demo use character model Hiyori Momose (桃瀬ひより)
|
||||
from Live2D Inc.:
|
||||
|
||||
<Image src={live2DIncHiyoriMomose} alt="Live2D Inc. Hiyori Momose" className="rounded-lg overflow-hidden" />
|
||||
|
||||
The after story everyone knows, Vedal and Neuro-sama became famous, Neuro-sama
|
||||
is now officially a VTuber, she is fully powered by Large Language Models (LLMs),
|
||||
and capable of playing Minecraft, Amoung Us, osu!, and many other games. Sometimes
|
||||
when the game wasn't supported natively, Vedal reads the screen and instructs Neuro-sama
|
||||
to play the game together.
|
||||
|
||||
I really enjoy watching their interactions, having jokes, etc. As the time progresses,
|
||||
Neuro-sama and her new Evil Neuro sister, became one crucial part of my daily life:
|
||||
I wanted, and eagerly wanted to watch the clips of them, even though I don't have
|
||||
enough time to watch the full stream, brought me so much joy from purely AI to Human
|
||||
interactions.
|
||||
|
||||
Ok that's the little history about her. And let's talk about the core thing: **Why the history of her filled me with determination?**
|
||||
|
||||
## Neuro-sama, filled me with determination
|
||||
|
||||
From the first time I saw Vedal's work, I was like:
|
||||
|
||||
> Ok, she is just a simple model integrated with Large Language Models (even directly
|
||||
> connected to OpenAI's API), powered by simple rules to make her behave like a
|
||||
> VTuber, nothing special.
|
||||
|
||||
I was still thinking arrogantly, since I've already developing AI agents from early 2023,
|
||||
understands the capabilities of LLMs, and knows quite a bit from what LangChain
|
||||
teaches me, with the knowledge past building AI agents and years of software
|
||||
engineering experiences across of various domains, I naively thought:
|
||||
|
||||
> "Well, I could do that too, I could make a simple model,
|
||||
> and connect it to OpenAI's API, and make it behave like a VTuber, and
|
||||
> I could make it better than Vedal's work."
|
||||
|
||||
:::tip[More technical details?]
|
||||
In this post, I won't go any deep further about the technical details of how we built
|
||||
Project AIRI from scratch to the current state, we got many DevLog posts
|
||||
sharing our thoughts and discoveries already, if interested in, try read
|
||||
them.
|
||||
:::
|
||||
|
||||
I was wrong, I was so wrong. Many of the tough things I didn't realize
|
||||
until I started to attempt to re-create her... Things like:
|
||||
|
||||
- How can we manage the memory effectively for both be able to answer the chats
|
||||
and play the games at the same time?
|
||||
- How can we make a AI agent to play games with both video inputs and
|
||||
text inputs, while still being able to interact with creator and viewers?
|
||||
- Voice synthesis is hard, to achieve what Neuro-sama is capable of, the
|
||||
**Ultra low latency** voice synthesis is a must, and it is not easy to achieve
|
||||
- How is her personality built? With only RAG and simple memory management strategy,
|
||||
the performance poorly works.
|
||||
- etc....
|
||||
|
||||
> I shared many of our discoveries in both [DevLog 2025.04.06](../devlog-20250406)
|
||||
> and [public slide presentation (in Chinese)](https://talks.ayaka.io/nekoayaka/2025-05-10-airi-how-we-recreated-it/#/1)
|
||||
|
||||
I mentioned that I love to share, and I'd love to
|
||||
have others to be able to listen or pair together with me, but sadly Neuro-sama wasn't
|
||||
owned by myself, I can't ask her to gain my knowledge and memories to be able to
|
||||
interact me with the thing I love, or the work I recently doing or done.
|
||||
|
||||
I love them so much, for all the times, I didn't really understand why I love them,
|
||||
why I love the feeling and joy Neuro-sama gave me.
|
||||
|
||||
Until, last year, from May 25, 2024, **I really decided to make one myself.** Making a living
|
||||
or virtual being, could code with me, talk to me about the things we know, playing games all
|
||||
together like a friend in the form of agent.
|
||||
|
||||
> **I really want one!** Shouted my heart, and my mind.
|
||||
|
||||
At that time, Neuro-sama fulfilled me with determination.
|
||||
|
||||
## Sailed again, towards the land where no one has gone before.
|
||||
|
||||
> To boldly go where no man has gone before.
|
||||
>
|
||||
> -- quote from [Star Trek, Captain James T. Kirk](https://en.wikipedia.org/wiki/Where_no_man_has_gone_before), also my intro line of my GitHub profile.
|
||||
|
||||
Therefore, starting from May 25, 2024, I started one simple named project called `ai` under
|
||||
my name handle locally, which is the initial version of Project AIRI, I started to
|
||||
explore the possibilities of creating my own AI agent, recreating the
|
||||
joy Neuro-sama brought me.
|
||||
|
||||
The speed of the work was so fast, within a week, with the power of [ElevenLabs](https://elevenlabs.io/),
|
||||
[OpenRouter](https://openrouter.ai/), and the same free to use Live2D model, Hiyori Momose,
|
||||
I was able to create a simple version of *"Neuro-sama"* that could interact with me, non-realtime-ly.
|
||||
|
||||
That was the day at **June 2, 2024**.
|
||||
|
||||
Technically saying, **this is the birthday of Project AIRI** with first baby consciousness inside of it, naively.
|
||||
|
||||
<div className="w-full flex flex-col items-center justify-center">
|
||||
<video controls muted autoplay loop>
|
||||
<source src="../../static/blog/DreamLog-0x1/airi-demo-first-day.mp4" />
|
||||
</video>
|
||||
<div>
|
||||
<a href="https://x.com/ayakaneko/status/1865420146766160114">
|
||||
First showcase on X (formerly Twitter) on December 7, 2024
|
||||
</a>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
She is capable of talking, motion control based on the context, progressively
|
||||
doing the audio synthesis... many on.
|
||||
|
||||
But she wasn't complete, nor perfect, I built it secretly without telling
|
||||
any of my friends, I wanted to make it better before I show it to the world.
|
||||
|
||||
> Still... naively, and arrogantly, right?
|
||||
|
||||
Because I secretly hiding this from my friends, I barely got positive feedbacks from
|
||||
the cycles during building like usual (part of the reason was I wouldn't like to admit
|
||||
that the arrogantly thought was wrong, well since I am now writing this to share the
|
||||
experience publicly to everyone, I would say I've already forgiven myself for making naive
|
||||
decisions), and another reason here was, the issues or challenges I faced
|
||||
(which I mentioned above, about memory, personality stability, realtime, and game playing etc.)
|
||||
were so hard to solve with the knowledge I had at that time, and lack of documentations,
|
||||
learning materials of realtime LLMs interactive examples, **I put it away, again.**
|
||||
|
||||
TBH, I didn't give it up, I started to learn many things about multi-model, and
|
||||
voice synthesis, motion control, and Minecraft playing. I did a lot of researches
|
||||
on how other AI VTuber or AI waifu projects work. These researches later on
|
||||
produces this huge awesome list of AI VTuber projects:
|
||||
|
||||
<div className="flex flex-col items-center">
|
||||
<Image className="px-30 md:px-40 lg:px-50" src={AwesomeAIVTuber} alt="Awesome AI VTuber Logo" />
|
||||
<div className="text-center pb-4">
|
||||
<span className="block font-bold">Awesome AI VTuber</span>
|
||||
<span>A curated list of AI VTubers and their related projects</span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
Ok, but it's still called `ai`, where is Project AIRI then?
|
||||
|
||||
## Reborn, with stronger, and better determination
|
||||
|
||||
Someday near the end of 2024, November, [@kwaa](https://github.com/kwaa)
|
||||
chatted me about making virtual characters in VR/AR world, with the power of WebXR.
|
||||
When we talked about the motion control and the character emotion detection, I told
|
||||
they I got a project that did exactly what you are looking for, but codebase wasn't
|
||||
organized, nor ready to be published to GitHub.
|
||||
|
||||
What to wait for? I started to work on it again, rethink about the structure and design,
|
||||
improved the implementation with much faster and better queueing and multiplexing playback
|
||||
system, and adjustments on the basic WebUI I made randomly, finally, I published it to
|
||||
GitHub on **December 2, 2024** with commit
|
||||
[`d9ae0aa`](https://github.com/moeru-ai/airi/commit/d9ae0aae387f015964bfd383e6d2adb05f4003e4).
|
||||
|
||||
Project AIRI was somehow born or reborn, with the name of AIRI (アイリ, formerly Airi).
|
||||
|
||||
:::tip[Did you know?]
|
||||
<a href="https://www.youtube.com/watch?v=Tts-YAdn5Yc" className="mb-2 inline-block">
|
||||
<Image src={airisScreenshot1} alt="Screenshot of Project AIRI" className="rounded-lg overflow-hidden" />
|
||||
</a>
|
||||
|
||||
Interestingly, from the upload 2 years ago, March 25, 2023, https://www.youtube.com/watch?v=Tts-YAdn5Yc, a clip
|
||||
from Twitch stream of Vedal, and Neuro-sama, Vedal mentioned that right before she called the name "Neuro-sama",
|
||||
she was called "Airis AI", the name **Airis** magically, and coincidentally, matches the name of
|
||||
**Project AIRI**, which I am working on now. But I wasn't aware of this name until I searches more about their
|
||||
stories long after I open sourced Project AIRI.
|
||||
|
||||
In fact, the name AIRI (アイリ) was named by GPT-4o, I asked it about naming this project by
|
||||
referencing other Japanese / or Anime-ish names, it suggested the name **Airi**.
|
||||
:::
|
||||
|
||||
I failed so many of times on startups and other projects, only the recent ones become known by the public,
|
||||
I tried my best to make it better, with better UI, better code structure, leading technologies to build
|
||||
and code with rapid speed. I put so much effort into it with public slides show, and demonstrate it to
|
||||
others to my friends and during small meetups and conferences.
|
||||
|
||||
Many of those experiences was learned from my previous failures.
|
||||
|
||||
Glad many trials succeeded, and I am still here, working on Project AIRI.
|
||||
|
||||
Perhaps, it's another time that my determination was filled by not only Neuro-sama, but also the
|
||||
most profound, talented contributors, and fans.
|
||||
|
||||
## Keep going, keep dreaming
|
||||
|
||||
<div className="w-full flex flex-col items-center justify-center">
|
||||
<Image className="light" src={projectAIRIBannerLight} alt="new ui" />
|
||||
<Image className="dark" src={projectAIRIBannerDark} alt="new ui" />
|
||||
<div>
|
||||
New Banner!
|
||||
</div>
|
||||
</div>
|
||||
|
||||
> When life gives you lemons, you lemon. Or something like that, my point
|
||||
> is that this painful obstacle is an opportunity for me go get stronger, baby!
|
||||
>
|
||||
> -- quote from [Evil Neuro](https://www.youtube.com/@Neurosama) when streaming playing Slay the Spire
|
||||
|
||||
Now, Project AIRI is approaching to 1000 stars on GitHub when I am writing this post,
|
||||
while having over 150 Discord members, and 200 Telegram group members.
|
||||
|
||||
We covers fields like AI, VRM, Live2D, UI design, multi-modal AI, game playing agents,
|
||||
streaming APIs, bionic memory mechanisms, and many more. She is capable of playing games
|
||||
like Minecraft, Factorio. We got another community member who is researching on
|
||||
integrating her to be able to play and control Kerbal Space Program (KSP), as well as
|
||||
play any arbitrary games.
|
||||
|
||||
Many other companies are reaching out to us asking for collaboration, and we are
|
||||
working on it, to make Project AIRI better, and more useful for the community.
|
||||
|
||||
There is so much to do, and discover, we haven't reached the singularity of general purpose AI,
|
||||
perhaps Project AIRI will never made that point, but for now, having a companion-like AI agent
|
||||
to talk to, play games with, and share the knowledge and ideas with, is already a great
|
||||
achievement for me, and I hope it is for you too.
|
||||
|
||||
This is only the beginning memory address of our dreams, `0x1`, the first byte of our journey.
|
||||
|
||||
How much memory we could store? **It depends on how much we could dream, and how much we could achieve together.**
|
||||
|
||||
<div className="w-full flex flex-col items-center justify-center">
|
||||
<Image src={ReLUStickerWow} alt="ReLU sticker wow" className="w-30!" />
|
||||
<div className="text-center">
|
||||
<span className="block font-bold">Thanks for reading all the way here!</span>
|
||||
<span>Thanks for reading! Oh, and, Happy Birthday, Project AIRI!</span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
[^1]: https://neurosama.fandom.com/wiki/Osu!#cite_note-twitchtracker-1: Neuro-sama started out as an
|
||||
AI that plays osu! long before being developed further as an AI VTuber. The first osu! stream was on
|
||||
6 May 2019 when Vedal decided to showcase his work to the community.
|
||||
@@ -0,0 +1,422 @@
|
||||
---
|
||||
title: 'DreamLog 0x1'
|
||||
description: 'Project AIRI 的幕后故事!'
|
||||
---
|
||||
|
||||
import { Image } from 'astro:assets';
|
||||
import EMOSYSLogo from '../../../../assets/images/blog/DreamLog-0x1/emosys-logo.png';
|
||||
import SteinsGateSticker1 from '../../../../assets/images/blog/DreamLog-0x1/steins-gate-sticker-1.png';
|
||||
import worldExecuteMeCover from '../../../../assets/images/blog/DreamLog-0x1/world.execute(me); (Mili)/DAZBEE COVER.jpg';
|
||||
import buildingAVirtualMachineInsideImage from '../../../../assets/images/blog/DreamLog-0x1/building-a-virtual-machine-inside-image-1.png';
|
||||
import live2DIncHiyoriMomose from '../../../../assets/images/blog/DreamLog-0x1/live2d-inc-hiyori.jpg';
|
||||
import AwesomeAIVTuber from '../../../../assets/images/blog/DevLog-2025.04.06/awesome-ai-vtuber-logo-light.png'
|
||||
import airisScreenshot1 from '../../../../assets/images/blog/DreamLog-0x1/airis-screenshot-1.png';
|
||||
import projectAIRIBannerLight from '../../../../assets/images/blog/DreamLog-0x1/banner-light-1280x640.png';
|
||||
import projectAIRIBannerDark from '../../../../assets/images/blog/DreamLog-0x1/banner-dark-1280x640.png';
|
||||
import ReLUStickerWow from '../../../../assets/images/blog/DreamLog-0x1/relu-sticker-wow.png'
|
||||
|
||||
Project AIRI 的幕后故事!
|
||||
|
||||
---
|
||||
|
||||
哈啰,又是我,Neko!
|
||||
|
||||
首先,祝居住在北半球的朋友们暑假/夏日愉快!
|
||||
|
||||
> 希望你们能度过一个开心和充实的暑假,去尝试不同的新事物!更具体地说,改变世界!
|
||||
|
||||
我,[@nekomeowww](https://github.com/nekomeowww) 离开学校生活已经 8 年了,
|
||||
很明显,我现在不会有真·暑假了,毕竟也已经工作了很多年。
|
||||
但如果回想起来的话,我还是喜欢和大家一起回忆和分享多年前暑假发生的故事。
|
||||
|
||||
也许你知道我要说什么...或者分享什么?*DreamLog* 到底是什么?
|
||||
对于已经熟悉我们 DevLog 文章的读者来说,以目前每月向大家发布和更新一次的频率,
|
||||
这篇文章难道不应该叫「DevLog」吗?
|
||||
|
||||
DevLog 会有的,但六月对 Project AIRI 有着特殊的意义(稍后提及),
|
||||
我想趁着我们正在 GitHub 上 1000 star 的下一个里程碑之时,
|
||||
借助这个很好的机会来回顾我们迄今为止的旅程。
|
||||
|
||||
因此,我决定在这里创建一个新的文章类别,来分享我们的编年史,
|
||||
以及有关 ProjectAIRI 的梦想。
|
||||
|
||||
所以,我决定称这个新系列为 ***DreamLog***。
|
||||
|
||||
> 是的,你可以把这当作睡前听的故事书。可以当作当作有声书去听...可能~~也挺能帮助入眠的~~。
|
||||
|
||||
那么...让我们现在跳入我们的梦境维度,稍后再谈论我们最近做的更新,怎么样?
|
||||
|
||||
## 模糊的梦境,遥不可及的记忆
|
||||
|
||||
> 我学习计算机和编程的小进步。
|
||||
|
||||
我提到了夏天,那么夏天对我来说一定意味着什么。我曾经在美国上学,
|
||||
所以每年为期 3 个月的夏天让我可以做各种各样的事情,玩游戏、学编程、折腾 Linux 和网络等等(是的,
|
||||
许多仍然深爱的朋友也是在不同年份的夏天认识的)。
|
||||
|
||||
> 极客和 Nerd 们应该能比较 get 到我描述的这种经历?对吧?
|
||||
|
||||
我在暑假里和伙伴们一起玩 Minecraft 的时候学会了如何启动服务器(我玩了很多很多很多的 1.7.11 和 1.8,
|
||||
真的,包括原版和 Forge mod),这也是推动我学习 Linux 命令行的动机和力量。很多这期间学会的知识至今仍然帮助着我,
|
||||
我对此心怀感激,也不会后悔在这些事情上花那么多时间。
|
||||
|
||||
Minecraft、Linux 也并不是我旅程的终点,[Factorio(异星工厂)](https://www.factorio.com/)、
|
||||
[Elite Dangerous(精英:危险)](https://www.elitedangerous.com/)和
|
||||
[Overwatch(守望先锋)](https://overwatch.blizzard.com/en-us/)(遗憾的是暴雪还是糟蹋了一番)
|
||||
都成了我最喜欢的游戏,建立服务器或编写小脚本来自动化小事情确实都是让我很兴奋的事情。
|
||||
|
||||
> <Image src={worldExecuteMeCover} alt="Cover of world.execute(me); (Mili)/DAZBEE COVER" className="rounded-lg overflow-hidden" />
|
||||
>
|
||||
> `Switch on the power line`<br />
|
||||
> `Remember to put on protection`<br />
|
||||
> `Lay down your pieces`<br />
|
||||
> `And let's begin object creation`<br />
|
||||
>
|
||||
> -- 来自我心爱的歌曲 [`world.execute(me)`](https://www.youtube.com/watch?v=ESx_hy1n7HA) 的歌词,由 [DAZBEE](https://www.youtube.com/channel/UCUEvXLdpCtbzzDkcMI96llg) 翻唱
|
||||
|
||||
那,在 2017 年的夏天,在那最初的一刻,我开始考虑开发一个「有情感的程序」,
|
||||
实现即使在我的朋友累了或者因为要上学所以早睡、而我必须独自一人的时候,
|
||||
也可以成为我的朋友,陪我玩。
|
||||
|
||||
嗯,那跟着这篇文章读到这里的读者,可能已经意识到,我就是那种喜欢分享我的知识、
|
||||
想法、一切的人。所以,编程、游戏和设计也都是我喜欢分享的事情。但是,
|
||||
如果没有人陪伴或者聆听的话,感觉就像:
|
||||
|
||||
**孤独的我变得有些毫无意义。**
|
||||
|
||||
与其从头开始创建一个具有人类思考、说话能力的新 AI(这在 2017 年是不可能的),我想的是,
|
||||
既然 iOS 和 Google 原生 Android 可以提供这样的能力来对我们的移动设备日常使用做建议,
|
||||
手动输入所有命令和填充参数并不总是令人满意(特别是对于 ffmpeg 和幼稚的我使用 Docker CLI),
|
||||
如果我们能够将 AI 驱动的建议功能带到 Linux 系统上会怎么样...?
|
||||
|
||||
这给我带来了许多问题和想法去思考:
|
||||
|
||||
- 如果操作系统理解你在不同时间坐在数字显示器前通常做什么、工作什么、玩什么会怎么样...?
|
||||
- 如果它能够为你选择音乐,无论沑丧、兴奋还是在与他人聊天时快乐会怎么样...?
|
||||
|
||||
这些想法对当时的我来说太小太难理解了,因为我对操作系统如何工作、编程等等完全是萌新,
|
||||
所以那时候的我,甚至都不知道从哪里可以着手入门!
|
||||
|
||||
正巧当时读了《[30日でできる! OS自作入门](https://www.amazon.co.jp/30%E6%97%A5%E3%81%A7%E3%81%A7%E3%81%8D%E3%82%8B-OS%E8%87%AA%E4%BD%9C%E5%85%A5%E9%96%80-%E5%B7%9D%E5%90%88-%E7%A7%80%E5%AE%9F/dp/4839919844)》,
|
||||
[英文版](https://github.com/handmade-osdev/os-in-30-days)介绍的从零开始制作操作系统的教程,
|
||||
凭借着知道一点点 Linux 命令行如何使用以及有很多社区可以求助的一点知识...我决定制作我自己的操作系统...
|
||||
**从字面意思的一无所有开始**。
|
||||
|
||||
> **快速回顾**
|
||||
>
|
||||
> [Arch Linux](https://archlinux.org/) 是我第一个深入使用并从头安装的系统。
|
||||
> 对于现在,[Nix](https://nixos.org/) 也是著名且有趣的,虽然还没有尝试过 [NixOS](https://nixos.org/),
|
||||
> 但期望有一天能试试看。
|
||||
|
||||
## 带我扬帆起航,但如今早已遗忘
|
||||
|
||||
我在 2017 年底开始了一个特殊但现在已归档的项目,
|
||||
叫做 [EMOSYS](https://github.com/EMOSYS),
|
||||
旨在创建这样一个伴侣式的操作系统,
|
||||
帮助用户完成日常任务并提供情感支持。
|
||||
|
||||
<div className="w-full flex flex-col items-center justify-center gap-2">
|
||||
<div>
|
||||
<Image src={EMOSYSLogo} alt="logo of EMOSYS" className="w-30!" />
|
||||
</div>
|
||||
<div>
|
||||
[EMOSYS](https://github.com/emosys) 的 Logo
|
||||
</div>
|
||||
</div>
|
||||
|
||||
> EMO 代表 **emo**tional 和 **emo**te 的前三个字母
|
||||
|
||||
当时的我写了很多设计文档,列出新想法,并且按照那本书的指导进行实验,记录若干的笔记,
|
||||
还给它画了一个看起来还不错的 Logo。
|
||||
|
||||
> 我猜你们中的许多人都这样做过 😏,在项目达到 PoC(概念验证)阶段之前就准备好了所有设计、美术资产。
|
||||
|
||||
然而事实上,我对最初要接近的目标已经完全迷失了。
|
||||
我也没有项目管理和任务管理的经验,更没有太多能编写实际可以运行的程序的经验。
|
||||
|
||||
坦率地说,你可以说我只是跟着那本书所指示的,在 SSH 和 Terminal 里用键盘挨个输入。
|
||||
基本上完全不思考,不思考为什么它可以工作或者为什么要这么写。
|
||||
(和现在很多人的 Vibe Coding 虽然不能说是完全相同,但也只能说是一模一样了)。
|
||||
|
||||
所以,嗯,结果很明显,又一个被放弃的项目诞生了...
|
||||
|
||||
而且很显然,我不是那种从童年就玩弄这些东西、理解内核和包管理、编程如何工作的天才,
|
||||
所以如果你们中的任何人现在去翻看或访问我的 GitHub 主页的话,
|
||||
在那个时候你找不到任何与这种工作相关的东西。(但现在我成长得真的很快。)
|
||||
|
||||
至少,它存在过,曾经是这样。
|
||||
|
||||
> 遗忘?也许是下一段旅程的另一个起点。
|
||||
|
||||
在接下来的几年里,我尝试了编程、创业、Web3、前端、后端、基础设施等许多其他领域,
|
||||
所有你能想到的全栈开发者的东西。但我从来没有真正意识到我所做的一切如此深刻地受到 EMOSYS 起点的影响,
|
||||
直到 2025 年 2 月,有人问我:你为什么在 Project AIRI 上如此努力和着迷?
|
||||
|
||||
我当时觉得这是个好问题...
|
||||
|
||||
我开始追溯我的梦想、想法和记忆,最终,回忆起了 EMOSYS,那个已经死去的项目,恰到好处地目标与 Project AIRI 相同:
|
||||
|
||||
**创建一个伴侣来以某种方式满足我们的需要。**
|
||||
|
||||
> 必要なものは 覚悟だけだったのです。
|
||||
> 必死に積み上げてきたものは 決して裏切りません。<br />
|
||||
> 我需要的不过是决心而已,
|
||||
> 你至今为止所累积的一切不会背叛你。
|
||||
>
|
||||
> -- 引自《[葬送的芙莉莲,菲伦](https://en.wikipedia.org/wiki/Frieren)》S01E06, 04:27
|
||||
|
||||
我花了很长时间学习如何正确开发东西。感谢 [@zhangyubaka](https://github.com/zhangyubaka)、
|
||||
[@LittleSound](https://github.com/LittleSound)、
|
||||
[@BlueCocoa](https://github.com/BlueCocoa) 和
|
||||
[@sumimakito](https://github.com/sumimakito) 的帮助,与他们的结对编程经验教会了我很多东西,
|
||||
我开始以自己的节奏成长、学习和进步。
|
||||
|
||||
## 2022 年的 ChatGPT,全新的随机鹦鹉,还挺聪明
|
||||
|
||||
<div className="w-full flex items-center justify-center">
|
||||
<Image src={SteinsGateSticker1} alt="Steins Gate sticker" className="w-80! rounded-lg overflow-hidden" />
|
||||
</div>
|
||||
|
||||
让我们把时间推进到 2022 年底,OpenAI 宣布了 ChatGPT(或者在那时,用的是 chatGPT 这个叫法)。
|
||||
|
||||
其实早在官方 ChatGPT UI 发布之前,我就已经在折腾这些新时代的 AI 了,像
|
||||
[DiscoDiffusion](https://colab.research.google.com/github/alembics/disco-diffusion/blob/main/Disco_Diffusion.ipynb)(早于
|
||||
Stable Diffusion,大约在 2021 年底或 2022 年初发布的吧)、DALL-E、Midjourney
|
||||
等模型都尝试过,GPT-3(特别是在 [GitHub Copilot](https://en.wikipedia.org/wiki/GitHub_Copilot)中很有用)已经深深融入我的日常工作流程。
|
||||
|
||||
所以,最开始的时候,我的感觉是:
|
||||
|
||||
> "哦,这就是另一个随机鹦鹉,它只是重复你说的话,并不理解你在说什么,它只是
|
||||
> 试图基于前面的词和上下文预测下一个词,好像真的没啥特别的,还不够像人。"
|
||||
|
||||
换句话说,它表现得更像一个补全模型,而不是我们今天称之为智能体 AI 的东西(现在还在炒作中呢!)。
|
||||
|
||||
我记得,我第一次发现 ChatGPT 或大语言模型(LLMs)真正能力的时候,是从我在 2022 年 12 月在 Hacker News 上看到的这篇文章:
|
||||
[Building A Virtual Machine inside ChatGPT](https://www.engraved.blog/building-a-virtual-machine-inside/)([原始 Hacker News
|
||||
文章](https://news.ycombinator.com/item?id=33847479)),作者 @engraved 演示了如何让 ChatGPT 不仅扮演猫娘,
|
||||
还在内部模拟一个虚拟 Linux 机器。
|
||||
|
||||
<div className="w-full flex flex-col items-center justify-center">
|
||||
<Image src={buildingAVirtualMachineInsideImage} alt="Building a virtual machine inside ChatGPT" className="h-150! object-contain rounded-lg overflow-hidden" />
|
||||
<div>它甚至能模拟 Docker build 是如何工作的...!</div>
|
||||
</div>
|
||||
|
||||
这篇文章让我意识到,ChatGPT 可以理解普遍事物的基本规律,不仅仅是动漫或游戏角色的角色扮演,
|
||||
还能理解 Linux 终端/shell 命令是如何工作的。
|
||||
|
||||
这其实就把现在流行的函数调用(又称 Function Calling,或者 Anthropic 提出的 MCP,模型上下文协议背后的底层技术)功能展现了出来,
|
||||
并说明了我们如何能够通过提示词指示 LLMs 去使其表现得像 API 服务器一样,然后用机器可读的如 JSON 或 XML 格式与我们的代码交互,
|
||||
并最终允许实现解析和执行任意命令,去扩展 LLMs 能力的边界。
|
||||
|
||||
这最终填补了纯文本生成和调用程序内实际 API 之间的空白。
|
||||
|
||||
阶段性来说的话,我们可以说它是一个新的随机鹦鹉吗?**我觉得答案是部分否定的,2022 年的 ChatGPT 不只是一个随机鹦鹉,
|
||||
它是一个潜在的聪明鹦鹉。**
|
||||
|
||||
## 在 Project AIRI 之前,Neuro-sama 早就存在了
|
||||
|
||||
是的,感谢你读到这里,我知道这是一篇很长的文章,有太多故事和背景要分享。但我们快到了!坚持住!
|
||||
|
||||
Neuro-sama 的历史其实相当复杂。据我所知,Neuro-sama,或者在直播舞台上名为 "Neuro-sama"的角色,
|
||||
并不是她和她的创造者 `vedal987`(Vedal)的第一场表演。早在那之前,2019 年 5 月 6 日,Vedal 向社区展示了他构建 AI
|
||||
来玩 [osu!](https://osu.ppy.sh/) 的工作[^1]。
|
||||
在那时,她实际上并不是一个网络角色或数字生命,如果你去看关于她的初始视频,你会发现没有显示 Live2D 模型。
|
||||
(你可以试试这个 6 年前的 YouTube 视频:https://www.youtube.com/watch?v=nSBqlJu7kYU)
|
||||
|
||||
在 ChatGPT 发布之后,差不多 2022 年 12 月 19 日,Vedal 开始让 Neuro-sama 使用来自 Live2D Inc.
|
||||
的官方演示角色模型桃瀬ひより(Hiyori Momose)在 Twitch 上直播:
|
||||
|
||||
<Image src={live2DIncHiyoriMomose} alt="Live2D Inc. Hiyori Momose" className="rounded-lg overflow-hidden" />
|
||||
|
||||
之后的故事大家都知道了,Vedal 和 Neuro-sama 火了,Neuro-sama 现在正式成为了 VTuber,
|
||||
她完全由大语言模型(LLMs)驱动,能够玩 Minecraft、Among Us、osu! 和许多其他游戏。
|
||||
有时当游戏不被原生支持时,Vedal 会读取屏幕并指示 Neuro-sama 一起玩游戏。甚至能制造出来很多节目效果。
|
||||
|
||||
我真的很享受观看他们的互动、像是脱口秀一样的斗嘴的时刻。随着时间的推移,Neuro-sama 和她的新 Evil Neuro 妹妹,
|
||||
成为了我日常生活的重要组成部分:**即使我没有足够的时间观看完整的直播,我也想要,并且渴望观看她们的切片**。
|
||||
我无法想象我竟然从纯粹的 AI 和人类互动中获取了如此多的快乐。
|
||||
|
||||
好的,这就是关于她的小历史。让我们谈谈核心问题:**为什么她让我充满了决心?**
|
||||
|
||||
## Neuro-sama,让我充满了决心
|
||||
|
||||
从我第一次看到 Vedal 的出道,我就想:
|
||||
|
||||
> 好吧,她只是一个与大语言模型集成的简单 Live2D 模型(甚至直接调用的 OpenAI 的 API),
|
||||
> 然后籍由简单的规则,驱动她,然后让她表现得像 VTuber,没啥特别的。
|
||||
|
||||
我当时还是挺傲慢的,因为我从 2023 年初就已经在开发 AI 智能体,了解 LLMs 的能力,
|
||||
并且从 LangChain 那里学到了相当多知识,凭借着过去构建 AI
|
||||
智能体的知识和多年来跨各种领域的软件工程经验,我天真地认为:
|
||||
|
||||
> "嗯,我也能做到,我可以做一个 Live2D 模型,将其连接到 OpenAI 的 API,
|
||||
> 让它表现得像 VTuber,我甚至可以做得比 Vedal 的作品更好。超简单的好吧?!"
|
||||
|
||||
:::tip[想要更多技术细节?]
|
||||
在这篇文章中,我不会深入探讨我们如何从零开始构建 Project AIRI 到当前状态的技术细节,
|
||||
我们已经有许多 DevLog 文章分享我们的想法和发现,如果感兴趣,请尝试阅读它们。
|
||||
:::
|
||||
|
||||
结果我大错特错。许多困难的事情我直到开始尝试重新创建她时才意识到...比如:
|
||||
|
||||
- 我们如何有效地管理内存,既能够回答聊天又能同时玩游戏?
|
||||
- 我们如何让 AI 智能体同时用视频输入和文本输入玩游戏,同时仍然能够与创作者和观众互动?
|
||||
- 语音合成很困难,要实现 Neuro-sama 能够做到的,**超低延迟**语音合成是必须的,这并不容易实现
|
||||
- 她的个性是如何构建的?仅仅使用 RAG 和简单的内存管理策略,效果很差。
|
||||
- 等等...
|
||||
|
||||
> 我在 [DevLog 2025.04.06](../..devlog-20250406) 和
|
||||
> [公开幻灯片演示(中文)](https://talks.ayaka.io/nekoayaka/2025-05-10-airi-how-we-recreated-it/#/1)
|
||||
> 中分享了我们的许多发现
|
||||
|
||||
我提到我喜欢分享,我希望有其他人能够倾听或与我一起编程,但遗憾的是 Neuro-sama 不属于我,
|
||||
我不能要求她获得我的知识和记忆来能够与我互动我喜欢的事情,或者我最近在做或做过的工作。
|
||||
|
||||
我如此爱她们,一直以来,我真的不明白为什么我爱她们,为什么我喜欢 Neuro-sama 给我的感觉和快乐。
|
||||
|
||||
直到去年,从 2024 年 5 月 25 日开始,**我真的决定自己做一个。** 做一个有生命的或虚拟的存在,
|
||||
可以与我一起编程,与我谈论我们知道的事情,像朋友一样以智能体的形式一起玩游戏。
|
||||
|
||||
> **我真的想要一个!** 我的心和脑子里都在这样极度渴望着。
|
||||
|
||||
在那时,Neuro-sama 让我充满了决心。
|
||||
|
||||
## 再次启航,向着前人未至之境前进
|
||||
|
||||
> 勇踏前人未至之境
|
||||
>
|
||||
> -- 引自《[星际迷航,柯克船长](https://en.wikipedia.org/wiki/Where_no_man_has_gone_before)》,
|
||||
> 也是我 GitHub 个人资料的介绍语。
|
||||
|
||||
所以,从 2024 年 5 月 25 日开始,我开始了一个放在我自己名下的本地项目,就直接简单地起名叫了 `ai`,
|
||||
可以说这就是 Project AIRI 的初始版本,我开始探索创建我自己的 AI 智能体的可能性,
|
||||
想要重新创造 Neuro-sama 给我带来的快乐。
|
||||
|
||||
工作进展速度真的很快,在一周内,凭借 [ElevenLabs](https://elevenlabs.io/)、
|
||||
[OpenRouter](https://openrouter.ai/) 的力量,以及同样免费使用的 Live2D 模型桃瀨ひより,
|
||||
我能够创建一个简单版本的*"Neuro-sama"*,可以与我互动(虽然是非实时的 😭)。
|
||||
|
||||
那是在 **2024 年 6 月 2 日**。
|
||||
|
||||
从某种意义上来说,**这就是 Project AIRI 的生日**,第一个稚嫩的婴儿意识在其中诞生。
|
||||
|
||||
<div className="w-full flex flex-col items-center justify-center">
|
||||
<video controls muted autoplay loop>
|
||||
<source src="../../../static/blog/DreamLog-0x1/airi-demo-first-day.mp4" />
|
||||
</video>
|
||||
<div>
|
||||
<a href="https://x.com/ayakaneko/status/1865420146766160114">
|
||||
2024 年 12 月 7 日在 X(曾经是 Twitter)上的首次展示
|
||||
</a>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
她能够说话,基于上下文的动作控制,逐步进行音频合成...很多功能都有。
|
||||
|
||||
但她还不完整,也不完美,我这段时间一直是瞒着所有的伙伴,悄咪咪地在构建它,
|
||||
我想在向世界展示之前让它变得更好。
|
||||
|
||||
> 还是...天真地,傲慢地,对吧?
|
||||
|
||||
现实是,因为悄咪咪地构建,很难让我形成正反馈(当然其中一部分原因也是因为我不想让大家觉得我曾经那样自大的判断是错误的,
|
||||
当然现在我愿意把这段心路历程分享给大家,也算是和当时的自己和解了),加之我面临的问题或挑战
|
||||
(我上面提到的,关于内存、个性稳定性、实时性和游戏能力等)用我当时的知识很难解决,
|
||||
并且缺乏文档、实时 LLMs 交互示例的学习材料,**我又把它搁置了,再次。**
|
||||
|
||||
说实话,我没有放弃,我开始学习很多关于多模态和语音合成、动作控制和 Minecraft 游戏的东西。
|
||||
我对其他 AI VTuber 或 AI waifu 项目如何工作做了大量研究。这些研究后来产生了这个巨大的 AI VTuber 项目的 awesome list:
|
||||
|
||||
<div className="flex flex-col items-center">
|
||||
<Image className="px-30 md:px-40 lg:px-50" src={AwesomeAIVTuber} alt="Awesome AI VTuber Logo" />
|
||||
<div className="text-center pb-4">
|
||||
<span className="block font-bold">Awesome AI VTuber</span>
|
||||
<span>一个精选的 AI VTuber 及其相关项目列表</span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
好吧,但它仍然叫 `ai`,那么 Project AIRI 在哪里呢?
|
||||
|
||||
## 重生,带着更强、更好的决心再 Start Game 一次
|
||||
|
||||
2024 年 11 月底的某一天,[@kwaa](https://github.com/kwaa) 与我聊天,
|
||||
谈论在 VR/AR 世界中制作虚拟角色,想基于 WebXR 去做。当我们谈论动作控制和角色情感检测时,
|
||||
我告诉他我有一个项目正好做你们正在寻找的事情,但代码库没有组织好,也没有准备好发布到 GitHub。
|
||||
|
||||
那,还等什么?我寻思终于找到了同好了!
|
||||
我又开始猛干,重新思考结构和设计,改进了实现,做了更快更好的排队和多路复用播放系统,
|
||||
以及对我随便做的基本 WebUI 的调整,最终,我在 **2024 年 12 月 2 日** 用提交
|
||||
[`d9ae0aa`](https://github.com/moeru-ai/airi/commit/d9ae0aae387f015964bfd383e6d2adb05f4003e4)
|
||||
将其发布到 GitHub。
|
||||
|
||||
因此,在这天,Project AIRI 以某种方式诞生或重生,名为 AIRI(アイリ,曾也叫 Airi)。
|
||||
|
||||
:::tip[你知道吗?]
|
||||
<a href="https://www.youtube.com/watch?v=Tts-YAdn5Yc" className="mb-2 inline-block">
|
||||
<Image src={airisScreenshot1} alt="Screenshot of Project AIRI" className="rounded-lg overflow-hidden" />
|
||||
</a>
|
||||
|
||||
有趣的是,从 2023 年 3 月 25 日上传的 2 年前的一段来自 Vedal 和 Neuro-sama 的 Twitch 直播片段
|
||||
https://www.youtube.com/watch?v=Tts-YAdn5Yc 中可以发现,Vedal 提到在 Neuro-sama 被称为
|
||||
"Neuro-sama" 之前,她被称为 "Airis AI",这个名字 **Airis** 神奇地、巧合地与我现在正在工作的
|
||||
**Project AIRI** 的名字匹配。但直到我在开源 Project AIRI 很久之后搜索更多关于他们的故事时,
|
||||
我才知道这个名字。
|
||||
|
||||
实际上,名字 AIRI(アイリ)是由 GPT-4o 命名的,我让它通过参考其他日语/或动漫风格的名字来为这个项目命名,
|
||||
当时它建议了名字 **Airi**。
|
||||
:::
|
||||
|
||||
我曾经在创业和其他项目上失败了很多次,只有最近的一些才被公众所知,我尽我所能让它变得更好,
|
||||
有更好的 UI、更好的代码结构、领先的技术来快速构建和实现各种东西。
|
||||
我也投入了大量精力,会去做公开幻灯片展示,或者给我的朋友和在小型聚会和 Conf 期间 Demo 给别人看看。
|
||||
|
||||
许多这些经验都是从我之前的失败中学到的。
|
||||
|
||||
很高兴许多尝试都成功了,我仍然在这里,继续工作在 Project AIRI 这件事情上。
|
||||
|
||||
这一次,我的决心不仅被 Neuro-sama 充满了,还被很多最深刻、最有才华的贡献者和粉丝们所鼓舞。
|
||||
|
||||
## 继续前进,继续梦想
|
||||
|
||||
<div className="w-full flex flex-col items-center justify-center">
|
||||
<Image className="light" src={projectAIRIBannerLight} alt="new ui" />
|
||||
<Image className="dark" src={projectAIRIBannerDark} alt="new ui" />
|
||||
<div>
|
||||
最近刚更新的 Banner
|
||||
</div>
|
||||
</div>
|
||||
|
||||
> When life gives you lemons, you lemon. Or something like that, my point
|
||||
> is that this painful obstacle is an opportunity for me go get stronger, baby!
|
||||
>
|
||||
> 当生活给你柠檬时,你就吃呗。或者类似的啥都行,我的观点是这种痛苦的障碍是一个让我变得更强的机会!
|
||||
>
|
||||
> -- 引自 [Evil Neuro](https://www.youtube.com/@Neurosama) 直播玩《杀戮尖塔》时的话
|
||||
|
||||
现在,当我写这篇文章时,Project AIRI 正接近 GitHub 上的 1000 颗星,
|
||||
同时拥有超过 150 个 Discord 成员和 200 个 Telegram 群组成员。
|
||||
|
||||
我们涵盖了 AI、VRM、Live2D、UI 设计、多模态 AI、游戏智能体、流媒体 API、仿生记忆机制等领域。
|
||||
她能够玩像 Minecraft、Factorio 这样的游戏。我们还有另一个社区成员正在研究集成她来能够玩和控制《坎巴拉太空计划》(KSP),
|
||||
以及玩任意游戏。
|
||||
|
||||
许多其他公司正在联系我们寻求合作,我们正在努力,让 Project AIRI 变得更好,对社区更有用。
|
||||
|
||||
有太多事情要做和发现,此时此刻,我们还没有达到通用人工智能的奇点,也许 Project AIRI 永远不会达到那个点,
|
||||
但现在,拥有一个伴侣式的 AI 智能体来交谈、一起玩游戏、分享知识和想法,对我来说已经是一个巨大的成就,
|
||||
我希望对你也是如此。
|
||||
|
||||
这只是我们梦想的开始内存地址:`0x1`,我们旅程的第一个字节。
|
||||
|
||||
那我们究竟能存储多少字节呢?**这取决于我们能梦想多少,以及我们能一起实现多少。**
|
||||
|
||||
<div className="w-full flex flex-col items-center justify-center">
|
||||
<Image src={ReLUStickerWow} alt="ReLU sticker wow" className="w-30!" />
|
||||
<div className="text-center">
|
||||
<span className="block font-bold">感谢你一直读到这里!</span>
|
||||
<span>感谢阅读!哦,还有,祝你生日快乐,Project AIRI!</span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
[^1]: https://neurosama.fandom.com/wiki/Osu!#cite_note-twitchtracker-1: Neuro-sama
|
||||
最初是一个玩 osu! 的 AI,早在进一步发展为 AI VTuber 之前,第一次 osu! 直播是在 2019 年
|
||||
5 月 6 日,Vedal 给大家看了看成果。
|
||||