From 3c7b42a9137e51363518267a134436c04ff529fb Mon Sep 17 00:00:00 2001 From: Rin Date: Sat, 10 Jan 2026 19:38:28 +0800 Subject: [PATCH] doc(minecraft): update for perception pipeline --- services/minecraft/README.md | 130 +++++------------------------------ 1 file changed, 19 insertions(+), 111 deletions(-) diff --git a/services/minecraft/README.md b/services/minecraft/README.md index c6330ae87..9e9b00678 100644 --- a/services/minecraft/README.md +++ b/services/minecraft/README.md @@ -1,108 +1,6 @@ -# ⛏️ Minecraft agent player for [アイリ (AIRI)](https://airi.moeru.ai) +# WIP -> [!NOTE] -> -> This project is part of the [Project アイリ (AIRI)](https://github.com/moeru-ai/airi), we aim to build a LLM-driven VTuber like [Neuro-sama](https://www.youtube.com/@Neurosama) (subscribe if you didn't!) if you are interested in, please do give it a try on [live demo](https://airi.moeru.ai). - -An intelligent Minecraft bot powered by LLM. AIRI can understand natural language commands, interact with the world, and assist players in various tasks. - -## 🎥 Preview - -![demo](./docs/preview.avif) - -## ✨ Features - -- 🗣️ Natural language understanding -- 🏃‍♂️ Advanced pathfinding and navigation -- 🛠️ Block breaking and placing -- 🎯 Combat and PvP capabilities -- 🔄 Auto-reconnect on disconnection -- 📦 Inventory management -- 🤝 Player following and interaction -- 🌍 World exploration and mapping - -## 🚀 Getting Started - -### 📋 Prerequisites - -- 📦 Node.js 23+ -- 🔧 pnpm -- 🎮 A Minecraft server (1.20+) - -### 🔨 Installation - -1. Clone the repository: - -```bash -git clone https://github.com/moeru-ai/airi.git -cd services/minecraft -``` - -2. Install dependencies: - -```bash -pnpm install -``` - -3. Create a `.env.local` file with your configuration: - -> [!NOTE] -> For all online accounts, un-comment the following line to toggle Microsoft authentication. -> Link for authentication will popup when the bot starts. -> -> After signed in, according to [how Minecraft protocol was implemented](https://github.com/PrismarineJS/node-minecraft-protocol/blob/bf89f7e86526c54d8c43f555d8f6dfa4948fd2d9/src/client/microsoftAuth.js#L7-L16) -> and also, [authentication flow implemented here](https://github.com/PrismarineJS/prismarine-auth/blob/1aef6e1387d94fca839f2811d17ac6659ae556b4/src/MicrosoftAuthFlow.js#L59-L69), -> the token will be cached with [the cache IDs specified here](https://github.com/PrismarineJS/prismarine-auth/blob/1aef6e1387d94fca839f2811d17ac6659ae556b4/src/MicrosoftAuthFlow.js#L88-L93) -> in split files: -> -> - `${hash}_live-cache.json` -> - `${hash}_mca-cache.json` -> - `${hash}_xbl-cache.json` -> -> inside of the directory provided by [`minecraft-folder-path`](https://github.com/simonmeusel/minecraft-folder-path) -> -> Linux: `~/.minecraft/nmp-cache/` -> macOS: `~/Library/Application Support/minecraft/nmp-cache/` -> Windows: `%appdata%/.minecraft/nmp-cache/` -> -> where `${hash}` is the `sha1` hash of the username you signing in with (as Minecraft username). - -```env -OPENAI_API_KEY=your_openai_api_key -OPENAI_API_BASEURL=your_openai_api_baseurl - -BOT_USERNAME=your_bot_username -BOT_HOSTNAME=localhost -BOT_PORT=25565 -BOT_AUTH='microsoft' # comment if you use offline mode -BOT_VERSION=1.20 -``` - -1. Start the bot: - -```bash -pnpm dev -``` - -## 🎮 Usage - -Once the bot is connected, you can interact with it using chat commands in Minecraft. All commands start with `#`. - -### Basic Commands - -- `#help` - Show available commands -- `#follow` - Make the bot follow you -- `#stop` - Stop the current action -- `#come` - Make the bot come to your location - -### Natural Language Commands - -You can also give the bot natural language commands, and it will try to understand and execute them. For example: - -- "Build a house" -- "Find some diamonds" -- "Help me fight these zombies" -- "Collect wood from nearby trees" +**Caution: Documentation below may be out of date.** ## 🧠 Cognitive Architecture @@ -154,12 +52,19 @@ graph TB **Location**: `src/cognitive/perception/` -The perception layer acts as the sensory input hub, receiving and preprocesses all events from the Minecraft world and external sources. +The perception layer acts as the sensory input hub, collecting raw signals from Mineflayer and turning them into higher-level, rate-limited perception events. -**Components**: -- **Event Manager** (`event-manager.ts`): Centralized event distribution system - - Emits standardized `BotEvent` objects - - Supports event prioritization and concurrency +**Pipeline**: +- Mineflayer listeners collect **raw perception events** (sight/hearing/felt), including distance and line-of-sight when applicable. +- Raw events are queued in a buffer and drained on the cognitive tick. +- An attention detector aggregates events via leaky buckets and emits attention/perception events **only on threshold crossing** (e.g. sustained movement, punching, teabagging, interesting sounds). + +**Key files**: +- `mineflayer-perception-collector.ts` +- `raw-events.ts` +- `raw-event-buffer.ts` +- `attention-detector.ts` +- `pipeline.ts` ### Layer B: Reflex @@ -217,7 +122,11 @@ Player: "build a house" src/ ├── cognitive/ # 🧠 Perception → Reflex → Conscious → Action │ ├── perception/ # Event ingestion -│ │ └── event-manager.ts # Normalizes raw Mineflayer events +│ │ ├── mineflayer-perception-collector.ts +│ │ ├── raw-events.ts +│ │ ├── raw-event-buffer.ts +│ │ ├── attention-detector.ts +│ │ └── pipeline.ts │ ├── reflex/ # Fast, rule-based reactions │ │ └── reflex-manager.ts │ ├── conscious/ # LLM-powered reasoning @@ -266,7 +175,6 @@ src/ - **Reflex Layer**: - 🏃 Dodge hostile mobs - - 🍖 Auto-eat when health/hunger is low - 🛡️ Emergency combat responses - **Conscious Layer**: