doc(minecraft): update for perception pipeline
This commit is contained in:
Rin
+19
-111
@@ -1,108 +1,6 @@
|
||||
# ⛏️ Minecraft agent player for [アイリ (AIRI)](https://airi.moeru.ai)
|
||||
# WIP
|
||||
|
||||
> [!NOTE]
|
||||
>
|
||||
> This project is part of the [Project アイリ (AIRI)](https://github.com/moeru-ai/airi), we aim to build a LLM-driven VTuber like [Neuro-sama](https://www.youtube.com/@Neurosama) (subscribe if you didn't!) if you are interested in, please do give it a try on [live demo](https://airi.moeru.ai).
|
||||
|
||||
An intelligent Minecraft bot powered by LLM. AIRI can understand natural language commands, interact with the world, and assist players in various tasks.
|
||||
|
||||
## 🎥 Preview
|
||||
|
||||
|
||||
|
||||
## ✨ Features
|
||||
|
||||
- 🗣️ Natural language understanding
|
||||
- 🏃♂️ Advanced pathfinding and navigation
|
||||
- 🛠️ Block breaking and placing
|
||||
- 🎯 Combat and PvP capabilities
|
||||
- 🔄 Auto-reconnect on disconnection
|
||||
- 📦 Inventory management
|
||||
- 🤝 Player following and interaction
|
||||
- 🌍 World exploration and mapping
|
||||
|
||||
## 🚀 Getting Started
|
||||
|
||||
### 📋 Prerequisites
|
||||
|
||||
- 📦 Node.js 23+
|
||||
- 🔧 pnpm
|
||||
- 🎮 A Minecraft server (1.20+)
|
||||
|
||||
### 🔨 Installation
|
||||
|
||||
1. Clone the repository:
|
||||
|
||||
```bash
|
||||
git clone https://github.com/moeru-ai/airi.git
|
||||
cd services/minecraft
|
||||
```
|
||||
|
||||
2. Install dependencies:
|
||||
|
||||
```bash
|
||||
pnpm install
|
||||
```
|
||||
|
||||
3. Create a `.env.local` file with your configuration:
|
||||
|
||||
> [!NOTE]
|
||||
> For all online accounts, un-comment the following line to toggle Microsoft authentication.
|
||||
> Link for authentication will popup when the bot starts.
|
||||
>
|
||||
> After signed in, according to [how Minecraft protocol was implemented](https://github.com/PrismarineJS/node-minecraft-protocol/blob/bf89f7e86526c54d8c43f555d8f6dfa4948fd2d9/src/client/microsoftAuth.js#L7-L16)
|
||||
> and also, [authentication flow implemented here](https://github.com/PrismarineJS/prismarine-auth/blob/1aef6e1387d94fca839f2811d17ac6659ae556b4/src/MicrosoftAuthFlow.js#L59-L69),
|
||||
> the token will be cached with [the cache IDs specified here](https://github.com/PrismarineJS/prismarine-auth/blob/1aef6e1387d94fca839f2811d17ac6659ae556b4/src/MicrosoftAuthFlow.js#L88-L93)
|
||||
> in split files:
|
||||
>
|
||||
> - `${hash}_live-cache.json`
|
||||
> - `${hash}_mca-cache.json`
|
||||
> - `${hash}_xbl-cache.json`
|
||||
>
|
||||
> inside of the directory provided by [`minecraft-folder-path`](https://github.com/simonmeusel/minecraft-folder-path)
|
||||
>
|
||||
> Linux: `~/.minecraft/nmp-cache/`
|
||||
> macOS: `~/Library/Application Support/minecraft/nmp-cache/`
|
||||
> Windows: `%appdata%/.minecraft/nmp-cache/`
|
||||
>
|
||||
> where `${hash}` is the `sha1` hash of the username you signing in with (as Minecraft username).
|
||||
|
||||
```env
|
||||
OPENAI_API_KEY=your_openai_api_key
|
||||
OPENAI_API_BASEURL=your_openai_api_baseurl
|
||||
|
||||
BOT_USERNAME=your_bot_username
|
||||
BOT_HOSTNAME=localhost
|
||||
BOT_PORT=25565
|
||||
BOT_AUTH='microsoft' # comment if you use offline mode
|
||||
BOT_VERSION=1.20
|
||||
```
|
||||
|
||||
1. Start the bot:
|
||||
|
||||
```bash
|
||||
pnpm dev
|
||||
```
|
||||
|
||||
## 🎮 Usage
|
||||
|
||||
Once the bot is connected, you can interact with it using chat commands in Minecraft. All commands start with `#`.
|
||||
|
||||
### Basic Commands
|
||||
|
||||
- `#help` - Show available commands
|
||||
- `#follow` - Make the bot follow you
|
||||
- `#stop` - Stop the current action
|
||||
- `#come` - Make the bot come to your location
|
||||
|
||||
### Natural Language Commands
|
||||
|
||||
You can also give the bot natural language commands, and it will try to understand and execute them. For example:
|
||||
|
||||
- "Build a house"
|
||||
- "Find some diamonds"
|
||||
- "Help me fight these zombies"
|
||||
- "Collect wood from nearby trees"
|
||||
**Caution: Documentation below may be out of date.**
|
||||
|
||||
## 🧠 Cognitive Architecture
|
||||
|
||||
@@ -154,12 +52,19 @@ graph TB
|
||||
|
||||
**Location**: `src/cognitive/perception/`
|
||||
|
||||
The perception layer acts as the sensory input hub, receiving and preprocesses all events from the Minecraft world and external sources.
|
||||
The perception layer acts as the sensory input hub, collecting raw signals from Mineflayer and turning them into higher-level, rate-limited perception events.
|
||||
|
||||
**Components**:
|
||||
- **Event Manager** (`event-manager.ts`): Centralized event distribution system
|
||||
- Emits standardized `BotEvent` objects
|
||||
- Supports event prioritization and concurrency
|
||||
**Pipeline**:
|
||||
- Mineflayer listeners collect **raw perception events** (sight/hearing/felt), including distance and line-of-sight when applicable.
|
||||
- Raw events are queued in a buffer and drained on the cognitive tick.
|
||||
- An attention detector aggregates events via leaky buckets and emits attention/perception events **only on threshold crossing** (e.g. sustained movement, punching, teabagging, interesting sounds).
|
||||
|
||||
**Key files**:
|
||||
- `mineflayer-perception-collector.ts`
|
||||
- `raw-events.ts`
|
||||
- `raw-event-buffer.ts`
|
||||
- `attention-detector.ts`
|
||||
- `pipeline.ts`
|
||||
|
||||
### Layer B: Reflex
|
||||
|
||||
@@ -217,7 +122,11 @@ Player: "build a house"
|
||||
src/
|
||||
├── cognitive/ # 🧠 Perception → Reflex → Conscious → Action
|
||||
│ ├── perception/ # Event ingestion
|
||||
│ │ └── event-manager.ts # Normalizes raw Mineflayer events
|
||||
│ │ ├── mineflayer-perception-collector.ts
|
||||
│ │ ├── raw-events.ts
|
||||
│ │ ├── raw-event-buffer.ts
|
||||
│ │ ├── attention-detector.ts
|
||||
│ │ └── pipeline.ts
|
||||
│ ├── reflex/ # Fast, rule-based reactions
|
||||
│ │ └── reflex-manager.ts
|
||||
│ ├── conscious/ # LLM-powered reasoning
|
||||
@@ -266,7 +175,6 @@ src/
|
||||
|
||||
- **Reflex Layer**:
|
||||
- 🏃 Dodge hostile mobs
|
||||
- 🍖 Auto-eat when health/hunger is low
|
||||
- 🛡️ Emergency combat responses
|
||||
|
||||
- **Conscious Layer**:
|
||||
|
||||
Reference in New Issue
Block a user