Running Llama 2 Large Model in Minecraft: An AI Miracle Achieved with Pure Commands

This article introduces an amazing project that fully runs the Llama 2 large language model in Minecraft using only vanilla commands, supporting text generation and multi-turn dialogue functions.

• Original Author/Maintainer: terryguo3180-eng
• Source Platform: GitHub
• Original Title: Minecraft-LLM
• Original Link: https://github.com/terryguo3180-eng/Minecraft-LLM
• Release Date: May 28, 2026

Can you imagine running a complete large language model in the blocky world of Minecraft? It sounds like a fantasy, but the Minecraft-LLM project has made it happen—using only Minecraft's vanilla commands, it implements the full inference process of the Llama 2 large language model in the game, including tokenization, forward propagation, and sampling generation. This project not only demonstrates the technical possibility but also reflects the creator's deep understanding of game mechanics and AI principles.

This project was inspired by two important open-source projects:

First, Andrej Karpathy's llama2.c project, a minimalist Llama 2 inference engine implemented in pure C, which shows that the core principles of large language models can be realized with extremely concise code. Second, Xiaodou's Math Lib project, a floating-point arithmetic library based on Minecraft's scoreboard system, which proves the feasibility of performing complex mathematical calculations in the game's command system.

Combining the capabilities of these two projects, the creator had a bold idea: since llama2.c can implement LLM inference in C, and Math Lib can do floating-point operations in Minecraft, then theoretically it is completely possible to run the Llama 2 model in Minecraft.

Implementing a large language model in Minecraft faces many challenges:

1. Computational Power Limitation: Minecraft's command system is not designed for complex calculations, and the number of commands that can be executed per tick is limited
2. Data Representation Issue: Need to find a way to store and compute neural network parameters in the command system
3. Floating-point Operations: Neural networks rely on floating-point operations, but Minecraft natively only supports integers
4. Memory Limitation: Model parameters require a lot of storage space, and the game's data packs have size limits
5. Performance Bottleneck: Command execution speed is much slower than native code, leading to significantly longer inference time

The core breakthrough of the project is the implementation of floating-point operations using Minecraft's scoreboard system. The scoreboard was originally a mechanism for tracking player scores, but the creator cleverly transformed it into a numerical storage and calculation unit.

Specifically, the project uses fixed-point representation: multiply the floating-point number by a large constant (such as 10000) and approximate it with an integer. For example, the value 3.14159 is stored as 31416. During operations, the system automatically handles decimal point alignment and precision issues.

The model's weight parameters are explicitly written in the params.mcfunction file, where each parameter is a command that stores the value into a specific scoreboard variable. Although this storage method is inefficient, it is fully compatible with Minecraft's command system.

For a dialogue model with 15M parameters, these commands form a huge data pack. Although the loading time is long, once loaded, the model can run normally in the game.

The forward propagation process of the model is fully implemented by command chains:

1. Embedding Layer: Convert input token IDs into embedding vectors
2. Attention Mechanism: Calculate query, key, and value matrices, and perform scaled dot-product attention
3. Feedforward Network: Process the attention output through two linear transformations and activation functions
4. Layer Normalization: Normalize the output of each layer
5. Output Sampling: Calculate the probability distribution based on logits and sample to generate the next token

Each operation step corresponds to a large number of scoreboard operation commands, which are carefully organized into function files and executed in order.