Prometeu: Running Distributed Large Language Model Inference on 2011 Vintage Hardware

Prometeu is an open-source project developed by maxwellmelo. Its core goal is to implement multi-node collaborative large language model inference on 2011 Intel Sandy Bridge architecture (e.g., i7-2620M) vintage hardware via the RPC backend of llama.cpp. The project does not rely on modern GPUs at all, successfully runs the Qwen 2.5 1.5B quantized model, and provides a public online demo (prometeu.mx3dev.com). Its value lies in proving that old hardware can participate in meaningful AI computing, offering a feasible alternative for resource-constrained scenarios.

The Prometeu project is available on GitHub (link: https://github.com/maxwellmelo/prometeu, updated on June 16, 2026). The target hardware is the Intel i7-2620M processor released in 2011 (8GB RAM, no discrete graphics card), which does not support modern instruction sets such as AVX2, BMI2, FMA, or F16C. The project successfully runs the Qwen 2.5 1.5B quantized model on it, achieving an inference speed of approximately 9 tokens per second.

The Prometeu system consists of three core components: a master node and two worker nodes, all running in Linux containers on the same physical machine. The master node runs llama-server, providing an OpenAI-compatible HTTP API, loading model weights, and coordinating RPC communication; the worker nodes run rpc-server, each carrying out computation tasks for part of the model's tensor graph. Additionally, Cloudflare Tunnel is used to enable public HTTPS access, FastAPI handles request forwarding and cluster telemetry, and the frontend supports SSE streaming output and real-time node status display.

The Sandy Bridge architecture lacks subsequent instruction set extensions like AVX2 and BMI2, leading GCC to potentially generate illegal instructions (e.g., shlx) in -march=native mode. The project solves this with a dual strategy: 1. CMake configuration explicitly disables unsupported instruction set options; 2. Compiler flags add parameters such as -mno-bmi, -mno-bmi2, -mno-avx2, -mno-fma, -mno-f16c to disable them, and lock the target architecture to -march=sandybridge to ensure stable operation of the binary file.

In a LAN three-node configuration, the inference speed is approximately 9.18 tokens per second; for public HTTPS access, it is about 6.7 tokens per second. First token latency: about 0.5 seconds after cache warm-up, and cold start requires 30 to 90 seconds to complete model loading. Although the speed is not as fast as modern GPUs (the same model can reach over 100 tokens per second), it is cost-effective and proves that old hardware can undertake meaningful AI computing tasks.

Deployment requirements: Debian/Ubuntu Linux host (master node), worker nodes of the same architecture, and a quantized model in GGUF format. Installation requires compiling llama.cpp from source (enabling RPC support and configuring compiler flags matching the hardware), and the repository contains complete build scripts to simplify the process. For a quick experience, you can visit the online demo: prometeu.mx3dev.com.

The Prometeu project breaks the perception that 'AI inference requires expensive dedicated hardware' and allows old devices to participate in AI computing through a distributed architecture. It has important reference value for edge computing, IoT integration, AI popularization in resource-constrained areas, and e-waste recycling. Summary: The project is highly creative; although its performance is not comparable to GPUs, it has zero cost and high accessibility, making it suitable for education, experiments, and edge deployment, and provides a reference implementation for developers to explore the boundaries of AI inference.