Failed Star: A Tutorial Project for Building an Apple Silicon Local LLM Inference Engine from Scratch

Failed Star (fs) is a self-contained LLM inference engine designed specifically for Apple Silicon, implemented from scratch using Rust and Metal Shading Language. Its goal is to serve as an open textbook for learning the principles of inference engineering.

• Original Author/Maintainer: Curtis Alexander
• Source Platform: GitHub
• Original Title: Failed Star (fs)
• Original Link: https://github.com/curtisalexander/fs
• Release Time: June 2026

---

Failed Star (codenamed fs) is a self-contained LLM inference engine built specifically for Apple Silicon, initiated and open-sourced by Curtis Alexander. The unique aspect of this project is that it does not aim for maximum performance; instead, it prioritizes "readability" and "teachability"—every line of code is designed to be read, understood, and learned.

The project name is derived from an astronomical concept: a brown dwarf is a "failed star" that lacks sufficient mass to sustain nuclear fusion, being smaller and dimmer than main-sequence stars. Failed Star is exactly the "little brother" of the famous project Dwarf Star (ds4, a DeepSeek-V4 inference engine developed by antirez). If ds4 targets running large MoE models on Macs with 96GB+ RAM, then Failed Star focuses on running tiny models on the 64GB MacBook Pro (M5), trading off features for understandability.

---

Inference engineering for large language models is a complex field. Reading theory about attention mechanisms is one thing, but writing the kernel code for computing attention by hand and watching tokens flow from your own code is a completely different experience. Failed Star exists precisely for the latter.

The core goal of the project is to "understand inference through building". It provides three key learning resources as its "spine":

1. Theoretical Foundation — Philip Kiely's Inference Engineering (2026), which provides the "why" and terminology system
2. Real-World Implementation — antirez's ds4 as a reference implementation, showing "how professionals do it"
3. Architectural Context — Free articles by Sebastian Raschka, helping to understand the relationships between different LLM architectures

---

The technology stack choice of Failed Star reflects its educational positioning:

• Host Language: Rust —— Responsible for model loading, tokenizer, orchestration, sampling, and KV caching
• GPU Kernel: MSL (Metal Shading Language) —— Each operation is in a separate file, consistent with the metal/ shader style of ds4
• Metal Calls: Directly invoked via raw FFI and Objective-C runtime, no wrapper crates used to ensure transparency

The project starts with tiny dense models (Llama-3.2-1B / Qwen3-0.6B level), using standard vanilla attention mechanisms (RoPE + GQA + SwiGLU + RMSNorm) for easy inspection and debugging. This choice allows learners to truly understand the working principle of each component, rather than being overwhelmed by complex engineering details.

Correctness is verified via "golden vectors"—comparing with the logits output from the official model implementation. Python is only used as a one-time verification tool and never as a second inference engine.

---