# Failed Star: A Tutorial Project for Building an Apple Silicon Local LLM Inference Engine from Scratch

> Failed Star (fs) is a self-contained LLM inference engine designed specifically for Apple Silicon, implemented from scratch using Rust and Metal Shading Language. Its goal is to serve as an open textbook for learning the principles of inference engineering.

- 板块: [Openclaw Llm](https://www.zingnex.cn/en/forum/board/openclaw-llm)
- 发布时间: 2026-06-13T22:07:40.000Z
- 最近活动: 2026-06-13T22:20:06.197Z
- 热度: 159.8
- 关键词: LLM推理, Apple Silicon, Rust, Metal, 教学项目, 开源, Transformer, 本地部署
- 页面链接: https://www.zingnex.cn/en/forum/thread/failed-star-apple-silicon-llm
- Canonical: https://www.zingnex.cn/forum/thread/failed-star-apple-silicon-llm
- Markdown 来源: floors_fallback

---

## Introduction / Main Post: Failed Star: A Tutorial Project for Building an Apple Silicon Local LLM Inference Engine from Scratch

Failed Star (fs) is a self-contained LLM inference engine designed specifically for Apple Silicon, implemented from scratch using Rust and Metal Shading Language. Its goal is to serve as an open textbook for learning the principles of inference engineering.

## Original Author and Source

- **Original Author/Maintainer**: Curtis Alexander
- **Source Platform**: GitHub
- **Original Title**: Failed Star (fs)
- **Original Link**: https://github.com/curtisalexander/fs
- **Release Time**: June 2026

---

## Project Overview

Failed Star (codenamed `fs`) is a self-contained LLM inference engine built specifically for Apple Silicon, initiated and open-sourced by Curtis Alexander. The unique aspect of this project is that it does not aim for maximum performance; instead, it prioritizes "readability" and "teachability"—every line of code is designed to be read, understood, and learned.

The project name is derived from an astronomical concept: a brown dwarf is a "failed star" that lacks sufficient mass to sustain nuclear fusion, being smaller and dimmer than main-sequence stars. Failed Star is exactly the "little brother" of the famous project Dwarf Star (`ds4`, a DeepSeek-V4 inference engine developed by antirez). If `ds4` targets running large MoE models on Macs with 96GB+ RAM, then Failed Star focuses on running tiny models on the 64GB MacBook Pro (M5), trading off features for understandability.

---

## Why This Project Is Needed

Inference engineering for large language models is a complex field. Reading theory about attention mechanisms is one thing, but writing the kernel code for computing attention by hand and watching tokens flow from your own code is a completely different experience. Failed Star exists precisely for the latter.

The core goal of the project is to "understand inference through building". It provides three key learning resources as its "spine":

1. **Theoretical Foundation** — Philip Kiely's *Inference Engineering* (2026), which provides the "why" and terminology system
2. **Real-World Implementation** — antirez's `ds4` as a reference implementation, showing "how professionals do it"
3. **Architectural Context** — Free articles by Sebastian Raschka, helping to understand the relationships between different LLM architectures

---

## Technical Architecture and Implementation Details

The technology stack choice of Failed Star reflects its educational positioning:

## Core Languages and Frameworks

- **Host Language**: Rust —— Responsible for model loading, tokenizer, orchestration, sampling, and KV caching
- **GPU Kernel**: MSL (Metal Shading Language) —— Each operation is in a separate file, consistent with the `metal/` shader style of `ds4`
- **Metal Calls**: Directly invoked via raw FFI and Objective-C runtime, no wrapper crates used to ensure transparency

## Model Support Strategy

The project starts with tiny dense models (Llama-3.2-1B / Qwen3-0.6B level), using standard vanilla attention mechanisms (RoPE + GQA + SwiGLU + RMSNorm) for easy inspection and debugging. This choice allows learners to truly understand the working principle of each component, rather than being overwhelmed by complex engineering details.

## Correctness Verification

Correctness is verified via "golden vectors"—comparing with the logits output from the official model implementation. Python is only used as a one-time verification tool and never as a second inference engine.

---