Reading
mlx-serve is a native LLM inference server written in the Zig language, optimized specifically for Apple Silicon. It provides APIs compatible with OpenAI and Anthropic, with no Python dependencies required.
Section 01
mlx-serve is a pure Zig-language LLM inference server optimized for Apple Silicon (M1/M2/M3/M4), free of Python dependencies. It provides OpenAI and Anthropic compatible APIs, and includes a macOS GUI app MLX Core. Key features include lightweight deployment, high performance, tool calling, and multi-modal support.
Section 02
Python's dominance in AI inference brings deployment complexity and bloated dependencies. mlx-serve uses Zig to directly call Apple's MLX C interface, avoiding Python runtime overhead. Its "No Python" design ensures native execution from model loading to token generation, aiming for faster startup, lower memory usage, and simpler deployment.
Section 03
mlx-serve offers HTTP APIs compatible with OpenAI (supports /v1/chat/completions, /v1/completions, /v1/models) and Anthropic (/v1/messages). It supports both streaming and non-streaming responses, plus tool calling for external tool integration.
Section 04
Key optimizations include KV cache reuse (boosts multi-turn dialogue speed), full sampling parameter control (temperature, top-k, top-p, etc.), and integration of Gemma4's SigLIP encoder for multi-modal reasoning via image_url content blocks. It also supports reasoning/thinking mode with configurable token budget.
Section 05
MLX Core is a macOS menu bar app with: model browser (HuggingFace download with resumable support, architecture detection), multi-session chat (Markdown rendering), agent mode (10 built-in tools like shell, file operations, web search), customizable system prompts, persistent memory, and skill system (add .md files to ~/.mlx-serve/skills/).
Section 06
Supported architectures: Gemma4/3, Qwen3/3.5/3.6, Nemotron-H, Llama, Mistral (examples: gemma-4-e2b-it-4bit, Llama3). Installation options: Homebrew (brew tap ddalcu/mlx-serve then install mlx-core and mlx-serve) or source build (zig build -Doptimize=ReleaseFast, then run with model path and port). API example via curl is provided.
Section 07
mlx-serve represents a direction of using system-level languages (Zig) to rebuild AI infrastructure, free from Python dependency. For Apple Silicon users, it's a high-performance local LLM solution. Its agent and tool calling features make it a full local AI assistant platform. As model quantization and Apple Silicon improve, such native solutions will play a bigger role in edge AI.
Keep going with more reads from the same topic.
Splinter is a minimalist, high-performance key-value (KV) and vector storage system enabling zero-latency inter-process communication via shared memory and atomic operations. With only 766 lines of core code, it supports millions of operations per second and 768-dimensional vector storage, offering a new architectural approach for local LLM inference and data-intensive applications.
Folkering OS is the world's first AI-native bare-metal operating system, entirely written in Rust no_std without relying on Linux, POSIX, or libc. It can generate commands from scratch, compile them into WASM, and run them in 10 seconds, achieving true self-evolution.
This project explores how to use Large Language Models (LLMs) to detect logical vulnerabilities in Ethereum smart contracts, especially business logic flaws that are difficult for traditional static analysis tools to capture, such as economic manipulation and execution order errors.
This article introduces an open-source project for building modern large language models from scratch. The project evolves from a GPT-2-style basic architecture to a Llama 2/3-style production-level implementation, covering complete tutorial implementations of key technologies such as RMSNorm, RoPE, SwiGLU, GQA, and MoE.