LlamaPad：原生 macOS/iOS 本地大模型聊天应用，隐私优先的端侧 AI 体验

LlamaPad is a native macOS/iOS chat app that runs large language models (LLMs) entirely locally, prioritizing user privacy. Built on llama.cpp and Apple's MLX framework, it features sandboxed design, zero cloud dependency, Kokoro TTS integration, and Jinja template support. This post breaks down its background, tech, features, and future plans.

LlamaPad was born to address privacy risks and offline limitations of cloud-based AI tools. Its design focuses on three key principles:

1. Localization: All inference happens on-device, no data sent to servers.
2. Privacy: Sandboxed architecture with read-only access to selected files, no network/microphone access.
3. Native Experience: Optimized for Apple Silicon devices to leverage their hardware capabilities.

LlamaPad uses two core components:

• llama.cpp: A highly optimized C++ engine for efficient LLM inference across hardware.
• MLX: Apple's ML framework for M-series chips, utilizing neural engines and unified memory. It supports GGUF model format (standard in llama.cpp ecosystem), with configurable inference parameters (temperature, Top-P, repetition penalty, DRY/XTC). Users can download models from Hugging Face etc.

LlamaPad offers full dialogue management: create/rename/copy/delete threads, each with independent system prompts. UI features:

• Modern message bubbles.
• Collapsible <think> sections for model reasoning (e.g., DeepSeek-R1, Gemma4).
• Message controls: edit, regenerate, delete,续写. This allows flexible task-specific conversations.

To boost long-dialogue speed, LlamaPad uses:

• Anchored window strategy: Slides window to keep recent context, reducing reprocessing.
• KV cache quantization: Compress F16 cache to lower precision for less memory. Note: Some models may have compatibility issues with quantized cache + Flash Attention (documented).

Built-in Kokoro-82M TTS (via MLX) generates natural English speech offline (no cloud). Users can trigger manual/auto play. Future plans: Add visual model support (image analysis), speech-to-text for full voice interaction.

Upcoming features:

• MCP protocol for tool calls.
• Pure MLX backend & OpenAI-compatible API support.
• Token probability visualization, memory system. Deployment steps: Clone repo + submodules, build llama.cpp Apple framework, run in Xcode (signing needed for iPad). Model selection: Choose size based on device memory; lazy loading (no auto-load on startup) optimizes resources.

LlamaPad demonstrates that modern Apple devices can run LLMs locally while protecting privacy. It's ideal for privacy-focused users, offline AI needs, and tech enthusiasts. As end-side model efficiency and Apple Silicon power grow, local AI apps like LlamaPad will have wider use cases.