mlx-swift-chain: A Local LLM Long Document Processing Framework for Apple Silicon

mlx-swift-chain is a document processing chain framework designed specifically for MLX Swift. It aims to address the context bottleneck of local LLMs on Apple Silicon devices, offering three processing strategies (Stuff, MapReduce, Adaptive), supporting professional chunkers, enabling fully local and privacy-first long-document reasoning, and integrating SwiftUI components to facilitate application development.

Running local LLMs on Apple Silicon devices is an important privacy-preserving choice, but local models are usually limited by small context windows (e.g., Gemma only has 8192 tokens). Truncating long documents (like 20,000 words) leads to loss of key information, so mlx-swift-chain was developed to focus on long-document reasoning above the model layer, enabling fully local and private processing.

Three Key Processing Strategies

• StuffChain: When text fits into the context, call once with zero extra overhead.
• MapReduceChain: Split ultra-long documents into chunks for reasoning (Map) then merge and reduce (Reduce), supporting recursive reduction.
• AdaptiveChain: Default recommendation; automatically selects Stuff/MapReduce based on input length and other factors.

Professional Chunkers

Optimized for specific document types: TranscriptChunker (meeting records), MarkdownHeadingChunker (MD documents), DocumentStructureChunker (PDF/structured documents), LogChunker (Xcode logs), AppleCrashReportChunker (crash reports), CodeBlockAwareChunker (MD with code blocks).

Token Budget Management

AdaptiveChain makes decisions based on system prompts, task prompts, input length, and reserved output tokens (default: 512). It supports precise token counting or heuristic estimation to avoid prompts taking up too much context.

SwiftUI Integration

Provides an @Observable and @MainActor ChainRunner component, supporting real-time display of processing stages, streaming token output, and ChainResult (including source chunk references and performance metrics). It's natively designed in Swift with no Python bridges or HTTP overhead.

Privacy-First Design

• All processing is done on the device; no network required.
• Supports fully offline scenarios.
• No telemetry, no data reporting.
• Source text references allow tracing conclusions back to their origins.

Typical Applications

• Meeting record summarization: Extract key decisions and action items, preserving speaker attribution.
• Development log analysis: Locate root causes of Xcode build/test crashes.
• Offline document reading: Generate hierarchical summaries.
• Personal voice memos: Organize into action lists.

Ecosystem Collaboration

As a supplementary layer above MLX Swift, mlx-swift-chain focuses on orchestration issues like document chunking, prompt budgeting, and result reduction. Underlying model loading and inference are handled by the MLX ecosystem.

Performance Best Practices

• Default single concurrency (maxConcurrentMapTasks:1) to adapt to Apple Silicon GPU serialized inference.
• Minimal overhead for streaming output; MLXBackend internally uses streamResponse by default.

mlx-swift-chain fills an important gap in local LLM long-document processing within the Apple ecosystem. Through intelligent orchestration and divide-and-conquer strategies, it expands the practical boundaries of underlying models, providing a useful tool for developers who value privacy and need to process sensitive long documents on the device.