iki-nano: An Elegant Solution for Running Large Language Models Locally on iPhone

iki-nano is an open-source iOS app that supports local downloading and running of quantized language models on iPhones, enabling a fully offline AI conversation experience. The project uses SwiftUI and MVVM architecture, integrates MediaPipe Tasks GenAI and LiteRT-LM frameworks, and provides a complete reference implementation for local LLM inference on mobile devices. Its design philosophy emphasizes privacy protection (data remains local), no network dependency (usable offline), and instant response (eliminating network latency).

Most mainstream LLM applications currently rely on cloud APIs, which have privacy risks and network dependency issues. iki-nano aims to achieve fully local model inference on iOS devices through MediaPipe Tasks GenAI and Google LiteRT-LM frameworks. The significance of local-first design includes: user data always stays on the device, works normally without a network, and eliminates network latency for instant response.

iki-nano is developed in Swift, uses SwiftUI to build the UI, follows MVVM architecture, supports iOS 17.0 and above, and uses CocoaPods for dependency management. The machine learning framework supports dual engines: MediaPipe Tasks GenAI (cross-platform ML solution) and LiteRT-LM (lightweight runtime framework). Core features include remote model download, local storage management, model configuration, and a SwiftUI interactive chat interface, which can load compatible models from platforms like Hugging Face.

The app requires MediaPipe GenAI-compatible .bin format models, recommending models with 2B parameters and int4 quantization (size 1-2GB, adapted to iPhone storage and memory). Quantization technology reduces size by lowering weight precision while maintaining inference quality as much as possible. The Gemma 2B model has been verified to work properly; it is a lightweight open-source series launched by Google, suitable for mobile deployment.

The project code is well-organized, with core components including:

• ModelFileService: Responsible for model download and verification
• LLMInferenceEngine: Abstract protocol for inference engines
• MediaPipeInferenceEngine: Concrete implementation of the MediaPipe framework
• LiteRTLMInferenceEngine: Concrete implementation of the LiteRT-LM framework
• LiteRTLM/: C++ bridge layer and runner, handling Swift-C++ interoperability The layered architecture supports easy switching or extension of inference backends.

Usage Steps:

1. Clone the repository and install CocoaPods dependencies
2. Copy the configuration template file and fill in the model URL (supports direct download links from platforms like Hugging Face)
3. The app automatically handles model download, storage, and loading
4. Conduct local interactive conversations via the SwiftUI interface (no network required)

Privacy Protection: All inference is done locally; conversation content never leaves the device. Models are stored in the app's sandbox directory, protected by iOS security mechanisms, making it suitable for privacy-sensitive scenarios. Educational Value: Provides a complete reference implementation for iOS developers, helping them understand mobile AI inference, MediaPipe integration, and Swift-C++ interoperability. Community Contribution: Open-source under the MIT license; contributions via Issues or Pull Requests are welcome.

iki-nano represents the shift of mobile AI from cloud dependency to local autonomy. With the improvement of model compression technology and mobile hardware performance, running practical LLMs on phones is becoming increasingly feasible. This project provides a technical path and implementation reference for this trend, and is worthy of attention from mobile developers and AI researchers.