Local-LLM: Offline Intelligent Document Analysis Workstation for Apple Silicon

A secure offline intelligent workstation optimized for Apple Silicon (M4), supporting sensitive document analysis using large language models and RAG technology in a fully local environment, achieving 100% data sovereignty.

In today's era where data privacy is increasingly concerned, how to securely process sensitive documents locally has become an important issue. local-llm is a secure offline intelligent workstation optimized for Apple Silicon (M4 chip), allowing users to analyze sensitive task documents using large language models (LLM) in a fully isolated network environment, while achieving persistent knowledge management through Retrieval-Augmented Generation (RAG) technology.

The core value of this project lies in 100% data sovereignty—all data processing is done locally without connecting to external APIs or cloud services, making it particularly suitable for handling confidential information, military mission documents, or any scenarios requiring strict confidentiality.

The project uses Ollama as the local inference engine, supporting direct operation of large language models on Apple Silicon's GPU. Recommended models include:

• Gemma 4 26B: An efficient open-source model launched by Google, which performs excellently on Apple's unified memory architecture
• Qwen 3.6 35B: From Alibaba's Tongyi Qianwen series, supporting multilingual and long-text understanding
• Nomic Embed Text: A dedicated embedding model for document vectorization
• Moondream: A lightweight visual model supporting image understanding

These models run via Ollama's local service, bound to the address 127.0.0.1:11434, ensuring no external network exposure risks.

The highlight of the project is its task-specific RAG (Retrieval-Augmented Generation) implementation. Unlike simple single-session conversations, the system uses ChromaDB as the vector database to build a persistent long-term memory system:

1. Document Indexing: Uploaded PDF documents are automatically split, embedded, and stored in the local vector database
2. Cross-Session Query: Historical task information can be retrieved and referenced across different conversation sessions
3. Source Tracing: The system automatically tracks file names and page number information to ensure answers are verifiable and traceable

This design upgrades the system from a "single-task workstation" to a "theater-level intelligence archive", allowing accumulated knowledge to be continuously reused.

For scenarios involving sensitive document processing, the project has built-in military-grade data destruction mechanisms:

• Three-pass Overwrite Deletion: Uploaded PDF files are immediately deleted with three-pass overwriting using rm -P after processing, ensuring physical irrecoverability
• Local-only Binding: The application is hard-coded to communicate with Ollama only via 127.0.0.1, eliminating any possibility of remote access
• Archive Cleanup: Provides a one-click function to clear the entire long-term memory archive (rm -rf mission_db)

Considering the slow generation speed of large models, the project implements asynchronous streaming output. Users can see each word generated by the model in real time, which not only improves the user experience but also avoids UI timeout issues caused by long waits.

In addition to text processing, the system also supports visual analysis. By integrating visual models such as Moondream, users can upload tactical maps, drone screen captures, or satellite images for comprehensive analysis together with text task reports. This provides richer information processing capabilities for military and intelligence analysis scenarios.

The project is specifically optimized for Apple Silicon's unified memory architecture. Unlike traditional GPUs that require frequent data transfer between video memory and RAM, Apple chips' unified memory architecture allows model and document data to share the same block of high-speed memory, significantly improving performance when processing large documents (over 50 pages).