Amadeus-chat: A Local CLI Large Model Chat Tool with Hybrid RAG and Intelligent Memory Compression

Amadeus-chat is a fully locally-run command-line large model chat tool designed specifically for privacy-sensitive users. Its core features include: support for hybrid RAG retrieval (BM25 + semantic search), intelligent memory compression mechanism, and convenient model management functions. All computations are done locally without an internet connection, fundamentally ensuring data privacy and security.

In an era where data privacy is increasingly valued, many users want to use LLM capabilities without uploading data to the cloud. Amadeus-chat is designed with the concept of '100% local operation'—all computations are done on the user's device, eliminating the risk of data leakage and meeting the needs of enterprises, research institutions, and individuals handling sensitive information.

Hybrid RAG Retrieval System

Combines BM25 (keyword exact matching) and semantic search (vector embedding for deep semantic understanding) to achieve a balance between high recall and precision.

Intelligent Memory Compression

For long conversation scenarios, it compresses redundant information via algorithms, retains key points, maintains context understanding ability, and reduces memory usage and computational overhead.

Model Management

Supports downloading/switching open-source models, configuring parameters, and managing local cache and storage.

1. Privacy-First Work Environments: Can be safely used by professionals like lawyers and doctors, complying with regulations such as GDPR and HIPAA;
2. Offline Usage: Provides full AI chat functionality even in network-restricted or confidential locations;
3. Personalized Knowledge Base Q&A: Import personal/professional documents to create a dedicated knowledge assistant for accurate retrieval and Q&A.

1. Pure Local Architecture: No internet connection required throughout; data storage and inference are done locally;
2. Modular Design: Components like RAG, memory management, and model management are decoupled for easy expansion;
3. CLI Interface: Lightweight and fast-responsive, suitable for technical users to operate efficiently;
4. Open-Source Ecosystem: Built on open-source models and toolchains, lowering the barrier to use.

Amadeus-chat represents an important direction for local large model applications. With the improvement of open-source model capabilities and the growth of hardware performance, pure local AI tools have significant advantages in privacy protection and data sovereignty. For users who want to control their data, this project is worth paying attention to, and its hybrid RAG and memory compression features also provide references for local LLM development.