Sift: A Local Hybrid Search Framework for Agent Workflows with No Infrastructure Overhead

Sift is a local hybrid search tool written in Rust that combines BM25 lexical search, vector semantic search, and local LLM re-ranking. It provides high-performance document retrieval capabilities with zero configuration and no daemon processes via a single binary file. Its core positioning is as a local hybrid search for agent workflows, aiming to solve the problems of high infrastructure costs and complex configurations in traditional RAG solutions, making it suitable for scenarios with sensitive data privacy, offline work, or the need to reduce operational complexity.

Traditional RAG solutions require deploying vector databases and configuring complex service architectures, leading to significant infrastructure costs. Sift adopts a "local-first" design philosophy, where all operations are executed locally without the need for external services or daemon processes. Its core innovation lies in a three-layer architecture: BM25 for precise keyword matching, vector search for understanding conceptual relevance, and local LLM re-ranking for fine-tuning—balancing the reliability of traditional search with the semantic understanding capabilities of modern AI.

Sift uses a layered pipeline architecture (query expansion, retrieval, fusion, re-ranking). Resource pipeline optimization: heuristic caching + CAS to avoid duplicate processing; supports multi-format document extraction (plain text, HTML, PDF, Office), embedding models to generate vectors, and caches results in the user directory. Hybrid search strategy: the default "hybrid" mode combines BM25 and vector search results via RRF fusion, followed by local LLM re-ranking. Agentic support: build context-aware multi-turn dialogue applications via the search_turn and search_controller APIs.

Command Line Interface: Core commands include sift search (one-time search with strategy specification), sift eval (performance comparison), sift dataset (dataset management), sift optimize (prompt template optimization), and supports --json output for structured results.

Embedded Library Integration: The Rust library provides a concise API, such as Sift::builder() to construct instances, the search method to perform retrieval, supports configuration of strategies, retrievers, fusion methods, etc., and also supports complex scenarios like context assembly and protocol output.

Sift accelerates vector operations via SIMD instructions, eliminates duplicate computations with content-addressable storage, and improves large file reading efficiency with memory-mapped I/O, capable of handling millions of documents. Engineering practices: continuous integration covers multi-platform builds, a benchmark test suite compares retrieval quality and latency, and data-driven optimization ensures performance improvements across versions.

Sift is suitable for the following scenarios: 1. Local knowledge base Q&A (sensitive documents do not need to be uploaded to the cloud); 2. Development workflow integration (IDE plugins or code search); 3. Edge device deployment (single binary with no dependencies); 4. Agent system construction (multi-turn dialogue APIs provide retrieval infrastructure). Its value lies in lowering the threshold for building high-quality RAG applications, balancing functionality and deployment convenience.

Sift integrates hybrid search, semantic re-ranking, and agent workflow support into a single binary file, lowering the threshold for building RAG applications. It is suitable for developers who need rapid prototype verification, data privacy protection, or simplified operations. As local LLM capabilities improve, such "zero-infrastructure" tools will play a more important role in the AI ecosystem.