rlm-rs: A Long Document Processing Tool Based on Recursive Language Model (RLM) Pattern

rlm-rs is an RLM (Recursive Language Model) pattern CLI tool implemented in Rust, designed to address the limited context window issue of large language models (LLMs). It supports processing documents 100 times larger than the LLM context window. Its core features include intelligent chunking, hybrid semantic search, SQLite persistence, and deep integration with AI programming assistants like Claude Code, providing a systematic solution for long-context task processing.

Challenges of Long-Context Processing

Large language models are powerful but limited by fixed context windows. When dealing with ultra-long documents (such as large codebases, technical manuals, or collections of research papers), traditional approaches either truncate content (losing information) or require complex custom processing workflows.

Solutions with RLM Pattern

The RLM (Recursive Language Model) pattern provides a systematic solution. rlm-rs is implemented based on MIT CSAIL research papers; through intelligent chunking, vector indexing, and recursive sub-LLM calls, it enables AI assistants to naturally process long documents.

Core Idea of RLM Architecture

The RLM pattern abstracts long-document processing into three layers of collaboration:

• Root LLM: Main conversational large model (e.g., Claude Opus/Sonnet), responsible for task decomposition and result synthesis.
• Sub-LLM: Lightweight models (e.g., Claude Haiku), handling small chunks of content and working in parallel.
• External Environment: State persistence and data management implemented using SQLite.

Key Technical Features

1. Hybrid Semantic Search: Combines semantic search (BGE-M3 embeddings) with BM25 keyword search, balancing relevance and exact matching via the RRF algorithm.
2. Multiple Chunking Strategies: Semantic chunking (default, optimized for Markdown/prose), code-aware chunking (supports multiple languages), fixed chunking (logs/plain text), parallel chunking (ultra-large files).
3. Reference Passing Mechanism: Sub-LLMs reference chunks via content IDs, reducing context consumption and supporting chunk get <id> retrieval.
4. State Persistence: SQLite saves processing states, enabling cross-session recovery and supporting incremental embedding updates.

rlm-rs has designed a Claude Code plugin (rlm-plugin) to implement the complete RLM architecture. A typical usage workflow:

1. Initialize the database: rlm-cli init
2. Load documents: rlm-cli load document.md --name docs --chunker semantic (choose chunking strategy)
3. Hybrid search: rlm-cli search "your query" --buffer docs --top-k 10
4. Complex task processing: Use the dispatch/aggregate pattern to distribute chunks to parallel sub-agents for processing and then aggregate the results.

Codebase Analysis

When dealing with a million-line unfamiliar codebase, after loading with code-aware chunking, you can quickly locate relevant modules via natural language queries (e.g., "authentication middleware implementation"), even finding semantically relevant content without exact keywords.

Technical Document Q&A

After loading product documents, API references, etc., you can directly ask questions (e.g., "How to configure high-availability deployment on Kubernetes?"), and the system will automatically retrieve relevant sections without manual browsing.

Research Paper Review

After loading dozens of papers, for cross-document analysis (e.g., "Compare improvements in attention mechanisms"), rlm-rs locates relevant sections and the root LLM performs comprehensive comparisons.

Performance Considerations

• Efficient file processing: Memory-mapped I/O (mmap) avoids memory pressure from large files.
• Fast search: HNSW vector indexing provides approximate nearest neighbor search, balancing recall rate and latency.
• Memory efficiency: The BGE-M3 embedding model occupies 90MB of memory; shared instances avoid repeated loading, and incremental embedding updates are supported.

Solution Comparisons

• vs RAG: RLM supports multi-round retrieval-analysis-aggregation to handle complex tasks requiring global understanding, while traditional RAG only directly retrieves relevant fragments.
• vs Long-Context Models: RLM has lower costs and controllable latency, is not limited by hard context length, and can theoretically scale infinitely (for documents at the million-token level).

Installation Methods

1. Cargo installation: cargo install rlm-cli
2. Homebrew installation: brew tap zircote/tap && brew install rlm-rs
3. Source code build: git clone https://github.com/zircote/rlm-rs.git && cd rlm-rs && make install Requires Rust 1.88+ (2024 edition), and uses cargo-deny for supply chain security checks.

Summary

rlm-rs transforms academic research into a practical tool, endowing AI programming assistants with long-document processing capabilities. Its value lies not only in technical implementation but also in providing a reusable long-context processing pattern, which will become an important part of the developer toolchain in the future.