Doc2Atom: A Compositional Parametric Memory Framework Revolutionizing Long-Document Reasoning

Original Authors and Source

• Original Authors/Maintainers: Paper author team (standard arXiv authorship)
• Source Platform: arXiv
• Original Title: Doc-to-Atom: Learning to Compile and Compose Memory Atoms
• Original Link: http://arxiv.org/abs/2606.12400v1
• Publication Time: 2026-06-10

Core Insights

This paper proposes the Doc2Atom compositional parametric memory framework, which decomposes documents into semantically typed knowledge atoms and compiles them into independent micro-LoRA adapters to achieve query-specific dynamic composition. This framework outperforms the Doc-to-LoRA baseline on six QA benchmarks while significantly reducing the memory cost of document internalization, revolutionizing the way long-document reasoning is done.

Computational Dilemma of Long-Document Processing

Large Language Models (LLMs) face a quadratic complexity bottleneck in their attention mechanism when processing long documents; as input sequences grow, computational and memory costs increase sharply.

Rise of Context Distillation

To address this issue, the "context distillation" method compresses document information into model parameters, avoiding long-sequence processing during inference. The core is to pre-internalize documents into parameters, and only load compressed representations during inference.

Limitations of Doc-to-LoRA

Doc-to-LoRA generates a document-specific LoRA adapter via a single forward pass, but has three major issues:

1. Irrelevant query interference: A single adapter mixes multi-topic information, leading to scattered answers or hallucinations;
2. Limited compositional recall: Difficult to combine multiple parts of information to handle complex queries;
3. Poor scalability for long documents: Information volume growth exceeds the capacity of a single adapter.

Core Idea: Knowledge Atomization

Doc2Atom decomposes documents into knowledge atoms—semantically typed sub-units, each containing coherent concepts and semantic labels, which can be independently compiled into parameters and dynamically combined.

System Architecture

1. Document Decomposer: Segments documents into atoms based on semantics, annotates types, and optimizes boundaries;
2. Atom Compiler: Compiles each atom into a lightweight micro-LoRA adapter, associated with a source retrieval key;
3. Query Router: Analyzes queries, selects relevant atoms, and assembles a composite adapter to inject into the base model.

End-to-End Training

Trained via multi-objective distillation:

• Atom quality: Ensure atoms accurately encode segment information;
• Routing accuracy: Train the router to select relevant atoms;
• Compositional ability: Handle multi-atom composition for complex queries;
• Efficiency optimization: Minimize computational costs. Training data is automatically generated, including atom-question-answer pairs, complex queries, and negative samples.

Benchmark Datasets

Validated on six QA benchmarks: Natural Questions, HotpotQA, MS MARCO, NarrativeQA, QASPER, DocRED.

Key Results

1. Performance improvement: Outperforms Doc-to-LoRA on all benchmarks, with an average increase of over 10% (e.g., HotpotQA +12.7%, NarrativeQA +15.2%);
2. Memory efficiency: Parameters for storing the same information are reduced by 40-60%, and only a few micro-LoRA adapters are loaded during inference, with more obvious advantages for long documents.

Ablation Studies

• Atomization itself improves performance, proving decomposition reduces interference;
• Dynamic routing further enhances performance;
• Semantic type annotation contributes significantly (performance drops by 15% without annotation);
• Micro-LoRA is more efficient than standard LoRA.

Four Key Advantages

1. Information isolation: Atoms physically isolate irrelevant information, eliminating interference;
2. Compositional flexibility: Dynamic routing combines atoms on demand to handle simple/complex queries;
3. Parameter efficiency: Micro-LoRA requires only hundreds of parameters, with total parameters far lower than a single adapter;
4. Interpretability: Selected atoms can be viewed to understand the basis for the model's answers.

Core Application Scenarios

1. Enterprise knowledge base QA: Dynamically combine atoms for products, technologies, customer cases, etc.;
2. Legal document analysis: Adapt to structured atoms like contract clauses and precedents;
3. Academic paper assistant: Combine atoms for abstracts, methods, experiments, etc., on demand;
4. Multi-document reasoning: Unified indexing of cross-document atoms, supporting cross-document information combination.

Current Limitations

1. Decomposition quality: Automatic decomposition may be imprecise;
2. Type system: Predefined/learned type systems have limited coverage;
3. Routing errors: The router may select wrong atoms;
4. Training cost: End-to-end training requires large resources.

Future Directions

1. Adaptive decomposition: Learn optimal decomposition strategies;
2. Hierarchical atoms: Support hierarchical structures from chapters → paragraphs → sentences;
3. Cross-document association: Identify semantic associations between atoms from different documents;
4. Incremental updates: Support partial updates of documents;
5. Multimodal extension: Cover multimodal documents like images and tables.

Doc2Atom represents an important advancement in the field of context distillation, solving the fundamental limitations of monolithic adapters through atomization and dynamic composition. Its "LEGO brick"-style information organization approach opens up new possibilities for long-document reasoning. As LLMs expand their applications in knowledge-intensive tasks, Doc2Atom will become a key infrastructure for efficiently utilizing massive document information.