InftyThink: Breaking the Length Limit of Long-Context Reasoning for Large Language Models

The InftyThink framework developed by the REAL Lab at Zhejiang University successfully breaks the length limit of long-context reasoning for traditional large language models (LLMs) through an innovative segmented reasoning mechanism, enabling efficient understanding and reasoning of ultra-long texts. This work has been accepted by ICLR 2026, addressing core issues faced by current LLMs such as scattered attention and the "Lost in the Middle" phenomenon. It constructs a hierarchical reasoning architecture that mimics human reading patterns, balancing computational efficiency and deep understanding capabilities.

Current LLMs (e.g., GPT-4, Claude) support context lengths of hundreds of thousands of tokens, but their reasoning quality decreases significantly as text length increases. Key challenges include: the quadratic complexity of attention mechanisms leading to high computational costs and scattered attention; the "Lost in the Middle" phenomenon (weaker recall of information in the middle of the text compared to the beginning and end); and a lack of global structure awareness, making it difficult to integrate full-text information for complex reasoning.

InftyThink adopts a hierarchical reasoning architecture: 1. Intelligent Semantic Segmentation: Splitting based on semantics rather than fixed length to ensure each segment has a complete theme; 2. Local Reasoning: Independently extracting key information and intermediate conclusions from each segment to generate structured outputs; 3. Global Aggregation: Establishing connections between segments via a lightweight graph attention network, integrating results to form a global understanding.

In long-context benchmark tests such as LongBench and ∞Bench, InftyThink shows significant performance: computational overhead is reduced by over 60% compared to baselines; the accuracy of ultra-long document question answering is improved by 15-25 percentage points; it can recursively process texts exceeding the model's native context length, theoretically supporting infinite-length inputs.

Application scenarios include law (case file understanding), finance (market report/financial statement analysis), scientific research (literature organization), etc. Limitations: The choice of segmentation strategy has a significant impact on performance; improper segmentation may lead to semantic breaks; the global aggregation module still faces computational pressure when the number of segments is extremely large.

InftyThink represents an important breakthrough in the field of long-context reasoning, proposing a new paradigm of hierarchical reasoning that mimics human cognition. In the future, more intelligent adaptive segmentation strategies and more efficient global aggregation mechanisms can be explored, and we look forward to the technology being implemented to unlock the potential of LLMs in ultra-long text understanding.