Thinking as Compression: A New Paradigm of Reasoning Models as Context Compressors

This article proposes the Thinking as Compression (TaC) new paradigm, whose core is to use the reasoning model's own thinking process to compress long contexts without a dedicated compression module. At 4x/8x compression ratios, TaC-C (enhanced version) outperforms the strongest baseline by 17.4% and 23.4% in F1 score respectively, providing an efficient and low-cost solution for long context processing.

As the context window of LLMs expands to millions of tokens, the inference cost (computational complexity, latency, memory usage) grows quadratically. Traditional compression methods rely on dedicated modules or specific training, which have problems such as high additional architecture/training costs and difficulty balancing compression ratio and information retention.

Core Viewpoint: The thinking process of a reasoning model is essentially information compression (extracting key elements, establishing logical connections).

Basic TaC Process:

1. Input long context into the reasoning model
2. Prompt to generate thinking traces (e.g., step-by-step analysis)
3. Use thinking traces as compressed context
4. Inference for downstream tasks

Without additional modules or training, it already outperforms most dedicated compression methods.

The original TaC has problems such as difficulty in budget control and the model's tendency to generate superficial thinking, so TaC-C (Constrained) is proposed:

• Design a reward function to encourage compact and information-rich thinking
• Optimize the thinking generation strategy through reinforcement learning
• Achieve a balance between controllable compression ratio and high information density.

Evaluation results on 4 long context QA benchmarks:

Compression Ratio	F1 Improvement	EM Improvement
4x	+17.4%	+15.7%
8x	+23.4%	+21.7%

Key Findings:

1. TaC achieves competitiveness without dedicated training
2. The training cost of TaC-C is much lower than end-to-end compression models
3. Thinking traces are structured, preserving key logical relationships

The higher the compression ratio, the more obvious the advantage.

Theoretical Implications: The understanding process itself is effective information encoding, revealing the deep characteristics of intelligent systems.

Practical Value:

• Plug-and-play: No need to modify the model architecture or additional training
• Cost-effectiveness: Avoid additional computation from dedicated modules
• Interpretability: Thinking traces are easier to verify than black-box vectors

Potential Expansions: Explore the impact of different reasoning strategies, task-adapted prompts, and joint optimization of compression and reasoning.

Limitations:

1. Dependent on the quality of the base reasoning model; weak models tend to generate low-quality compression
2. Some tasks (e.g., precise numerical calculation) are difficult to compress through natural language thinking
3. Generating thinking traces has additional computational overhead; need to balance benefits and costs

Conclusion: TaC re-examines the internal connections of LLM capabilities, inspiring future model designs to make more use of inherent capabilities rather than stacking dedicated modules.