ContextRL: Enhancing Long-Range Reasoning and Multimodal Capabilities of Large Models via Context-Aware Reinforcement Learning

ContextRL is a context-aware reinforcement learning method published on arXiv in June 2026. Its core is to train models to identify key evidence through contrastive context selection tasks, solving the problem of key evidence localization in large models' long-range reasoning and multimodal scenarios. It achieves a 2.2% improvement in code agent tasks and a 1.8% improvement in multimodal reasoning tasks.

Current large models have shortcomings in tasks that rely on long text details, code execution traces, or specific regions of images. The causes include: traditional supervised learning ignores the evidence extraction process; standard RL (e.g., GRPO) lacks explicit training for evidence localization; long contexts lead to attention dilution.

ContextRL designs a contrastive selection task: given a question, an answer, and two similar contexts, the model needs to determine which context supports the question-answer pair, forcing the model to understand the logical connection between the context and the answer rather than surface features.

Code agent domain: Using program execution traces to generate about 1000 pairs of contrastive samples; Multimodal domain: Constructing about 7000 pairs of image contrastive samples through generative editing and similarity search, simulating real scenarios where subtle differences determine the answer.

On 5 long-range reasoning benchmarks, ContextRL achieved an average improvement of 2.2% over standard GRPO; on 12 visual question answering benchmarks, it achieved an average improvement of 1.8%, proving the transferability of its context-aware capabilities.

Reorganizing the contrastive data into a traditional format as a baseline, the baseline showed no performance improvement, proving that ContextRL's gains come from the contrastive selection training objective rather than the additional data volume.

ContextRL provides a new idea for enhancing the context understanding of large models. It can improve fine-grained evidence localization capabilities without increasing annotation costs, and is applicable to scenarios such as code review, document question answering, and medical image analysis. In the future, it can be extended to video/audio modalities, or combined with process reward models to improve reasoning transparency.