CORA: A New Method to Resolve the Discrepancy Between Thinking and Answer in Multimodal RLVR

Basic Information about CORA Research

• Original Authors/Maintainers: Paper author team (arxiv:2606.14691v1)
• Source Platform: arXiv
• Original Title: CORA: Analyzing and bridging thinking-answer gap in Multimodal RLVR via Consistency-Oriented Reasoning Alignment
• Original Link: http://arxiv.org/abs/2606.14691v1
• Release Time: 2026-06-12

Core Insights

This paper proposes the CORA (Consistency-Oriented Reasoning Alignment) method, which addresses the discrepancy between the thinking process and final answer of large vision-language models (LVLMs) in multimodal reinforcement learning with verifiable rewards (RLVR) scenarios by introducing a consistency reward model and hybrid reward advantage separation (HRAS) technique, enhancing the credibility of model reasoning and its practical application effects.

Reinforcement learning with verifiable rewards (RLVR) has achieved significant results in stimulating the reasoning ability of large language models, but when extended to multimodal scenarios, existing methods have key flaws:

1. Existing multimodal RLVR research focuses on improving the visual coverage of reasoning trajectories and alleviating visual hallucinations, but ignores the semantic inconsistency between the thinking process and final answer;
2. In practical applications, LVLMs often exhibit the phenomenon of "thinking one way and saying another": the reasoning chain is complete, but the final answer contradicts the reasoning, reducing model credibility and limiting the application effect of RLVR in the multimodal field.

The research team analyzed GRPO training rollouts and found that thinking-answer discrepancy has the following characteristics:

• Persists during training: Not a temporary phenomenon in the early stage, but runs through the entire training process;
• Exists in the inference phase: After training is completed, the model still disconnects during reasoning;
• Harms credibility: Seriously affects users' trust in the model's reasoning ability.

The root cause lies in the fact that the traditional RLVR optimization goal only focuses on the correctness of the final answer, lacking effective constraints on the internal consistency of the reasoning process—models learn to generate seemingly reasonable reasoning chains, but they may not lead to correct answers.

CORA (Consistency-Oriented Reasoning Alignment) is a lightweight plug-and-play framework, with core innovations including:

1. Consistency Reward Model

Takes the reasoning process and final answer as input, outputs a consistency score, and evaluates whether the reasoning chain truly supports the final answer semantically (not just superficially related).

2. Hybrid Reward Advantage Separation (HRAS)

Decomposes strategy optimization into two stages: streaming reasoning and deep reasoning, providing fine-grained advantage allocation:

• Format reward: Ensures compliance with effective reasoning protocols;
• Accuracy reward: Maintains final task performance;
• Adaptive thinking reward: Encourages delay-aware computation allocation.

Technical Implementation Details

• No need to modify the base model architecture, can be seamlessly integrated with existing LVLMs;
• Controllable computational overhead, does not significantly increase training costs;
• Strong generality, applicable to a variety of mainstream LVLMs.

The research team verified the effectiveness of CORA on multiple multimodal reasoning benchmarks:

Performance Improvements

• Task accuracy improvement: Achieved performance gains on multiple benchmarks;
• Enhanced reasoning credibility: Reasoning trajectories are more faithful to the final answer;
• Optimized consistency metrics: Thinking-answer consistency scores improved significantly.

Cross-Model Generalization Ability

CORA performs well on LVLMs of different architectures and scales, with wide practical value.

The value of CORA in the multimodal AI field:

1. Improve interpretability: Ensure consistency between thinking and answer, make the decision-making process more transparent, suitable for high-credibility scenarios such as medical diagnosis and legal consultation;
2. Enhance human-AI collaboration: Users can easily understand and verify the basis of model decisions, establishing stronger trust;
3. Promote RLVR development: Provide a new optimization direction for multimodal RLVR, demonstrating the potential of reward mechanism design to solve alignment problems.

CORA systematically analyzes and solves the thinking-answer discrepancy problem, making important contributions to the development of multimodal RLVR:

• Technical innovation: Achieve reasoning alignment through consistency reward model and HRAS technology;
• Core insight: Reward design needs to balance "correct results" and "reasonable processes".

Future directions:

• Extend consistency constraints to more complex reasoning scenarios;
• Design more refined reward mechanisms to guide models to generate accurate and credible reasoning processes.