MR-ALIGN: Enhancing Factual Accuracy of Large Reasoning Models via Meta-Reasoning

MR-ALIGN is a meta-reasoning guided alignment framework that enhances the factual accuracy of large reasoning models by tracking state transition probabilities in reasoning trajectories, improving the performance of factual question answering without external validators. The related research has been accepted by ACL 2026 Findings.

Large Reasoning Models (LRMs) perform well on complex reasoning tasks, but still have obvious shortcomings in evidence-dependent factual question answering (factual QA). Researchers found that models may encounter correct factual information during the thinking process but fail to transfer it to the final answer, which is called "reasoning-answer disconnection". The core problem lies in the gap between the reasoning process and the final output, and the lack of a mechanism to reinforce correct reasoning patterns.

Core Ideas

MR-ALIGN proposes a new alignment paradigm that focuses on the reasoning process itself, optimizes the entire reasoning trajectory through meta-reasoning, does not rely on external validators, and uses self-supervised alignment to reduce deployment costs and maintain generalization ability.

Technical Mechanisms

1. Reasoning Trajectory Tracking: Treat thinking steps as a sequence of states, track state transition probabilities to identify correct thinking patterns;
2. Transition-Aware Implicit Reward: Construct a reward function based on transition probabilities, apply it to atomic thinking segments to reinforce beneficial patterns and weaken harmful ones;
3. Probability-Aware Scoring: Convert token-level signals into segment scores to make reward signals more stable and help learn structural reasoning strategies.

Evaluation results on multiple factual QA benchmarks show:

• Significantly improved model accuracy;
• Significantly enhanced authenticity of generated content;
• Significantly reduced frequency of misleading reasoning;
• Maintained good factual consistency in long-text scenarios. This proves that aligning the reasoning process is crucial for building factual models.

The project's GitHub repository provides complete open-source resources:

• Training/evaluation code;
• Configuration files and data scripts;
• Reproducibility documentation;
• Model checkpoints and training logs (if applicable). The open attitude helps the development of factual alignment research and provides a reference path for researchers.

The value of MR-ALIGN is reflected in:

1. Reduced deployment costs: No need for external validators, reducing latency and infrastructure costs;
2. Enhanced user trust: Reliable factual output increases the credibility of AI systems;
3. Scalability: Applicable to reasoning models of various scales with strong generality;
4. Research implications: Opens up new directions for the application of meta-reasoning in model alignment.

MR-ALIGN represents an important progress in alignment technology for large reasoning models. By focusing on the reasoning process rather than just the final answer, it provides a feasible path to realize more reliable and factual AI systems. As reasoning models become more popular, this process-oriented alignment method will become increasingly important.