Hallucination Phenomena in Multimodal Reasoning Models: Is RL Post-Training Really Learning Visual Information?

Recent research reveals a surprising finding: even without real visual information, reinforcement learning (RL) post-training can still significantly improve the reasoning ability of multimodal large models (MLLMs). Through the "hallucination induction" mechanism, this study found that pure hallucination training even outperforms standard training in some tasks, challenging our traditional understanding of MLLM training mechanisms—performance improvements from RL post-training may stem more from reasoning strategy optimization than visual information understanding.

From Text to Multimodal Transition

The success of models like OpenAI o1 and DeepSeek-R1 in mathematical reasoning has promoted RL post-training to expand into the multimodal domain. However, visual reasoning involves more complex modal interactions, and there is doubt whether the improvement comes from visual understanding or text reasoning strategies.

Hallucination: An Overlooked Diagnostic Tool

Model hallucinations are usually regarded as flaws, but this study puts forward a counterintuitive view: hallucinations can be used as a tool to understand the model's learning mechanism. By inducing hallucinations, we can strip away the influence of visual information and observe the real effect of RL training.

Hallucination Induction Strategies

• Image-level damage: blurring, occluding key areas, replacing with irrelevant images
• Text-level interference: inserting misleading information or removing visual-related descriptions
• Cross-modal mismatch: pairing questions with irrelevant images

Experimental Conditions

1. Standard training: normal image-text pairs
2. Pure hallucination training: using damaged data throughout
3. Mixed training: normal + hallucination data By comparing the performance of the three, the real contribution of visual information is quantified.

Experimental Results

• MathVista mathematical chart understanding: accuracy increased by 12-15%
• MMMU multidisciplinary Q&A: improved by 8-10%
• ScienceQA scientific reasoning: pure hallucination training outperformed standard training

In-depth Analysis

RL training improves:

1. Reasoning strategy optimization (decomposing problems, verifying steps)
2. Knowledge retrieval enhancement (extracting information from internal knowledge bases)
3. Answer format learning (identifying format patterns) These abilities do not rely on real visual information.

Challenging Existing Paradigms

• Evaluation flaws: Traditional benchmarks cannot distinguish between visual understanding and text guessing
• Nature of modal fusion: Current MLLMs may be shallow concatenation rather than deep fusion
• RL limitations: Better at optimizing reasoning than perceptual abilities

Future Directions

1. Modal-aware RL design: clearly distinguish between visual and reasoning learning
2. Strict evaluation benchmarks: detect hallucination dependence
3. Cross-modal causal reasoning: identify causal relationships in visuals

Evaluation Advice

• Hallucination stress test: compare performance under normal and damaged images

Training Data

• Focus on answer distribution and format patterns, not just image content

Multimodal Value

• Think about whether the task really needs visual information; a pure text model with reasoning strategies may be sufficient

This study forces us to rethink the definition of "understanding": when a model answers questions correctly without valid visual input, is it super reasoning or not really "seeing"? In the future, we need to simultaneously promote reasoning ability improvement and visual understanding training, clearly distinguish between "visual understanding" and "reasoning guessing", and guide multimodal AI towards maturity. Hallucination is no longer a flaw but a signpost leading to true understanding.