Reverse Information Flow: Generation-Understanding Synergy Mechanism in Large Multimodal Models

This paper proposes the Generation-to-Understanding (G2U) collaborative framework, which treats visual generation as an explicit intermediate reasoning step and enhances perceptual understanding through self-generated visual thinking feedback. Evaluations on 12 benchmark tests show that this reverse information flow can consistently improve multimodal understanding capabilities. It also discusses the limitations of models independently deciding what to generate, providing a new direction for unified cognition in multimodal AI.

The long-term goal of multimodal AI is to build a unified model where visual understanding and generation mutually enhance each other. The current mainstream paradigm is a unidirectional flow of 'understanding → generation', where generation is regarded as a downstream application of understanding, ignoring the deep understanding value of the generation process. In human cognition, imagination and generation can deepen the understanding of things.

The core of the G2U framework is to treat visual generation as an explicit intermediate reasoning step. The process is: 1. Controlled generation behavior; 2. Self-generated visual thinking; 3. Feedback-enhanced perception (no retraining required). The three generation strategies include detail enhancement (magnifying details), context expansion (supplementing incomplete information), and structural visualization (structuring abstract concepts).

In 12 multimodal benchmark tests (visual question answering, image classification, visual reasoning, etc.), the G2U framework significantly improves understanding capabilities: VQA accuracy increases by an average of 3-5%, fine-grained recognition shows more significant improvement, and visual reasoning paths are clearer. Key findings: Generation fidelity is positively correlated with perceptual gain; descriptive, comparative, and structured prompts are more efficient.

Through meta-generation tasks, it is found that models can generate reasonable visual edits but lack stable task alignment (the generated content does not fully match the current task requirements). This indicates that models lack metacognitive ability and cannot reliably judge 'what information is most helpful for the current task'.

The G2U framework echoes constructivist theory (actively constructing an understanding of the world). Implications for model architecture: support for bidirectional information flow, explicit reasoning steps, and metacognitive modules. Open issues include: determining the optimal generation strategy for specific tasks, balancing effectiveness and efficiency, improving interpretability, and cross-modal expansion.