Alternating Visual Reasoner: Breaking the Dual Bottlenecks of Unified Multimodal Models

This paper addresses the understanding-generation gap in unified multimodal models and proposes an adaptive alternating generation framework. This framework allows the model to autonomously switch between three strategies—direct generation, self-reflection, and multi-step planning—based on instruction complexity. Combined with a hierarchical data pipeline and a two-stage training strategy, it significantly improves the fidelity and text alignment of arbitrary-to-image (X2I) generation tasks.

The core issue of unified multimodal models is the understanding-generation gap, which is particularly prominent in X2I tasks. It faces two major bottlenecks: 1. Attention entanglement: Under complex prompts, the model struggles to allocate attention correctly, leading to ignored conditions or incorrect attribute binding; 2. Difficulty in visual refinement: Existing unstructured feedback mechanisms cannot effectively guide detail correction.

The core of the framework is to allow the model to autonomously choose generation strategies based on task complexity:

• Direct generation: Suitable for simple tasks, generates a complete image in one go with high efficiency;
• Self-reflection: Suitable for medium tasks, generates an initial image first, then refines it through a structured diagnostic report;
• Multi-step planning: Suitable for complex tasks, decomposes into subtasks and then integrates the results.

Data pipeline: Evaluate sample complexity through multiple dimensions such as the number of entities and attribute constraints, and build a high-quality dataset of over 50,000 samples covering all complexity levels; Training strategy:

1. Supervised fine-tuning: Enable the model to master the basic operations of the three modes;
2. Reinforcement learning: Use stepwise reasoning rewards to encourage logically consistent reasoning, and intra-group complexity penalties to avoid overusing complex modes.

The experiments performed excellently on multiple X2I benchmark tests:

• Image quality FID score increased by 20-30%, text alignment CLIP score increased by 15-25%, and human evaluations preferred images generated by this method;
• Cross-complexity analysis: Direct generation is optimal for simple tasks, self-reflection significantly improves medium tasks, and multi-step planning performs best for complex tasks;
• Ablation experiments: Removing the adaptive mode reduces performance by 15%, removing self-reflection reduces medium task performance by 25%, and removing multi-step planning reduces complex task performance by 35%.

Practical application value:

• Efficiency optimization: Adaptive strategies adapt to resource-constrained environments;
• Interpretability: Structured reasoning processes improve transparency;
• Error diagnosis: Reflection reports help with model debugging; Limitations and future directions: Mode switching has decision overhead, complexity evaluation needs to be more aligned with human perception, generalization ability in specific domains needs to be verified, and integration with external tools can be explored.

Research significance: Unified models do not have to sacrifice efficiency for specialized models; structured reasoning is crucial for generation tasks; high-quality datasets and refined pipelines are key to success; Conclusion: The Alternating Visual Reasoner effectively alleviates the understanding-generation gap, improving generation quality and efficiency while maintaining the advantages of a unified architecture.