InterleaveThinker: Enabling Interleaved Text-Image Generation via Multi-Agent Reinforcement Learning

This paper proposes the InterleaveThinker multi-agent pipeline, which enables existing image generators to perform interleaved text-image generation through collaboration between a planner agent and a critic agent. Using GRPO reinforcement learning for step-level instruction correction, it achieves performance comparable to Nano Banana and GPT-5 on interleaved generation benchmarks while significantly improving performance on reasoning benchmarks. This research is from arXiv (published in June 2026), with the original title "InterleaveThinker: Reinforcing Agentic Interleaved Generation".

Advances and Limitations of Image Generation

In recent years, image generators have performed excellently in single-image generation/editing, but are unable to achieve interleaved generation (alternating text-image sequence generation) due to architectural constraints.

Importance of Interleaved Generation

Applicable to scenarios such as visual storytelling (comics/storyboards), operation guidance (text-image tutorials), embodied intelligence (robot operations), etc.

Shortcomings of Existing Methods

The latest open-source unified multimodal models (UMMs) have limited performance in interleaved generation, and specialized methods are urgently needed to address this.

Dual-Agent Collaboration Mechanism

1. Planner Agent: Receives user instructions, decomposes tasks into ordered steps, generates text prompts, and maintains global consistency;
2. Critic Agent: Evaluates the matching degree between generated results and instructions, identifies deviant samples, and generates refined instructions.

Pipeline Flow

User instruction → Planner → Generate instruction → Image generator → Result → Critic evaluation → Instruction refinement (when re-generation is needed).

Training Strategy

• Constructed three datasets: Interleave-Planner-SFT-80k (Planner SFT), Interleave-Critic-SFT-112k (Critic SFT), Interleave-Critic-RL-13k (GRPO training);
• Adopted GRPO reinforcement learning, using accuracy rewards (single-step quality) and step-level rewards (inter-step coherence) to solve long trajectory optimization problems.

Benchmark Performance

Achieves performance comparable to Nano Banana (SOTA interleaved generation model) and GPT-5 (multimodal model) on interleaved generation benchmarks.

Cross-Model Generalization

Can empower various image generators without modifying the underlying architecture, featuring plug-and-play capabilities.

Improvement in Reasoning Ability

Unexpected finding: InterleaveThinker significantly enhances the reasoning ability of base models. For example, FLUX.2-klein shows a substantial improvement on WISE (visual reasoning) and RISE (reasoning and instruction following) benchmarks, possibly due to structured thinking training, feedback loop learning, and multi-step reasoning practice.

Technical Contributions

1. The first general interleaved generation framework, breaking specific architectural constraints;
2. Innovative dual-agent collaboration model, which can be extended to other complex generation tasks;
3. Efficient RL strategy to solve long trajectory optimization problems;
4. Discovered the cross-capability transfer phenomenon from generation ability to reasoning ability.

Overall Conclusion

InterleaveThinker breaks through the architectural limitations of existing image generators through agent collaboration. It not only achieves SOTA in interleaved generation tasks but also improves reasoning ability, demonstrating the potential of agent collaboration to expand the capabilities of AI systems.

Application Scenarios

Educational content generation (text-image textbooks), technical document creation (step-by-step guides), creative storytelling (comics/picture books), robot task planning (visual-language instructions).

Limitations

Agent overhead increases inference latency, relies on large amounts of training data, and performance degrades for long sequences (>50 steps).

Future Directions

Explore single-agent integration, support real-time interaction, extend to video/audio modalities, and optimize training and inference efficiency.