VRCD: Enhancing Parallel Decoding Efficiency of Multimodal Large Language Models via Visual Redundancy Control

Original Authors and Sources

• Original Authors/Maintainers: Yulin Yuan, Hongshuo Zhao, Xiangming Meng (paper authors) / infiniteYuanyl (code repository)
• Source Platforms: GitHub + arXiv
• Original Title: Visual-Redundancy-Controlled Parallel Decoding for Diffusion-Based Multimodal Large Language Models
• Original Links: https://github.com/infiniteYuanyl/VRCD / https://arxiv.org/abs/2605.25820
• Publication/Update Date: 2026-05-25 (paper submission), 2026-05-27 (code update)

Core Insights

VRCD is a lightweight plug-and-play decoding method for diffusion-based multimodal large language models (dMLLM). It addresses the overlapping visual dependency issue in parallel decoding by controlling visual redundancy, significantly improving decoding efficiency and accuracy, and achieving substantial improvements in multiple benchmark tests.

In recent years, dMLLM has demonstrated strong generative capabilities. Its parallel decoding strategy (iteratively predicting tokens at multiple masked positions) theoretically accelerates generation, but faces core challenges: it needs to judge reliable predictions and select subsequent context positions. Existing confidence-first methods independently rank masked positions and submit the top K high-confidence positions, which have step-level limitations.

The confidence-first strategy has key limitations in multimodal scenarios: high-confidence tokens selected in the same step may rely on overlapping visual grounding, leading to visual redundancy. Repeated visual information reduces complementary information for subsequent decoding, limiting the overall performance of the model.

The research team proposed the Visual Redundancy Index (VRI) to quantify the degree of overlap, then designed the VRCD method:

1. Attention Analysis: Identify the image regions that candidate tokens focus on
2. Redundancy Calculation: Compute the visual overlap (VRI) of the token set
3. Complementary Selection: Prioritize tokens with visual complementarity

This method requires no additional training and can be directly applied to existing dMLLM, enabling plug-and-play functionality.

Main Findings

• Visual redundancy is significantly reduced
• Entropy of remaining positions decreases (more certain predictions)
• Moderate overhead

Long Decoding Experiments

• M³CoT benchmark: 18.8% relative accuracy improvement
• MMBench benchmark: 6.9% relative accuracy improvement

The results verify the effectiveness of VRCD in complex multimodal reasoning.

Practical value of VRCD:

1. No training cost (inference-time method)
2. Strong generality (applicable to all dMLLM)
3. Interpretability (understand visual attention patterns via VRI)
4. Balance efficiency and quality (improve decoding quality with reasonable overhead)

VRCD provides an effective solution to the visual redundancy problem in parallel decoding of dMLLM, promotes the development of multimodal decoding technology, and offers new ideas for future exploration of visual-language collaboration mechanisms. As multimodal AI becomes more popular, such methods will have greater application value.