Silenced Visual Latents: A New Paradigm for Implicit Reasoning Optimization in Multimodal Large Models

This article uncovers the systematic suppression of visual latents in multimodal large language models and proposes a two-stage in-inference optimization method that does not require parameter updates. It can unleash the suppressed visual reasoning capabilities and open up a new path for enhancing the performance of multimodal models.

Continuous latent space reasoning provides a more compact alternative to text-based chain-of-thought for multimodal models. It can integrate high-dimensional visual evidence without explicit reasoning tokens, and theoretically combines efficiency and expressive power. However, there are long-overlooked optimization pathologies in actual training.

Phenomenon Description

The research team identified an optimization pathology: visual latents are semantically rich during training, but their contribution to final answer prediction is systematically suppressed.

Root Cause

In the shared parameter space, the autoregressive objective tends to rely on direct visual input shortcuts, causing latent tokens to be pushed into a transient state rather than meaningful reasoning content. This phenomenon is named "Silenced Visual Latents".

First Stage: Query-Guided Contrastive Alignment

Preheat visual latents through query-guided contrastive latent-visual alignment to prevent collapse and improve semantic quality, ensuring the capture of rich cross-modal information.

Second Stage: Confidence Progressive Reward

Optimize latent reasoning via confidence progressive rewards, incentivizing the prediction token distribution to gradually concentrate and guiding predictions through the latent reasoning path instead of bypassing it.

The research team conducted experiments on 8 benchmark tests and 4 model backbones:

• No parameter updates: All optimizations are completed during inference without modifying model parameters
• Significant performance improvement: Effectively unleash the suppressed visual latent reasoning capabilities
• Cross-model generalization: The method shows good transferability across multiple architectures

Technical Significance

Reveal the inherent conflict between the autoregressive objective and the visual reasoning objective in multimodal model training, shift optimization from the training phase to the inference phase, and open up a new path to enhance model capabilities without retraining.

Practical Application Implications

• Improve reasoning quality without modifying pre-trained models
• Provide a more efficient reasoning mechanism for vision-language tasks
• Offer a new perspective for understanding the internal working mechanisms of multimodal models

The revelation of the "Silenced Visual Latents" phenomenon and the two-stage optimization method provide a new technical path for unleashing the capabilities of multimodal large language models, and are expected to promote further development in the field of visual reasoning.