The "Pseudo-Unification" Dilemma of Unified Multimodal Models: Entropy Probing Reveals the Split in Information Flow Between Vision and Language

This paper focuses on the "pseudo-unification" phenomenon of Unified Multimodal Models (UMMs). By analyzing ten representative models using an information-theoretic probing framework, it reveals the dual roots—modal asymmetric encoding and mode-split responses—and points out that true multimodal synergy requires consistency in information flow rather than just parameter sharing.

The vision of UMMs is to integrate the reasoning capabilities of large language models (LLMs) with the generative capabilities of vision models, but in reality, there exists "pseudo-unification" (failure to achieve cross-modal capability transfer and synergy). Traditional probing methods have flaws: they lack insight into internal states, or separate the encoding and generation stages, making it difficult to capture the complete picture of multimodal information flow.

The researchers propose an innovative information-theoretic probing framework to jointly analyze the input encoding and output generation processes of UMMs. They introduce entropy (to quantify uncertainty), track the entropy change trajectories of visual/language inputs, compare the entropy distribution characteristics of text generation and image synthesis, and reveal the intrinsic patterns of information flow.

1. Modal Asymmetric Encoding: The entropy change trajectories of visual and language inputs are different (language inputs show rapid entropy reduction to focus on semantics, while visual inputs have more complex entropy distributions); 2. Mode-Split Responses: Text generation has high entropy (creative, logically coherent), while image synthesis has low entropy (constrained by fidelity), limiting the transfer of reasoning capabilities.

The study found that models that successfully unify encoding and generation information (e.g., through context prediction) exhibit stronger true unification characteristics and do not rely on large-scale parameters. The key to true multimodal synergy is consistency in information flow, not parameter sharing at the architectural level.

Future research can explore better cross-modal information alignment mechanisms and evaluate/optimize the degree of unification of multimodal systems. Understanding and overcoming "pseudo-unification" is crucial for improving the application performance of UMMs in fields such as creative tools and intelligent assistants.