ARM: Autoregressive Multimodal Model Based on Discrete Representation, Unifying Image Understanding, Generation, and Editing

Core Insights: ARM achieves the unification of image understanding, generation, and editing within a single autoregressive framework through a semantic visual tokenizer and reinforcement learning optimization, and discovers cross-task synergy effects. Original Author/Team: Paper author team (arXiv:2606.11188v1) Source Platform: arXiv Original Paper Link: http://arxiv.org/abs/2606.11188v1 Code Repository: https://github.com/wdrink/ARM Publication Date: June 9, 2026

The Unification Dilemma of Multimodal AI

In the development of AI, unifying multimodal intelligence is a long-term goal—allowing a single model to understand, generate, and edit visual content simultaneously. However, the reality is model fragmentation: understanding, generation, and editing models operate independently, leading to three major issues:

• Architecture Redundancy: Each task requires a dedicated model and training process
• Capability Isolation: Difficulty in converting understanding and generation capabilities
• Complex Interaction: Tedious interface conversion is needed for cross-task collaboration The proposal of ARM aims to break this impasse and prove that the autoregressive architecture can be the cornerstone of multimodal unification.

Three-Layer Architecture Design of ARM

ARM's success is based on three technical pillars:

1. Semantic Visual Tokenizer

Converts images into discrete token sequences, optimized via multi-objective supervision:

• Semantic Discriminability (distinguishing visual concepts)
• Language Alignment (aligning with the language space)
• Faithful Reconstruction (accurately restoring images)

2. 7B Autoregressive Multimodal Model

A 7-billion parameter model trained on text and image token sequences, with advantages:

• Natural multimodal fusion (learning joint distribution via next-token prediction)
• No explicit alignment module required
• Unified training objective simplifies optimization

3. Reinforcement Learning Preference Optimization

Improves generation/editing quality with optimization objectives:

• Visual Quality (aesthetic and realistic)
• Instruction Following (executing editing instructions)
• Editing Consistency (maintaining coherence)

Experimental Evidence of Cross-Task Synergy Effects

The most unexpected finding in ARM's experiments is the cross-task synergy brought by RL optimization:

• Text-to-Image Generation: WISE overall score increased from 0.50 to 0.56
• Instruction-Guided Editing: G_O metric on GEdit-Bench-EN increased from 5.75 to 6.68 More crucially, positive synergy emerged between the two tasks—optimizing generation capability helps editing, and vice versa. This indicates that task learning under a unified representation space can mutually promote each other.

Technical Significance and Industry Impact of ARM

ARM's research has multiple implications:

• Validating the Universality of Autoregressive Paradigm: Extending the successful autoregressive approach from NLP to the visual domain
• Value of Discrete Representation: Proving that discrete representation is suitable for unified language processing and cross-modal interaction, even under the dominance of diffusion models
• New Application of RL: Demonstrating the potential of RL in multimodal preference optimization
• Open-Source Contribution: Code has been open-sourced (https://github.com/wdrink/ARM), providing a foundation for community reproduction

Limitations and Future Directions of ARM

Despite significant progress, there are still directions for exploration:

• Resolution Expansion: Current resolution is limited; need to address high-resolution processing challenges
• Video Expansion: From static to dynamic video, introducing technical difficulties in the time dimension
• More Modalities: Unifying audio, 3D, tactile, and other modalities
• Efficiency Optimization: Autoregressive generation speed is slow; need to accelerate inference

Conclusion: An Important Step Towards Multimodal AI Unification

ARM represents a key step towards the unification of multimodal AI. It proves that through discrete representation and autoregressive modeling, understanding, generation, and editing can coexist in a single framework and mutually promote each other. This not only provides a technical solution but also demonstrates the possibility that future AI systems may perceive, understand, and create the world in a unified way.