STARFlow2: Achieving Truly Unified Multimodal Generation with Autoregressive Normalizing Flows

STARFlow2 addresses the current architectural dilemmas in multimodal generation and proposes a unified solution based on autoregressive normalizing flows. The core innovation lies in leveraging the isomorphism between autoregressive normalizing flows and Transformers (sharing causal masks and KV caches), fusing VLM flows and TarFlow flows via the Pretzel vertically interleaved architecture, constructing a unified FAE latent space to achieve unified generation and understanding of text and images. It also has cache-friendly interleaved generation capabilities, solving the structural mismatch problem of concatenated architectures.

Deep generative models have driven the demand for unified multimodal systems, but existing mainstream solutions adopt a 'concatenated' architecture (autoregressive language model + diffusion image generator), which has structural mismatches: language generation is a causal sequence decision, while image diffusion is iterative global denoising, requiring maintenance of two sets of computing mechanisms. There are computational overhead and information loss during modal switching, and the separation between text and image latent spaces limits cross-modal reasoning capabilities.

The core insight of STARFlow2 is the deep isomorphism between autoregressive normalizing flows and Transformers—both share causal masks, KV caches, and left-to-right generation structures. Normalizing flows map distributions via reversible transformations; when organized autoregressively, they can be applied to discrete text tokens and continuous image latent representations, providing a theoretical foundation for unified generation.

STARFlow2 is based on the Pretzel architecture, vertically interleaving pre-trained VLM flows and TarFlow flows, fusing them via residual connections, sharing causal masks and KV caches without the need for modal switching. It adopts a deep-shallow flow division of labor (deep flows capture semantics, shallow flows refine details) and constructs a unified FAE latent space, enabling direct comparison and combination of text and image representations, and conditional generation without additional alignment layers.

STARFlow2's cache-friendly design allows text and visual outputs to directly enter the shared KV cache, avoiding additional encoding overhead during modal switching. In interactive applications (such as conversational image editing), it can switch modalities instantly without delay accumulation, improving the efficiency of long-sequence generation.

Experiments show that STARFlow2 performs strongly in both image generation and multimodal understanding tasks: it can generate high-quality, semantically consistent images with fine-grained control capabilities; it inherits VLM's understanding capabilities, enabling accurate image content Q&A and visual reasoning, and the understanding and generation share mechanisms, supporting collaborative tasks such as iterative image optimization.

STARFlow2 proves that autoregressive normalizing flows can serve as a foundation for unified multimodal modeling, avoiding the compromises of concatenated architectures. Limitations include slow high-resolution generation and complex training; future directions can explore parallel decoding to accelerate generation, flow transformation architecture search, and expansion to modalities such as audio/video/3D, promoting the development of multimodal AI toward true unification.