Panoramic View of Diffusion Large Language Model (dLLM) Resources: A Technical Evolution Map from Theory to Practice

This article is based on the GitHub repository awesome-dLLM-resources (authors Susha Pai and Xiaojun Ren, MIT License, last updated May 23, 2026), systematically organizing the technical evolution of the dLLM field. As an emerging route in generative AI, dLLM adopts a reverse diffusion process from 'noise to data', contrasting with the token-by-token generation of autoregressive models. This article covers core directions such as model architecture, training methods, inference optimization, and application practices, providing a technical reference for researchers and developers. Original link: https://github.com/piesauce/awesome-dLLM-resources

Autoregressive models (e.g., GPT, Llama) have long dominated text generation, but dLLM is emerging as a new route. Core differences:

• Generation method: dLLM uses global iterative denoising (parallel), while AR uses token-by-token sequential generation (serial).
• Discrete space adaptation: dLLM needs to define noise in the token space (e.g., random masking) to solve the adaptation problem from continuous diffusion to discrete language.
• Controllability: dLLM achieves fine control via intermediate state intervention, while AR relies on prompt engineering. Comparison table:

  Dimension	Autoregressive Model (AR)	Diffusion Model (dLLM)
  Generation Method	Token-by-token sequential generation	Global iterative denoising
  Parallelism	Low (depends on previous output)	High (parallel denoising possible)
  Generation Steps	Equal to sequence length	Fixed/variable diffusion steps
  Controllability	Via prompt engineering	Via intermediate state intervention
  Training Stability	Relatively mature	Still under exploration and optimization
  Inference Cost	Linearly related to length	Related to diffusion steps

Model Evolution:

• Dream7B (August 2025): An early representative dLLM that verified the feasibility of language tasks.
• LLaDA series: 1.5 introduced VRPO optimization for alignment; 2.0 expanded to 100B parameters; UltraLLaDA supports 128K context length. Training Frameworks: DiRL (Diffusion Reinforcement Learning, combining RL and diffusion training), dLLM project (concise implementation lowers entry barriers). Architectural Innovations:
• Continuous latent space fusion: Continuous Latent Diffusion Language Model (continuous latent space diffusion + discrete token mapping), BitLM (bit-level continuous diffusion).
• Causality and position encoding: Causal Diffusion Language Models (introducing causal structure), ELF (embedding space language flow modeling).

Decoding Strategies:

• Adaptive remasking: "Don't Settle Too Early" (reflexive remasking), "Remask, Don't Replace" (fine-grained adjustment), "When to Commit?" (dynamic block decoding).
• Inference intervention: LogicDiff (logic-guided denoising), GeoBlock (block-level optimization). Inference Efficiency Optimization:
• Dedicated frameworks: dInfer (efficient inference), Streaming-dLLM (streaming generation).
• Architectural optimizations: Fast-dLLM v2 (block diffusion reduces steps), Spiffy (lossless speculative decoding acceleration), dLLM-Cache (adaptive caching).

Reinforcement Learning Adaptation:

• Beyond Mode-Seeking RL: Trajectory balance post-training (avoids mode collapse).
• Principled RL for Diffusion LLMs: Sequence-level RL framework (modeled as MDP). Distillation and Self-Improvement: Self-Distilled Trajectory-Aware Boltzmann Modeling (self-distillation), Fine-Tuning Masked Diffusion (provably self-correcting). Quantization and Safety:
• Quantization: Quant-dLLM (extreme low-bit quantization), Quantization Meets dLLMs (systematic research), Dllmquant (dedicated quantization).
• Safety alignment: DiffGuard (safety loss and recovery), Where to Start Alignment? (alignment strategy discussion), Jailbreaking Large Language Diffusion Models (security flaw analysis).

Current Applications:

• Code generation: Global denoising is suitable for structured output.
• Math reasoning: Iterative correction aids complex tasks.
• Controllable text generation: Intermediate state intervention enables fine-grained control. Future Directions:

1. Inference efficiency optimization: More efficient decoding and hardware co-design.
2. Multimodal fusion: Joint text-image modeling.
3. Real-time interaction: Streaming dLLM architecture.
4. Domain specialization: Optimization for code, math, and other fields.

dLLM represents an important exploration direction in generative AI. Although its maturity and ecosystem lag behind AR models, it has unique advantages in parallel generation, controllability, and theoretical elegance. awesome-dLLM-resources provides a complete resource chain from theory to practice, helping researchers dive deep. It is recommended to visit the original repository for the latest resources (https://github.com/piesauce/awesome-dLLM-resources) and follow updates; dLLM is expected to achieve large-scale deployment in the future.