DARE: Alignment and Reinforcement Learning Execution Framework for Diffusion Large Language Models

DARE (Diffusion Large Language Models Alignment and Reinforcement Executor) is a framework developed and open-sourced on GitHub by the yjyddq team. Specifically designed for diffusion large language models (dLLMs), it provides capabilities for supervised fine-tuning (SFT), parameter-efficient fine-tuning (PEFT), and reinforcement learning (RL) training, along with comprehensive evaluation support. This framework aims to fill the gap where existing RL frameworks cannot directly adapt to dLLMs, facilitating the development of the dLLM research community. The project was released in June 2026, original link: https://github.com/yjyddq/DARE.

In recent years, diffusion language models (dLLMs) have emerged as a new architectural paradigm, generating text through iterative denoising. However, most existing RL frameworks are designed for autoregressive models and cannot be directly applied to dLLMs. To address this issue, the DARE framework was developed to specifically adapt to the training and evaluation needs of dLLMs.

DARE consists of two core components:

1. Training Framework: Built on verl, supporting SFT, PEFT (e.g., LoRA), and multiple RL algorithms;
2. Evaluation Framework: Built on OpenCompass, providing fast inference acceleration, comprehensive benchmarking, and SGLang integration. Supported model types cover two main lines:

• Masked diffusion models: LLaDA series (e.g., LLaDA-8B-Instruct), Dream models;
• Block diffusion models: SDAR series (e.g., SDAR-30B-A3B-Chat), LLaDA2.X series.

DARE implements a rich 'zoo' of RL algorithms, covering applicable scenarios for different models:

Algorithm Name	Applicable Models
d1	General
Coupled-GRPO	LLaDA/Dream
VRPO	LLaDA/Dream
MDPO	LLaDA/Dream
CJ-GRPO	LLaDA/Dream
BGPO	LLaDA2.X
SPG	SDAR Series
EBPO	SDAR/LLaDA2.X
d-TreeRPO	LLaDA/Dream
Researchers can choose the appropriate algorithm based on their tasks.

Technical Highlights:

• Sequence parallelism: Split long sequences across multiple devices to extend context length;
• SGLang integration: Deeply optimized rollout and evaluation acceleration, with team-contributed PRs fixing sampling parameters;
• Multi-node training: Provides example configurations to support large-scale distributed training. Usage Guide:
• Training environment: Create a DARE virtual environment, install requirements and flash-attn;
• Evaluation environment: Create an opencompass environment, install DARE/opencompass;
• SGLang: Install using the compatible PR branch;
• Training examples: Scripts for SFT (including PEFT), RL, and multi-node training are available.

Evaluation System: Supports HumanEval (code generation), mathematical reasoning (requires additional dependencies), and OpenCompass comprehensive benchmarks. Future plans include expanding to multi-modal evaluation. Community Collaboration: The project is a work in progress; feedback and collaboration are welcome. Built on verl and OpenCompass, the open-source license ensures scalability.

Practical Significance: Lowers the threshold for dLLM research (unified interface), promotes fair algorithm comparison, accelerates model iteration, and drives community collaboration. Future Outlook: Plans to support more models and algorithms, and is expected to become a standard training framework in the dLLM field, supporting the development of this emerging direction.