CadLLM: Confidence-Aware Calibration Method to Improve Inference Throughput of Diffusion Language Models Without Training

CadLLM is the open-source implementation of an ACL 2026 Findings paper, which proposes a plug-and-play controller that dynamically adjusts decoding strategies using the model's own lightweight confidence signals. This method achieves up to 2.28x throughput improvement on GSM8K, MATH, MBPP, and HumanEval benchmarks while maintaining competitive accuracy. It is training-free and compatible with existing diffusion language models (e.g., LLaDA, DREAM).

Diffusion Language Models (dLLMs) generate text through iterative denoising and theoretically have parallel advantages, but their actual inference throughput is lower than optimized autoregressive models, limiting their application in latency-sensitive scenarios. Traditional solutions require complex architecture modifications or expensive retraining, consuming significant resources and potentially affecting original performance. Thus, there is an urgent need for lightweight, training-free solutions.

The core of CadLLM (Confidence-Aware Diffusion LLM) is to intelligently adjust decoding strategies using confidence signals generated by the model itself. Its key advantage is being training-agnostic—no fine-tuning or retraining is needed. As a plug-and-play controller, it dynamically adjusts the process during inference to balance throughput and accuracy.

1. Confidence Signal Extraction: Capture the certainty of token predictions in each denoising step—high-confidence tokens terminate iteration early, while low-confidence ones retain more rounds; 2. Dynamic Decoding Strategy: Adaptively adjust based on input and real-time feedback, leveraging dLLM parallelism to maximize resource efficiency; 3. Synergy with Existing Methods: Collaborate with efficient inference baselines like Fast-dLLM to achieve cumulative performance improvements.

Evaluated on four authoritative benchmarks: GSM8K (elementary math), MATH (competition problems), MBPP (Python programming), and HumanEval (code generation). Compared to the Fast-dLLM baseline, CadLLM achieves up to 2.28x throughput improvement while maintaining competitive accuracy with the original model across all benchmarks, successfully balancing efficiency and quality.

Deployment Advantages: Plug-and-play (quick integration into existing pipelines), resource-friendly (no additional computational overhead), model-agnostic (compatible with mainstream dLLMs like LLaDA, DREAM); Industry Significance: Narrows the efficiency gap between dLLMs and autoregressive models, opens up a new direction of 'intrinsic signal dynamic optimization', and the open-source implementation promotes community iteration.

Usage Guide: 1. Environment preparation (Python 3.10+ and dependencies); 2. Model acquisition (automatically download LLaDA/DREAM from HuggingFace); 3. Integration and deployment (connect to existing inference workflows); Future Outlook: Optimize task-specific thresholds, combine with more advanced baselines, and verify performance on ultra-large-scale models.