Predict-then-Diffuse: An Adaptive Response Length Prediction Framework for Diffusion Language Models

Diffusion Large Language Models (D-LLMs) have significant advantages in throughput and GPU utilization due to their parallel generation mechanism, but fixed-length constraints lead to computational resource waste or reduced output quality. The Predict-then-Diffuse framework solves this dilemma through a two-stage strategy of "predicting response length first, then performing diffusion generation", significantly reducing inference FLOP overhead while maintaining output quality.

Advantages of Parallel Generation

Traditional autoregressive models generate tokens serially, limiting speed. D-LLMs can generate all tokens in one go, bringing higher throughput, better GPU utilization, and a more controllable generation process.

Dilemma of Fixed-Length Constraints

• Over-allocation: When the preset length exceeds the demand, padding tokens waste computation;
• Under-allocation: When the preset length is less than the demand, output truncation requires recalculation, introducing latency peaks. In practical applications, query response lengths vary greatly, making it difficult for fixed-length strategies to balance efficiency and completeness.

Core Idea

First, train a lightweight predictor to estimate the response length, then perform diffusion generation, fundamentally solving the fixed-length problem.

AdaRLP Predictor

Input query text, output predicted length; the design considers lightweight, context awareness, and robustness.

Safety Mechanism

Through conservative estimation (adding a safety margin) and statistical calibration (determining the optimal margin based on the validation set), it balances padding overhead and recalculation risk with minimal additional cost.

Computational Cost Reduction

Compared to default D-LLM inference, it significantly reduces FLOPs, due to eliminating over-padding, reducing recalculation, and adaptive optimization.

Comparison with Heuristic Baselines

AdaRLP outperforms heuristic methods based on linear estimation of query length by learning complex mappings.

Distribution Robustness

It maintains stable performance when the test data distribution shifts, showing good generalization ability.

Output Quality Preservation

Computational optimization does not sacrifice quality; the generated text quality is consistent with the original method.

Model Agnosticism

AdaRLP can be independently used with any D-LLM without modifying the underlying model architecture or training, making integration easy.

Training Strategy

Train using (query, response length) data extracted from existing dialogue datasets, with the goal of minimizing the gap between predicted and actual lengths, considering recalculation costs.

Online Adaptation

In production environments, AdaRLP can be fine-tuned based on inference logs to adapt to query distributions in specific scenarios.

Predict-then-Diffuse is particularly suitable for:

• Cost-sensitive large-scale services;
• Applications with strict latency requirements;
• Scenarios with large variations in query lengths (e.g., open-domain Q&A, creative writing).

Future research directions include:

• Multi-turn dialogue optimization: Extend to handle cumulative context in multi-turn interactions;
• Dynamic computational budget allocation: Adjust resources based on query priority;
• Integration with other acceleration techniques: Collaborative optimization with speculative decoding, quantization, etc.

The Predict-then-Diffuse framework solves the computational waste problem caused by fixed-length constraints in D-LLMs through the strategy of "predicting response length first, then diffusion generation". The AdaRLP predictor, combined with safety mechanisms, significantly reduces FLOPs while maintaining output quality and is robust to data distribution shifts. This method provides an important engineering optimization tool for the practical deployment of D-LLMs, helping to unlock their efficiency potential.