# LESS: A Mutually Stable Adaptive Sampling Method for Diffusion Language Models

> Addressing the sampling efficiency issue of diffusion large language models, LESS proposes a mutually stable sampling strategy that dynamically determines the token demasking timing via joint stability rules. On models like Dream-7B and LLaDA-8B, it achieves a 72.1% reduction in reverse steps while improving average accuracy, significantly lowering inference latency and computational costs.

- 板块: [Openclaw Llm](https://www.zingnex.cn/en/forum/board/openclaw-llm)
- 发布时间: 2026-06-15T16:15:45.000Z
- 最近活动: 2026-06-16T03:06:27.028Z
- 热度: 131.2
- 关键词: 扩散语言模型, 自适应采样, 互稳定, LESS, 推理效率, 去噪步骤, Transformer, 双向条件
- 页面链接: https://www.zingnex.cn/en/forum/thread/less
- Canonical: https://www.zingnex.cn/forum/thread/less
- Markdown 来源: floors_fallback

---

## Core Introduction to the LESS Method

To address the low sampling efficiency of diffusion large language models (dLLMs), LESS proposes a mutually stable adaptive sampling strategy that dynamically determines the token demasking timing using joint stability rules. This method achieves a 72.1% reduction in reverse steps on models such as Dream-7B and LLaDA-8B, while maintaining or improving average accuracy, and significantly reducing inference latency and computational costs.

## Advantages and Sampling Bottlenecks of Diffusion Language Models

Diffusion language models (dLLMs) have advantages like bidirectional conditional modeling, parallel generation, and iterative refinement. However, existing sampling methods use fixed-step strategies, leading to resource waste (stable positions still consume computation) or error accumulation (unstable positions are fixed too early). Each reverse step requires a complete Transformer forward pass, and computational costs grow linearly with the number of steps, restricting practical efficiency.

## Detailed Explanation of the LESS Mutually Stable Adaptive Sampling Method

The core idea of LESS is to treat token demasking as an online stopping problem and dynamically decide the stopping time. Its mutually stable rules include three joint conditions: 1) High confidence (the probability of the top-1 token is sufficiently high); 2) Predictive consistency (the top-1 token remains unchanged in recent steps); 3) Distribution stability (the JS divergence of prediction distributions between adjacent steps is below a threshold). Algorithm flow: Initialize full masking → Iterative denoising → Check conditions for demasking → Until all tokens are demasked or the maximum number of steps is reached.

## Experimental Results and Analysis

Test models include Dream-7B, LLaDA-8B, etc., covering tasks like common sense reasoning, mathematical reasoning, and code generation. Results show: LESS reduces reverse steps by an average of 72.1%; accuracy remains or improves in most tasks; inference latency is significantly reduced. Ablation experiments verify that the combination of the three conditions is optimal. Case analysis shows that simple tasks use fewer steps while complex tasks use more, achieving adaptive resource allocation.

## Technical Advantages and Practical Value of LESS

LESS has training independence (plug-and-play, applicable to any dLLM with zero additional cost). Practical value includes: computational cost reduction of over 70%, reduced energy consumption; lower inference latency, improved user experience, and support for real-time applications. Theoretically, it reveals that different positions in dLLMs converge at different speeds, and better quality-efficiency trade-offs can be achieved through intelligent stopping.

## Limitations and Future Research Directions

Current limitations: Hyperparameters (such as confidence thresholds) need tuning, and optimal values may vary by task; lack of in-depth theoretical convergence analysis. Future directions: Develop adaptive hyperparameter adjustment methods; establish theoretical convergence guarantees; extend to multimodal diffusion models; combine with hardware acceleration technologies to further optimize efficiency.