dLLM-Cache：通过自适应缓存加速扩散大语言模型的PyTorch实现

dLLM-Cache is an open-source PyTorch implementation project targeting inference acceleration of diffusion large language models (dLLM). Its core is an adaptive cache mechanism that intelligently reuses intermediate computation results to reduce redundant calculations, significantly improving inference efficiency while maintaining generation quality. This project addresses the high computation cost bottleneck of diffusion models, enabling wider deployment in real-world scenarios.

Diffusion models have achieved great success in image generation and are now expanding to natural language processing (NLP). Unlike traditional autoregressive models, diffusion language models generate text through iterative denoising, offering unique advantages in generation quality and diversity. However, their multi-step iterative process leads to much higher computation costs than single forward-pass autoregressive models, severely limiting practical deployment.

dLLM-Cache uses an adaptive strategy instead of caching all intermediate results:

1. Cache Trigger Conditions: Dynamically evaluates similarity between current step and cached state, only using cache when收益 exceeds overhead, avoiding memory pressure while maintaining high hit rates.
2. Cross-step State Reuse: Identifies slow-changing outputs in adjacent steps (e.g., Transformer's FFN and attention layers) and reuses previous results as initial estimates.
3. Memory Management: Supports multiple cache precision options (FP16/FP32) for trade-off between precision and memory; implements cache eviction to release old entries under high memory pressure.

dLLM-Cache is a PyTorch-native implementation with:

1. Seamless Integration: Modular design allows easy integration into existing PyTorch-based diffusion language models with minimal modifications. Core cache logic is encapsulated as reusable modules.
2. Broad Compatibility: Adapts to various diffusion language model architectures, including discrete diffusion text models and continuous diffusion latent space models, serving as an infrastructure component for inference acceleration.

According to the paper, dLLM-Cache achieves:

1. Inference Delay Reduction: 30-50% lower inference time while preserving generation quality, critical for real-time applications like interactive dialogue.
2. Cost Savings: Faster inference reduces cloud service computation costs, enabling more user requests or fewer resources for the same workload.
3. Edge Deployment: Makes diffusion models feasible on mid-end or edge devices, expanding deployment scenarios.

Application Scenarios:

• Real-time dialogue systems: Meets latency requirements for open-ended conversations.
• Content creation: Provides near-instant feedback for writing assistance.
• Multi-modal generation: Extensible to image-text joint generation. Open Source Value: Offers infrastructure for the research community; PyTorch compatibility lowers adoption barriers; detailed docs and examples help users get started quickly.

dLLM-Cache represents a key advancement in diffusion language model inference optimization. Its adaptive cache mechanism balances efficiency and quality, reducing deployment costs and enabling wider application. As diffusion model research progresses, system-level optimizations like dLLM-Cache will become increasingly important, providing a valuable reference for researchers and developers.