Domino: A New Speculative Decoding Framework with Decoupled Causal Modeling and Autoregressive Drafting

Domino is an innovative framework for speculative decoding. By decoupling the parallel drafting backbone and the lightweight causal refinement module (Domino Head), it breaks through the quality-cost trade-off dilemma in traditional speculative decoding. On the Qwen3 model, Domino achieves a maximum throughput speedup of 5.8x, significantly improving the inference efficiency of large models.

Speculative decoding relies on a draft generator to produce candidate tokens and a validator to verify their correctness. Traditional solutions face a dilemma:

• Autoregressive draft generator: Accurately models causal dependencies, leading to high candidate acceptance rates, but sequential generation cannot be parallelized, resulting in slow speed;
• Parallel draft generator: Generates token blocks at once, which is fast, but ignores causal dependencies, leading to low candidate quality and acceptance rates. Existing solutions struggle to balance both.

Decoupled Architecture

1. Parallel Drafting Backbone: Fully parallel generation of the initial probability distribution for token blocks, maximizing hardware computing power utilization;
2. Lightweight Domino Head: Takes the backbone output as input, corrects the distribution by combining prefix context, introduces causal dependencies, and has low computational overhead.

Training Strategy

Adopts a "Foundation Anchoring" curriculum:

1. Independently train the parallel backbone to equip it with basic prediction capabilities;
2. Progressively co-train the Domino Head, shifting the optimization objective from matching the teacher distribution to causal consistency, alleviating exposure bias.

Evaluation results of Domino on the Qwen3 model family:

• End-to-end speedup: 5.49x on the Transformers inference backend, up to 5.8x under the SGLang service framework;
• Comparison with existing solutions: Reduces drafting latency while maintaining a similar acceptance rate, or increases acceptance rate to reduce validation waste;
• Scalability: The advantage becomes more obvious as block size increases, with high parallelization benefits and no quality degradation.

Domino is suitable for the following scenarios:

• High-throughput online services: Maximize GPU utilization and reduce per-request costs;
• Long text generation: Reduce inference steps and improve document/code generation efficiency;
• Resource-constrained environments: The lightweight Domino Head supports edge/mobile deployment;
• Combination with speculative execution: Further optimize end-to-end latency.

Domain Insights

• Decoupling is better than compromise: Split quality and speed objectives into different components for optimization;
• Value of lightweight correction: Post-processing correction is more effective than end-to-end models in specific scenarios;
• Key role of training strategy: The Foundation Anchoring curriculum solves training instability issues.

Future Directions

• Multi-scale Domino Head: Flexibly select based on computing budget;
• Adaptive block size: Dynamically adjust speculative block size;
• Cross-model migration: A universal Domino Head to serve multiple target models.