CLP: Zero-Loss Adaptive Multi-Token Inference Acceleration via Co-occurrence Length Prediction

CLP proposes a lightweight multi-token inference acceleration scheme, with the core being the Backbone-as-Architect design principle and an ultra-simple linear decision layer (CLP predictor). This scheme achieves 1.14x-1.29x end-to-end acceleration on the Qwen2.5 model series (0.5B, 1.5B, 7B) while maintaining zero quality degradation, solving the problem of generation quality decline caused by head-backbone competition in traditional MTP technologies.

Large language model inference is limited by the autoregressive decoding mechanism—each token generation requires one forward pass, and latency is proportional to output length. Although Multi-Token Prediction (MTP) technology can generate multiple tokens in parallel, in traditional schemes, there is a competitive relationship between the MTP prediction head and the backbone LM head. Accepting MTP results easily leads to repeated, incoherent outputs and severe quality degradation, which becomes a core obstacle to the practical application of MTP.

The core contribution of CLP is the Backbone-as-Architect design principle: the backbone LM head always takes charge of generating the first token (authoritative), while the MTP head only predicts subsequent additional tokens, eliminating competition between heads. The CLP predictor based on this principle is a lightweight span-level decision layer, with features including: only 4.6K-7.7K parameters (far fewer than the ~1M of previous work), a single-layer linear architecture (replacing complex gating networks), and predicting the number of safely acceptable additional tokens (instead of simple binary classification). Workflow: Input current hidden representation → single-layer linear calculation → output number of additional tokens → dynamically adjust acceptance length.

Experimental results of CLP on Qwen2.5 models:

• Acceleration ratio: 1.20x-1.29x for 1.5B models, 1.14x-1.20x for 7B models;
• Quality metrics: repetition rate <0.02 (gating network method >0.5), achieving zero quality degradation;
• Comparison with previous work: CLP has better acceleration effect and no quality degradation, while the gating method has negligible acceleration and severe quality decline.

Important findings of CLP:

1. Advantages of short prediction range (k=2): Recovers 24% higher MTP head accuracy on large models; conservative strategies are more effective for large models;
2. MTP accuracy is a constraint bottleneck: Improving MTP head architecture, training objectives, and collaboration mechanisms with the backbone are key to breaking the acceleration upper limit in the future.

Technical significance of CLP:

1. Architectural paradigm shift: The Backbone-as-Architect principle redefines the relationship between MTP and the backbone model from competition to collaboration;
2. Engineering practicality: The ultra-simple design (4.6K-7.7K parameters) brings extremely low computational overhead, is easy to integrate into existing models, and does not increase deployment complexity;
3. Zero-loss acceleration: For the first time, it achieves truly zero-loss multi-token inference acceleration, breaking the perception that "acceleration must degrade quality";
4. Scalability insights: The scale-aware principle provides guidance for optimizing models of different sizes, avoiding one-size-fits-all designs.

Limitations of CLP:

1. There is still room for the acceleration magnitude to reach the theoretical upper limit;
2. The MTP accuracy bottleneck needs to be broken;
3. Strategies for longer prediction ranges need to be explored. Future directions: Improve MTP head architecture, explore complex acceptance strategies, validate on larger-scale models, and combine with other inference optimization technologies such as quantization/pruning.