SOL: A New Self-Optimization Paradigm for Dynamic Computational Resource Allocation in Large Language Models

Abstract: Self-Optimizing Language Models (SOL) propose a dynamic computational budget allocation mechanism. Through a lightweight policy network, it selects the optimal computational configuration for each token during decoding, achieving a Pareto-optimal improvement in inference efficiency and quality while keeping model parameters unchanged.

Key Points: SOL does not modify the weights of the base model. It introduces a policy network to dynamically adjust computational resources (attention sparsity, MLP pruning, quantization bit-width), solving the resource mismatch problem of static optimization.

Background: The Dilemma of Static Optimization

Current LLM inference optimizations mostly adopt a "one-size-fits-all" strategy (quantization, pruning, sparse attention), assuming each generation step requires the same computational resources. However, in practice, the difficulty of generating different tokens varies significantly: predicting simple words only needs a small amount of computation, while complex reasoning requires full attention and precise activation values.

Static allocation leads to resource mismatch: simple tokens are over-computed, and complex tokens lack sufficient resources. Researchers need an intelligent solution to allow models to dynamically adjust computational intensity based on the actual needs of tokens.

Core Architecture of SOL

SOL introduces a lightweight policy network (without changing the base model weights) that reads the hidden state at each decoding step and selects discrete "efficiency actions" to control three dimensions:

1. Token-level attention sparsity: Reduce attention computation for simple tokens and maintain full coverage for complex tokens;
2. MLP layer structured activation pruning: Dynamically select a subset of neurons in the feed-forward network for activation, reducing overhead while preserving expressive power;
3. Activation quantization bit-width: Use high precision (e.g., FP16) for key steps and low precision (e.g., INT8) for regular generation.

Training Method: Counterfactual Scheduling and Group Relative Policy Optimization

SOL uses teacher-forcing training: fix the token sequence, sample multiple computation scheduling schemes (counterfactual scheduling), and change the efficiency action configuration for the same token path.

Through group relative policy optimization, the policy network learns to compare the likelihood of different scheduling schemes under the same supervision signal. The reward function balances output quality and a soft penalty term for budget, enabling the policy network to master the balance of resource allocation.

Experimental Results: Significant Quality-Efficiency Improvement

SOL performs excellently across multiple model variants and budget settings:

• Under the same budget constraint, output quality is better than static allocation strategies;
• Superior to random scheduling search baselines;
• Discovers a better quality-efficiency Pareto frontier, with accuracy improvements of up to 7.3% in the MMLU benchmark (higher accuracy at the same cost or lower cost at the same accuracy).

Technical Significance and Future Outlook

SOL opens up a new optimization dimension: traditional optimization focuses on reducing the cost of a single forward pass, while SOL achieves "intelligent scheduling" (the model self-adjusts computational intensity).

It is complementary to quantization, pruning, and speculative decoding. Future inference systems can combine multiple technologies (base model quantization/pruning + SOL dynamic scheduling). Additionally, the SOL training paradigm inspires adaptive computation in fields such as multimodal fusion and long-text processing.

Conclusion

Self-Optimizing Language Models represent an important direction for LLM inference optimization. It proves that models can achieve intelligent computational resource scheduling through lightweight policy networks (with frozen parameters). This paradigm of "letting the model decide its own computation method" may become a standard component of future efficient AI systems.