LLM Inference Acceleration in Advertising Scenarios: Model Compression and Parallel Validation Framework

To address the challenges of high LLM inference latency and large computational costs in real-time advertising systems, the research team proposes an efficient generative targeting framework. Through the collaborative work of three core technologies—adaptive quantization, hierarchical sparsification, and prefix tree parallel validation—it achieves significant acceleration while maintaining generation quality, and has been validated effective in real advertising scenarios. This framework provides a feasible path for the real-time deployment of LLMs in the advertising field.

Large Language Models (LLMs) show great potential in advertising scenarios, including applications such as ad creative generation and precise targeting. However, deploying LLMs to real-time advertising systems faces severe challenges: high inference latency and computational costs make direct deployment often infeasible. In the advertising field where every second counts, millisecond-level latency differences can mean huge revenue losses. How to achieve low-latency inference while maintaining generation quality has become a key problem in the advertising technology field.

The efficient generative targeting framework proposed by the research team includes three core technologies:

Adaptive Group Quantization

Dynamic group adjustment strategy, adaptive bit-width allocation (higher precision for key layers), and ad text pattern-aware optimized quantization tables, maintaining better generation quality at the same compression ratio.

Hierarchical Adaptive Sparsification

Inter-layer adaptive sparsity ratio, structured sparsity for hardware acceleration, and progressive sparsity to maintain convergence stability. Combined with quantization, it achieves dual optimization of computation and memory.

Prefix Tree Parallel Validation

Construct candidate token prefix trees, parallelly validate multiple candidate paths, and early prune invalid paths, significantly reducing generation validation overhead and supporting real-time inference.

The framework's effectiveness was validated in two real advertising scenarios:

Scenario 1: Ad Creative Generation

• Significant inference acceleration
• Ad copy attractiveness and relevance maintained at an acceptable level
• Generation diversity not significantly affected

Scenario 2: Precise Targeting

• Latency meets Real-Time Bidding (RTB) requirements
• Targeting accuracy loss is controllable
• Supports high concurrent requests

Comprehensive indicators: End-to-end latency is significantly reduced, FLOPs and memory usage are greatly decreased, generation quality passes manual and automatic evaluations, and business indicators (click-through rate, conversion rate) perform well.

The main technical contributions of the framework are:

1. End-to-end optimization: Full-link optimization from model compression to inference acceleration, not a single link.
2. Quality-efficiency balance: Significant acceleration while maintaining generation quality, with practical deployment value.
3. Scenario adaptation: Special optimization for short text generation and real-time requirements in advertising scenarios.
4. Scalability: Adaptable to models of different scales and hardware platforms.

Significance of practical deployment of the framework:

Advertising platforms: Reduce infrastructure costs, support larger-scale real-time requests, and improve response speed and user experience.

Advertisers: Obtain higher-quality creative generation, more precise audience targeting, and faster delivery feedback loops.

End users: See more relevant and attractive ads, and enjoy faster page loading and display speeds.

Current limitations and future directions:

• Model scale limitation: Experiments are aimed at medium-scale models; optimization for ultra-large-scale models needs to be explored.
• Multilingual support: Mainly adapted to Chinese and English; additional work is needed for other languages.
• Dynamic adaptation: Currently static optimization; future exploration of dynamic adjustment of compression strategies based on real-time load.
• Multimodal expansion: Expand to multimodal scenarios such as image-text and video ads.

Conclusion: This research provides important technical support for the application of LLMs in real-time advertising systems, balancing inference acceleration and quality. Efficient inference technology will be more important in the future. Paper link: http://arxiv.org/abs/2605.11582v1