WaveTune: Wave-Aware Bilinear Modeling Redefines the Efficiency Boundary of GPU Kernel Auto-Tuning

The WaveTune framework addresses the performance-efficiency trade-off in GPU kernel tuning through a wave-aware bilinear model and a lightweight dual-table retrieval mechanism. Its core lies in a modeling approach that integrates GPU hardware knowledge, delivering up to 1.83x kernel speedup and 1.33x end-to-end TTFT reduction across five GPU architectures. The decision overhead is reduced by five orders of magnitude compared to exhaustive search, providing a new path for improving LLM inference efficiency.

Modern LLM inference relies heavily on GEMM kernels, whose performance is sensitive to runtime parameters (e.g., tile size, number of pipeline stages, shared memory allocation). The parameter space is complex and non-convex. Traditional tuning methods have shortcomings: search-based auto-tuning is accurate but time-consuming; heuristic rules are fast but have poor adaptability; learning-based cost models need optimization in terms of overhead and generalization ability—all struggle to make near-optimal decisions quickly at runtime.

WaveTune builds a three-layer architecture based on insights into GPU wave structures: 1. Unified Mapping and Configuration Space Decomposition: Standardize heterogeneous inputs and decompose the high-dimensional configuration space into subproblems; 2. Wave-Aware Bilinear Model: Integrate GPU physical knowledge to explicitly model wave-level execution features (launch overhead, synchronization delay, etc.), using a bilinear structure to balance expressive power and efficiency; 3. Sparse Sampling and Dual-Table Retrieval: Sparse sampling of potential configuration subspaces based on wave structures, and use dual tables (exact solution + approximate solution) for hierarchical retrieval to compress decision time to the microsecond level.

Evaluations across three representative kernels and five GPU architectures (from consumer to data center grade) show: up to 1.83x kernel-level speedup; up to 1.33x end-to-end LLM inference TTFT reduction; decision overhead reduced by five orders of magnitude compared to exhaustive search; and the results span different architectures, demonstrating good generalization ability.

WaveTune breaks the traditional performance-efficiency trade-off in GPU kernel tuning, achieving both fast and high-quality tuning results. This paradigm shift is of great significance for scenarios such as edge devices, online services, and large-scale deployments, providing a new direction for AI system optimization.

The success of WaveTune highlights the importance of domain knowledge: hybrid methods integrating physical knowledge can achieve excellent results with limited resources. Its design decisions (wave awareness, bilinear structure, sparse sampling) stem from an understanding of hardware mechanisms and the essence of the problem, suggesting that engineers should first explore domain-specific constraints and structures before considering data and computing power during optimization.

WaveTune's methodology can be extended to broader scenarios: operator fusion with multi-kernel collaboration, task partitioning for heterogeneous computing, runtime adaptation for dynamic workloads, etc. As LLM scales grow, the 'knowledge + data' hybrid optimization paradigm may become the core methodology for next-generation AI system software.