In-depth Analysis of vLLM-XPU: Intel XPU Inference Performance Profiling and Visualization Tool

vllm-xpu-breakdown is a vLLM inference performance profiling tool specifically designed for Intel XPU. It can track and visualize the scheduling of operators across different backends (vllm-xpu-kernels, torch-xpu-ops, triton, cpu), helping developers optimize large model inference performance.

With the explosive growth in demand for Large Language Model (LLM) inference, Intel XPU, as an important accelerator alternative to GPUs, is gaining increasing attention. However, compared to the mature ecosystem of NVIDIA GPUs, the inference optimization toolchain on XPU is still relatively weak. When developers face performance bottlenecks, they often struggle to pinpoint whether the problem lies in custom kernels, PyTorch native operators, or Triton-compiled code.

The vllm-xpu-breakdown project was created to address this pain point. It provides a complete performance profiling and visualization solution, allowing developers to clearly see which backend each operator runs on, enabling targeted optimization.

The core innovation of this tool lies in establishing a refined backend classification system, dividing operator execution into five distinct categories:

This is a collection of custom kernels specifically written by the vLLM team for XPU, covering key operators such as RMSNorm, activation functions, attention mechanisms, MoE (Mixture of Experts), quantization operations, and cache management. Currently, the registry contains 68 operators distributed across 4 core modules. These kernels represent the most efficient implementations on XPU and are the primary targets for performance optimization.

Includes basic operations like linear transformations, matrix multiplication, and embedding lookups, accelerated on XPU via oneDNN and oneMKL. These operators represent framework-level general optimizations; while not as extreme as custom kernels, they offer good compatibility and stability.

Covers attention backends, sampling algorithms, and code generated by torch.compile. As an emerging GPU/XPU programming model, Triton can generate performance close to handwritten kernels while maintaining Python-level development efficiency, making it an important direction for inference optimization in recent years.

When certain operators do not yet have XPU support or encounter specific limitations, they may fall back to CPU execution. This part is usually a key target for optimization, as data transfer between CPU and XPU incurs significant overhead.

Includes overhead from tensor reshaping, memory operations, and the performance profiler itself. Although the single-invocation overhead is small, it is still worth attention in high-frequency call scenarios.