EnergyLens: An Energy Prediction and Optimization Framework for Multi-GPU Large Model Inference

EnergyLens is an end-to-end energy-aware optimization framework designed for multi-GPU large language model inference. It achieves energy prediction in the configuration space and Pareto optimal selection via the einsum interface and multi-GPU communication energy model, with a prediction error of 9.25%-13.19% on Llama3 and Qwen3-MoE models. It aims to address the pain points of existing energy optimization tools.

With the expansion of large language model scales, the energy consumption issue in the inference phase has become a focus. The daily energy consumption of a 100-billion-parameter model in a production environment is equivalent to the electricity usage of hundreds of households. Existing solutions have limitations: production-level code analysis requires intrusive code modifications and expensive hardware analysis, making it difficult to explore before deployment; simplified model estimation cannot capture complex energy consumption behaviors of multi-GPU systems, leading to large prediction errors.

EnergyLens is designed around three core goals: accuracy, usability, and practicality. It uses the einsum interface to describe model specifications, supporting complex pattern expressions; introduces load imbalance-aware modeling for MoE models to capture characteristics like routing imbalance; and establishes mapping relationships based on target hardware benchmark tests through an empirically driven multi-GPU communication energy model.

Validated on Llama3 and Qwen3-MoE models, the energy consumption prediction error in multi-GPU Prefill/Decode phases ranges from 9.25% to 13.19%, and the error for Megatron-style overlapping SM allocation is 12.97%. The energy consumption differences in the configuration space are significant: up to 1.47x in the Prefill phase and as high as 52.9x in the Decode phase; in some scenarios, clusters of multiple small GPUs are more energy-efficient than fewer large-capacity GPUs.

Traditional intuition suggests that more computation-communication overlap and maximizing GPU utilization are better, but EnergyLens finds that excessive overlap may lead to cache invalidation and synchronization overhead. The framework can identify Pareto optimal configurations, which are non-extreme intermediate states balancing latency and energy consumption.

EnergyLens supports pre-deployment configuration exploration: define candidate configurations → predict energy consumption → filter Pareto frontier optimal configurations → validate only promising configurations, reducing tuning costs. It can also quantify the benefits of different optimization strategies and help prioritize them (e.g., optimizing communication overlap is more effective than increasing batch processing).

Current limitations: empirical models need calibration for specific GPUs, assume stable loads, and lack sufficient modeling for dynamic scheduling. Future directions: optimize models with online learning combined with runtime feedback; multi-objective optimization (latency/energy consumption/cost); hardware co-design to provide energy consumption feedback.