Google Cloud GPU Benchmark Recipe Library: A Complete Guide to Large-Scale Machine Learning Training and Inference

The GPU benchmark recipe library open-sourced by Google's AI Supercomputing Team provides complete reproducible solutions for training and inference of mainstream large models (e.g., GPT-3, Llama-4), supporting multiple GPU models such as A3 Mega, A3 Ultra, A4, and A4X, helping researchers and engineers quickly find the optimal configuration.

As the parameter scale of large language models grows to hundreds of billions, efficient training and inference have become core challenges for AI infrastructure. The performance of different combinations of hardware, software, and orchestration tools varies significantly. Google's AI Supercomputing Team open-sourced this recipe library, providing a complete workflow from environment preparation to result analysis, offering directly implementable reference implementations for deploying large-scale ML workloads on Google Cloud.

The recipe library covers multiple Google Cloud GPU models:

• A3 Mega (H100)：A mainstream training platform that supports pre-training of models like GPT-3 175B, using the NeMo framework + GKE orchestration;
• A3 Ultra (H200)：Equipped with H200 GPUs, featuring improved memory and bandwidth, supporting pre-training of the Llama-3.1 series, using MaxText/NeMo frameworks;
• A4 (B200)：Based on the Blackwell architecture, excels in inference and fine-tuning, supports PaliGemma2 fine-tuning, using the Hugging Face Accelerate framework;
• A4X (GB200 NVL72)：The current most powerful training platform, supporting ultra-large models like Nemotron-4 340B.

The recipe library provides detailed inference benchmarks:

• Llama-4 inference: Using the SGLang framework on A3 Mega;
• DeepSeek R1 671B: Supports both SGLang and vLLM frameworks;
• GPT OSS 120B: Open-source inference solution on A3 Ultra. It includes key performance tuning parameters such as batch processing optimization and concurrent request handling.

The recipe library adopts a modular design, and each recipe follows a unified structure:

1. Environment Preparation: Infrastructure preparation such as cluster configuration, storage setup, and network optimization;
2. Benchmark Execution: Detailed execution steps to ensure reproducible results;
3. Result Analysis: Provides performance metrics and detailed logs for in-depth analysis. The standardized methodology contributes to the technical progress of the community.

For AI infrastructure teams:

• Quick Start: Directly start work based on validated solutions;
• Performance Benchmark: Understand the theoretical optimal performance of specific hardware;
• Tuning Reference: Compare different configurations to find optimization directions. For researchers: Reproducible results provide a basis for evaluating new algorithms, and detailed logs support performance analysis.

The release of this recipe library marks a step forward in the standardization and transparency of AI infrastructure. In the future, it will expand to more model architectures and hardware platforms, and we look forward to community contributions of optimization techniques and best practices. For teams planning or optimizing AI infrastructure, it is an open-source project worth in-depth research.