100-Day Inference Engineering Challenge: A Systematic Learning Path from CUDA Kernels to Multi-Cloud Auto-Scaling

This project is a systematic learning path built on Philip Kiely's Inference Engineering, aiming to help developers master the full-stack technologies of LLM inference engineering—from low-level CUDA kernel optimization to upper-layer cloud-native architecture design. Framed as a 100-day progressive learning journey, the project covers three core layers (single GPU optimization, multi-GPU collaboration, tools and observability) through runnable scripts and experiments, ultimately cultivating production-grade LLM deployment capabilities. Its features include practice orientation (all experiments are validated on DGX Spark clusters) and structured coverage, providing inference engineers with a complete knowledge system.

Inference engineering is a complex discipline spanning multiple domains such as CUDA optimization and cloud-native architecture. As Philip Kiely put it: "Doing inference well requires three layers: runtime, infrastructure, and tools." Current fragmented tutorials make it difficult to build a complete knowledge system, so the 100 Days of Inference project was born—based on the book Inference Engineering, it helps developers fully master skills in all aspects of LLM inference engineering through a systematic learning path.

The project is divided into three phases:

1. Single GPU Optimization (Days 1-18)：Covers LLM inference mechanisms, CUDA kernels, frameworks like vLLM/SGLang, and advanced techniques such as quantization and speculative decoding;
2. Multi-GPU & Infrastructure (Days19-27)：Includes GPU architecture (SM, HBM), containerization (Docker/NVIDIA NIMs), auto-scaling, and multi-cloud capacity management;
3. Tools & Observability (Days28-30)：Covers performance benchmarking, monitoring metrics (TTFT/TPOT), and client-side code design.

The project provides numerous runnable experiments, including:

• Core Implementations: Building BPE tokenizers from scratch, SDPA attention mechanisms;
• Quantization Optimization: INT8 quantization pipelines, GPTQ-style rounding;
• Caching & Parallelism: KV cache managers, tensor parallelism simulation;
• Deployment Practice: Triton custom CUDA kernels, vLLM/SGLang deployment benchmarking;
• System-Level Projects: Continuous batching simulation, Dockerfile writing. All experiments help learners turn theory into practical skills.

The project is suitable for AI infrastructure engineers, ML practitioners, technical leads, and researchers. Learning values include:

• Systematic Knowledge: Building a complete inference engineering system from bottom to top;
• Practical Skills: Mastering production-grade deployment through runnable code;
• Community Support: Opportunities for communication and contribution from open-source projects;
• Production Readiness: Directly addressing inference optimization issues in real production environments.

100 Days of Inference represents a new model of AI education—systematic, practical, and production-oriented. In today's era of rapid LLM development, inference engineering capabilities have become the core competence of AI infrastructure. The project is hosted on GitHub, with all code and documents open-source. Whether you follow the full 100 days or choose modular learning, you can start immediately. A 100-day investment will bring an in-depth understanding of the full stack of LLM inference, which is worth trying for developers.