llm_note: Systematic Learning Notes on Large Model Inference and High-Performance Computing

llm_note is an open-source technical note repository maintained by community developers, systematically organizing a complete knowledge system from Transformer basics to high-performance computing, from algorithm optimization to framework source code. It is suitable for deep learning engineers to deeply learn the underlying technologies of large models. Its core value lies in bridging the knowledge gap between the application layer and underlying principles, helping developers solve problems such as inference performance optimization and memory debugging, or providing knowledge support for interviews at major companies.

Background

Many developers are proficient in the application layer of large models but know little about the underlying principles, which becomes a bottleneck when optimizing inference performance, debugging memory, or during interviews.

Repository Positioning and Content Overview

llm_note takes 'from theory to practice' as its core concept, helping readers understand technical principles through three dimensions: paper interpretation, source code analysis, and code implementation. The content covers five major sections:

1. Transformer Model Basics
2. LLM Quantization Inference
3. LLM Inference Optimization
4. High-Performance Computing (CUDA/Triton)
5. Source Code Analysis of Mainstream Frameworks

Transformer Model Basics

• Paper Interpretation: Core concepts of 'Attention Is All You Need' (self-attention, multi-head attention, positional encoding), evolution of the GPT series, LLaMA family architecture (GQA, SwiGLU, RoPE)
• Code Implementation: Line-by-line analysis of tensor transformations, including multi-head attention projection, causal masking, LayerNorm/RMSNorm details, and the MLA structure of DeepSeek-V2 (low-rank compression to reduce KV cache memory)

LLM Quantization Inference

• SmoothQuant: No backpropagation required; transfers the difficulty of activation quantization to weights, including source code analysis and effect evaluation
• AWQ: Activation-aware weight quantization strategy that protects important weight channels, compared with methods like GPTQ

LLM Inference Optimization

• Algorithm Level: FlashAttention series (IO-aware to reduce HBM access), Online Softmax (streaming computation), Prompt Cache (long-context KV cache reuse)
• System Level: Core mechanisms of vLLM (PageAttention to eliminate memory fragmentation, Continuous Batching to improve GPU utilization, CUDA Graph to reduce latency), tensor parallelism (column/row parallelism, All-Reduce communication)

High-Performance Computing

• Triton Kernel Development: 5 tutorials (basic concepts, matrix multiplication, attention kernel, fused operators, performance tuning) for writing GPU kernels using Python DSL
• CUDA Programming: Understanding GPU architecture (SM, Warp, shared memory), programming model (thread/memory hierarchy, synchronization), memory optimization (avoiding bank conflicts, coalesced access), multi-card communication (impact of NVLink/PCIe), performance analysis (Nsight tools, Roofline model)
• GPU Architecture Evolution: Key innovations from Volta to Hopper (Tensor Core, asynchronous execution, DPX instruction set)

Framework Analysis and Practice

• Self-developed Inference Framework Course: Based on Triton+PyTorch, modular design, implementing high-performance kernels (FlashAttention, PageAttention, etc.), compatible with models like Qwen3 and LLaMA3, with performance 4x higher than the Transformers library
• Interview Questions Summary: 2025 real questions for high-performance computing/inference framework positions at major companies, covering Transformer, quantization, CUDA, and other directions

Learning Path Recommendations

• Application Developers: Transformer paper → LLaMA architecture → FlashAttention principles → vLLM optimization summary
• Performance Optimization Engineers: Quantization algorithms (SmoothQuant/AWQ) → FlashAttention series → PageAttention/Continuous Batching → Triton basics
• System/Framework Developers: Triton tutorials → CUDA programming → GPU architecture → Roofline analysis

Value Summary

llm_note provides a systematic and in-depth knowledge graph to help readers:

• Avoid information fragmentation and learn along a structured path
• Deeply understand source code instead of just calling APIs
• Prepare for interviews at major companies (classified summary of real questions)
• Gain engineering practice experience from paper to implementation

For AI infrastructure development engineers, this note is a valuable learning resource to help optimize production systems and stand out in interviews.