LLM Quantization Gallery: A Visual Encyclopedia of 93 Large Model Quantization Methods

In the era of large language models (LLMs) with billions to trillions of parameters, quantization is a core model compression technique to reduce computational and storage pressure while preserving performance. However, the proliferation of algorithms (from GPTQ/AWQ to QuaRot/AQLM) makes systematic understanding challenging. The LLM Quantization Gallery—an open-source knowledge base maintained by Arpit Singh Gautam—addresses this by visually organizing 93 quantization methods across 8 categories, each with flowcharts, technical cards, and cross-references, serving as a 'visual encyclopedia' for model compression learning.

Inspired by Sebastian Raschka’s llm-architecture-gallery, the Quantization Gallery was created to fill the gap of a centralized visual resource for quantization. Unlike traditional paper lists or code repositories, it emphasizes visual learning: each method includes SVG flowcharts and Mermaid diagrams to intuitively show algorithm mechanisms, helping readers quickly grasp core ideas, applicable scenarios, and technical evolution.

The gallery categorizes 93 methods into 8 key types:

1. Post-Training Quantization (PTQ)：Widely used in industry (e.g., GPTQ, AWQ, SmoothQuant, SpQR).
2. Quantization-Aware Training (QAT)：Simulate quantization during training (e.g., LLM-QAT, QLoRA, BitDistiller).
3. Outlier Handling：Solve activation outlier issues (e.g., LLM.int8(), Outlier Suppression+, QuaRot).
4. Inference-Optimized Formats：Hardware/engine-specific (e.g., GGUF series, EXL2, Marlin, FP6-LLM).
5. Fine-Grained & Adaptive Quantization：Beyond layer/tensor-level (e.g., AWQ channel scaling, OmniQuant, AQLM).

Key strengths include:

• Systematic Organization: Timeline (by publication date), lineage diagrams (method inheritance like GPTQ→SpQR→QuIP), symbol guide (W4A16, W8A8KV4), and glossary.
• High-Quality Visuals: Each method’s tech card has core idea, SVG flowcharts, technical details (bitwidth, group size, calibration data), and paper/code links.
• Open Source Collaboration: MIT license, standardized YAML files (methods.yml) for easy community contributions to keep the knowledge base updated.

The gallery is suitable for:

1. Model Deployment Engineers: Compare method features to choose optimal solutions for specific hardware.
2. AI Researchers: Understand the full technical landscape and identify research gaps.
3. Learners: Use visual charts to grasp complex quantization principles.
4. Technical Decision-Makers: Evaluate the impact of quantization strategies on performance and resource consumption.

Based on the gallery’s content, here are practical tips:

• Extreme Compression: Choose 2-3bit methods like QuIP# or AQLM (with minor precision loss).
• Production Deployment: GPTQ/AWQ with 4bit configuration (cost-effective).
• Edge Device Inference: GGUF format + llama.cpp (community-proven).
• Fine-Tuning Scenarios: QLoRA + NF4 (works on consumer GPUs).
• High-Throughput Services: SmoothQuant/Atom (W8A8) for INT8 Tensor Core-supported GPUs.

As LLM applications become widespread, quantization’s importance grows. The LLM Quantization Gallery serves as a structured knowledge hub via open collaboration. Its 'visual-first' design makes complex algorithms accessible—one clear flowchart often beats thousands of words. Whether you’re a beginner or a seasoned practitioner, this project is a must-bookmark resource for exploring LLM quantization.