SnowSurvey4EfficientLLM: A Systematic Literature Repository for Large Language Model Efficiency Research

SnowSurvey4EfficientLLM is a systematic literature repository focusing on the full-lifecycle efficiency optimization of large language models (LLMs). It covers seven core areas: architectural innovation, model compression, inference acceleration, training optimization, routing strategies, evaluation benchmarks, and open-source tools. It provides a clear knowledge path for researchers, engineers, and learners, addressing the deployment and computational challenges brought by the scaling of LLMs.

As models like GPT, Claude, and Llama exceed 100 billion parameters, training and inference costs have grown exponentially, making efficiency a key factor for LLM deployment. SnowSurvey4EfficientLLM builds a structured knowledge base covering the full lifecycle. Its uniqueness lies in its systematicness and completeness, dividing efficiency research into seven areas to form a well-structured knowledge graph.

The repository covers seven areas:

1. Surveys and Benchmarks: General/subfield surveys, efficiency evaluation benchmarks;
2. Architectural Optimization: Attention variants (GQA/MQA), MoE, alternative architectures (Mamba/RWKV);
3. Model Compression: Quantization (GPTQ/AWQ), pruning (SparseGPT), distillation (MiniLLM), low-rank decomposition (LoRA);
4. Inference Optimization: KV cache compression, speculative decoding, kernel optimization, system frameworks (vLLM);
5. Training Optimization: Efficient pre-training, parameter-efficient fine-tuning (QLoRA);
6. Multi-model Routing: Intelligently selecting and combining models of different scales;
7. Open-source Tools: Inference engines, optimization libraries, resource summaries.

Usage paths for different users: Beginners can build their understanding from surveys; researchers can dive into specific areas; engineers can find open-source tools. The directory includes topic READMEs, paper lists (with abstracts/links), and code links. The project uses the MIT license. The community is welcome to contribute papers according to the template. Maintainers update monthly, with the latest major update in April 2026.

This repository provides a shortcut to cutting-edge research for researchers, a reference for engineers in technology selection, and a knowledge textbook for learners. Efficiency optimization is a technical moat in the competition of large models. The included technical solutions are shaping the engineering practice standards for the next generation of LLMs, supporting reduced cloud costs, edge-side deployment, and improved real-time interaction experiences.

The LLM efficiency revolution is not over; breakthroughs in each link can bring order-of-magnitude performance improvements. With its systematicness and openness, SnowSurvey4EfficientLLM provides the knowledge infrastructure for this revolution and is an invaluable resource treasure for those participating in LLM efficiency research.