large_language_model_learning: A Collection of Large Language Model Learning Resources

A GitHub repository that collects learning materials for large language models (LLMs), providing systematic learning paths and resource indexes for learners who wish to deeply understand and master LLM technology.

large_language_model_learning is a GitHub repository created by KevinXie0131, aiming to provide systematic learning resources for LLM learners. With the rapid development of LLM technology today, learning resources are scattered everywhere. This project attempts to gather these resources and provide clear learning paths for both beginners and advanced learners. Although the repository's README is relatively concise, such resource aggregation projects have important reference value for the technical community.

The development speed of the large language model field is unprecedented:

• New models emerge one after another: From GPT to Claude, Llama to Qwen, new models are continuously released
• Rapid technology iteration: Architectures, training methods, and optimization techniques are continuously updated
• Expansion of application scenarios: From text generation to code, multimodal, Agent, etc., the scenarios are constantly expanding
• Surge in research papers: A large number of related papers are published on arXiv every day

Resource dilemmas faced by learners:

• Information overload: Massive resources make people unsure where to start
• Uneven quality: High-quality papers are mixed with low-quality blogs
• Lack of systematicness: There is no organic connection between knowledge points
• Update lag: Many tutorials are based on outdated technology versions

LLM learning requires interdisciplinary knowledge:

• Deep learning basics: Neural networks, backpropagation, optimization algorithms
• Natural language processing: Word embedding, attention mechanism, Transformer
• Distributed computing: Large-scale model training requires knowledge of distributed systems
• Engineering practice: Model deployment, inference optimization, API design

A high-quality LLM learning resource collection should provide:

Beginner Stage

• Basic concepts: What are large language models, basic principles
• Historical evolution: Development from RNN to Transformer to GPT
• Key papers: Milestone papers like Transformer, GPT series, BERT
• Hands-on practice: Quick experience using tools like Hugging Face

Advanced Stage

• In-depth understanding: Details like attention mechanism, positional encoding, layer normalization
• Training techniques: Training methods like pre-training, fine-tuning, RLHF
• Optimization techniques: Quantization, pruning, distillation, speculative decoding, etc.
• Open-source models: Research on open-source models like Llama, Qwen, ChatGLM

Expert Stage

• Cutting-edge research: MoE, multimodal, long context, reasoning enhancement
• Systems engineering: Distributed training, inference services, cost optimization
• Application development: Application patterns like RAG, Agent, tool usage
• Safety alignment: Alignment technology, safety research, red team testing

Paper Resources

• Must-read papers: Core papers for each stage
• Latest progress: Track the latest research trends
• Chinese interpretations: High-quality Chinese interpretations of papers
• Code implementations: Paper reproductions with accompanying code

Course Resources

• Online courses: Relevant courses on platforms like Coursera, edX
• Video tutorials: Teaching videos on platforms like YouTube, Bilibili
• Book recommendations: Classic textbooks and latest monographs
• Practical projects: Suggestions for hands-on projects

Tool Resources

• Development frameworks: PyTorch, TensorFlow, JAX, etc.
• Model libraries: Hugging Face, ModelScope, etc.
• Deployment tools: vLLM, SGLang, TensorRT-LLM, etc.
• Evaluation tools: Evaluation frameworks and datasets

Community Resources

• Technical blogs: Technical sharing from excellent bloggers
• Open-source projects: Open-source implementations worth learning
• Discussion communities: Communities like Reddit, Discord, Zhihu
• Conference materials: Papers and tutorials from conferences like NeurIPS, ICML, ACL

Suitable for developers who want to build applications using LLMs:

1. Basic understanding: Understand the basic capabilities and limitations of LLMs
2. API usage: Learn to call APIs like OpenAI, Claude
3. Prompt engineering: Master effective prompt design skills
4. RAG technology: Learn retrieval-augmented generation
5. Agent development: Build AI Agents that can use tools
6. Deployment optimization: Learn model deployment and inference optimization