MyLLM: A Complete Practical Framework for Building Large Language Models from Scratch

MyLLM is an education-oriented, research-friendly large language model framework designed to address the "black-box dependency" problem in the current LLM ecosystem—where developers rely on high-level abstraction libraries but have only a superficial understanding of the internal principles of Transformers. The framework covers the complete workflow from tokenization, attention mechanisms, training to RLHF and inference, and adopts a three-layer progressive architecture (Notebooks, Modules, Core Framework). Its core values lie in transparency, modifiability, and research-friendliness, making it suitable for learning and rapid experimentation, but it is not designed for production environments.

Currently, high-level libraries like Hugging Face and PyTorch Lightning have lowered the threshold for LLM development, but they have also led to many developers only being able to call APIs without understanding the internal working principles of Transformers, forming a "black-box dependency". The MyLLM project was born to address this; its core goal is to enable users to understand every line of code in the modern Transformer tech stack, building a clean, research-grade transparent implementation framework rather than a production tool that pursues ultimate performance.

MyLLM adopts a three-layer structural design:

1. Notebooks Layer: 21 Jupyter Notebooks covering data and tokenization, attention mechanisms, model architectures (GPT vs LLaMA comparison), training techniques (pretraining/SFT/PEFT), RLHF (PPO/DPO), and inference optimization (KV Cache/quantization). Each notebook supports independent running and experimentation (e.g., modifying attention masks to observe generation effects).
2. Modules Layer: Breaks the system into independent modules (data, model, training, fine-tuning, inference) for easy isolation and validation of new ideas.
3. myllm Core Layer: A pure PyTorch implementable framework that includes model definitions (GPT/LLaMA-style Transformers), an API layer, a configuration system, tokenizers (GPT2/LLaMA series), a training engine (SFT/DPO/PPO), and distributed support (DDP/DeepSpeed/FSDP).

MyLLM's design philosophy differs from existing libraries:

• Minimalism: Remove unnecessary abstraction layers; each line of code has a clear purpose for easy debugging and modification.
• Modifiability: All components are visible and editable, supporting replacement of attention mechanisms, trying new positional encodings, or modifying loss functions.
• Research Orientation: Built-in cutting-edge technologies like LoRA, QLoRA, PPO, DPO, and quantization, with transparent implementations for easy expansion.
• Built from Scratch: No reliance on pre-trained weight "magic"; all mechanisms are clearly demonstrated through code.

MyLLM's tests run on randomly initialized small models (2 layers/64 dimensions) and can be completed on CPU without pre-trained weights. The tests cover 128 use cases including:

• Configuration system (preset validation, save/load, memory estimation)
• Model components (MLP variants, KV Cache, RMSNorm, RoPE)
• Tokenizers (GPT-2 encoding/decoding, special token handling)
• API layer (generation functions and sampling modes)
• Training system (three trainers, checkpoint management)
• End-to-end flow (initialization → training → inference) Comprehensive coverage ensures framework reliability and provides numerous usage examples.

MyLLM's educational value is significant, suitable for the following groups:

• AI/ML students: Systematically learn full-stack LLM knowledge through 21 notebooks, achieving seamless connection from theory to practice.
• Researchers: Transparent code structure facilitates rapid experimentation of new ideas, avoiding getting lost in complex abstractions.
• Transitioning engineers: Gain in-depth understanding of LLM internal mechanisms, breaking through the limitation of only relying on API calls.
• Open-source contributors: Clear module division and comprehensive test system lower the threshold for contributions.

MyLLM is not designed for production environments; its applicable scenarios include:

• Learning tool: Practical material to understand LLM working principles
• Research prototype: Rapid experiment platform to validate new ideas
• Teaching resource: Supporting project for systematic LLM courses For scenarios pursuing ultimate performance or large-scale deployment, the Hugging Face ecosystem remains a more mature choice. It is recommended to first build basic cognition through MyLLM before moving to production tools.

MyLLM represents an important open-source paradigm: in an era of rampant high-level abstractions, consciously maintaining transparency and understandability. It is not just a codebase but a set of learning methodologies of "understand → experiment → frameworkize", helping developers build solid technical intuition. As LLM technology evolves, such transparent implementations built from scratch will become increasingly precious, providing a trustworthy foundation for the community and serving as a high-quality project for deepening into the LLM field.