IronCore: A Complete Practice of Building a Personal LLM Training Framework from Scratch

IronCore is a personal project maintained by haanjack, an end-to-end LLM training framework designed for individual developers, supporting the full workflow from pre-training to alignment. Based on YAML configuration, the framework retains core industrial-grade features (such as distributed training, parallel strategies, and alignment methods) while reducing complexity, allowing developers to conduct experiments on limited resources (e.g., dual RTX 3090), aiming to help developers deeply understand the internal mechanisms of LLM training.

Most current LLM developers are in the "user" role and lack an understanding of the complete training workflow. IronCore was born to fill this gap, positioned as a personal project for learning and experimentation, inspired by NVIDIA Megatron-LM and HuggingFace Transformers, focusing on simplicity and understandability, supporting individuals to complete end-to-end training on limited hardware.

Multi-Stage Training Support

Built-in four training modes: pre-training (streaming corpus processing), supervised fine-tuning (SFT), direct preference optimization (DPO), group relative policy optimization (GRPO), enabling full lifecycle training within the same framework.

Parallel Strategies

Implements tensor parallelism (TP), data parallelism (DP), expert parallelism (EP), and fully sharded data parallelism (FSDP), supporting combined use to adapt to different hardware scenarios.

MoE Architecture

Built-in Mixture of Experts (MoE) architecture support, including load balancing loss, Z-loss, and expert parallelism strategies, ensuring efficient computation of sparse activations.

LoRA Implementation

Provides LoRA compatible with tensor parallelism, training only a small number of low-rank matrices to adapt to downstream tasks, solving the correctness issues of gradient computation and parameter updates in TP mode.

Optimizers

Introduces the Muon optimizer (combining orthogonalization and AdamW), supports ZeRO-1 distributed optimizer, reducing memory usage and improving convergence characteristics.

GRPO is a featured function of IronCore, adopting an online learning paradigm:

• Generation Phase: Generate multiple candidate responses for each prompt, using KV caching for efficient generation;
• Evaluation Phase: Score via reward models, supporting multiple reward backends such as mathematical verification and code execution;
• Optimization Phase: Calculate intra-group relative advantages, stabilize training via IS ratio clipping, and use KL penalty to prevent deviation from the reference model. Suitable for complex scenarios like mathematical reasoning and code generation.

Data Preprocessing

Supports Fill-in-the-Middle (FIM) technology, using PSM format and configurable splitting strategies to enhance the bidirectional understanding ability of code models.

Unified Model Architecture

Shields underlying differences via the TransformerModel interface, supports multiple models such as GPT-2/3 and LLaMA, features include Pre-norm/Post-norm, GQA/MQA/RoPE, and multiple activation functions, switching architectures only requires modifying the configuration.

Containerized Workflow

Recommends using NGC PyTorch containers, provides Docker scripts supporting CUDA/ROCm backends, ensuring correct operation of optimization libraries like Flash Attention.

Configuration-Driven

Uses YAML configuration to define training tasks (model, data, parallel strategies, optimizers, etc.), reducing the complexity of experiment management.

Observability

Built-in MFU calculator to monitor efficiency, supports TensorBoard, WandB, and MLflow logging backends.

Limitations

The current version does not support sliding window attention, multimodal input, or encoder-decoder architecture, focusing on the core workflow of decoder-only models.

Summary

IronCore demonstrates the open-source model maintained by individuals, providing developers with the opportunity to participate in LLM training. For Chinese developers, it proves the feasibility of completing end-to-end training on consumer-grade hardware, making it an ideal platform for learning, research, or small-scale experiments.