PocketGrad: A Lightweight Implementation to Understand Automatic Differentiation and Backpropagation from Scratch

PocketGrad is a minimal automatic differentiation engine implemented purely in Python, developed by didarulilm. It references Andrej Karpathy's micrograd project and was released on GitHub on June 4, 2026. With a design philosophy prioritizing readability, it uses scalar-level computation graphs and the chain rule to help learners understand the backpropagation mechanism behind frameworks like PyTorch from first principles. It is positioned as an educational tool rather than a production tool.

Automatic differentiation in deep learning frameworks (such as PyTorch and TensorFlow) is the core of training, but it remains a black box for most users. This state of 'knowing the what but not the why' hinders understanding of the essence of deep learning. PocketGrad was created to solve this problem, allowing learners to dismantle the working principles of backpropagation with their own hands.

PocketGrad chooses scalar-level computation graphs (instead of tensors) for three reasons: 1. The chain rule can be traced, clearly showing the gradient propagation path; 2. Gradients can be manually verified, turning abstract concepts into concrete knowledge; 3. A built-in visualization module can render computation graphs as SVG, intuitively displaying node values and gradients.

PocketGrad consists of three core modules: 1. engine.py: Implements the Scalar class, maintains computation graph connection information, and defines the backward() method to perform backpropagation; 2. nn.py: Provides a micro neural network library (Module, Neuron, Layer, MLP) with an API style consistent with PyTorch; 3. visualize.py: Uses Graphviz to render computation graphs as SVG, aiding debugging and understanding.

Taking the two-moon dataset as an example, the complete training process is demonstrated: 1. Generate non-linearly separable binary classification data; 2. Define an MLP model (2-dimensional input, two hidden layers each with 16 neurons, 1-dimensional output); 3. Training loop: forward propagation → compute binary cross-entropy loss → backward() to calculate gradients → gradient descent to update parameters; 4. Result: The model achieves 100% classification accuracy and clearly separates the two-moon shapes.

PocketGrad explicitly excludes production-level features like vectorization and GPU acceleration to maintain readability. Compared to micrograd, it has engineering improvements (pyproject.toml, CI/CD, unit tests, pip installation) and feature extensions (more comprehensive visualization, support for additional operations, detailed documentation).

It is suitable for the following groups: 1. Deep learning beginners: Implement MLP with their own hands to understand the core process; 2. CS students: Study concepts like computation graphs and topological sorting; 3. Framework developers: A miniature model to understand PyTorch's internal mechanisms; 4. Educators: Ideal material for line-by-line classroom explanations.

The value of PocketGrad lies in helping you understand the essence of backpropagation, not for production applications. When you implement backpropagation with your own hands, your understanding of deep learning will undergo a qualitative change. If you are curious about the internal mechanisms of frameworks, need teaching examples, or enjoy exploring the essence, PocketGrad is worth trying.