Implementing MNIST Neural Network from Scratch with NumPy: A Hands-On Guide to Handwritten Digit Recognition

This article introduces the mnist project published by yacine204 on GitHub (link: https://github.com/yacine204/mnist, released on May 23, 2026). The project implements a handwritten digit recognition neural network from scratch entirely using NumPy, without relying on frameworks like TensorFlow or PyTorch. It aims to help learners understand the underlying mechanisms of neural networks (such as forward propagation, backpropagation, gradient descent, etc.) and is a high-quality resource for in-depth learning of deep learning principles.

MNIST is a classic handwritten digit dataset for deep learning beginners, containing 60,000 training images and 10,000 test images. Each image is a 28×28 grayscale image, covering 10 categories (0-9). Most learners use high-level frameworks to quickly build models, but these frameworks encapsulate underlying details, making it difficult to understand how neural networks work. This project solves this problem through pure NumPy implementation.

The neural network implemented in the project includes an input layer (784 neurons, corresponding to 28×28 pixels), a hidden layer (with activation functions like ReLU), and an output layer (10 neurons). Key steps:

1. Forward propagation: Linear transformation (Z = W·X + b) + activation function (ReLU/Sigmoid/Softmax);
2. Loss function: Cross-entropy loss (L = -Σy_true·log(y_pred));
3. Backpropagation: Calculate gradients using the chain rule;
4. Gradient descent: Update weights (W_new = W_old - learning_rate × gradient).

The project supports training, test evaluation, and custom image prediction. During training, it can monitor loss changes; the test set accuracy is about 95%, showing good performance; it can also recognize handwritten images provided by users, which is highly practical.

Pure NumPy implementation allows developers to write every formula step by step, helping to deeply understand: the reasons for weight initialization, the importance of activation functions, the mechanism of gradient vanishing/explosion, and the impact of learning rate. At the same time, it enables proficiency in basic data science skills such as matrix multiplication, broadcasting mechanism, and vectorized computation, laying a solid foundation for subsequent use of frameworks.

Recommended learning steps:

1. Run the project and observe the results;
2. Read the source code line by line to understand the role of each function;
3. Modify parameters (network structure, learning rate, activation function) and observe changes;
4. Try to implement it yourself without looking at the source code;
5. Implement the same structure using PyTorch/TensorFlow and compare the differences.

Possible optimization directions for the project:

1. Network structure: Add hidden layers/neurons, try different activation functions, and add Dropout;
2. Optimization algorithms: Implement Momentum/RMSprop/Adam, add learning rate decay and batch normalization;
3. Data augmentation: Rotate, translate, scale images, and add noise;
4. Upgrade to Convolutional Neural Network (CNN) to improve accuracy.

This project focuses on transparency and understandability and is an excellent teaching resource for deep learning beginners. By implementing the neural network by hand, learners' understanding of deep learning will far exceed those who only call framework APIs. MNIST is a starting point, and mastering this project will lay the foundation for learning complex models such as CNN, RNN, and Transformer.