MLP-From-Scratch: A Multilayer Perceptron Neural Network Implemented Purely with NumPy

This article introduces the open-source project MLP-From-Scratch, which builds a Multilayer Perceptron (MLP) neural network entirely from scratch using NumPy without any deep learning framework dependencies. Its core features include numerical stability optimization, a modular data pipeline, and explicit backpropagation implementation, aiming to help learners deeply understand the working principles of neural networks, with both educational value and engineering practice reference significance. The project is maintained by Sampanna-225 and hosted on GitHub.

• Project Source: Maintained by Sampanna-225, hosted on GitHub (link: https://github.com/Sampanna-225/MLP-From-Scratch), released on June 5, 2026.
• Project Positioning: An educational deep learning project that strips away framework abstractions by manually implementing core mechanisms such as forward propagation, backpropagation, and weight updates, allowing developers to directly observe data flow processes and deeply understand neural network principles.

Zero-Dependency Core Design

Implemented entirely based on NumPy without external deep learning library dependencies; the code is transparent and easy to read, suitable for teaching scenarios.

Numerical Stability Optimization

• Sigmoid Activation Function: Implements an overflow clipping mechanism to prevent infinite values, and uses a hybrid ReLU-style gradient strategy to alleviate gradient vanishing, ensuring stable propagation.
• Leaky ReLU: Solves the "dead neuron" problem of traditional ReLU by setting non-zero gradients for negative inputs to maintain activity.

Explicit Gradient Update

Manually implements the backpropagation algorithm, accurately calculates partial derivatives (∂L/∂W, ∂L/∂b), helping to understand the application of the chain rule and the gradient propagation process.

Image Processing Pipeline

For image data such as handwritten digits, uses OpenCV for automatic cropping, supports non-inverted format processing, and automatically extracts regions of interest to reduce noise.

ZIP File Processing Pipeline

Supports decompression of .zip format datasets, can extract MNIST image data, Titanic table data, etc., and convert them into formats usable by the model.

The modular design facilitates the expansion of new data sources; only the corresponding loader needs to be implemented.

Adaptive Training Strategy

• Small datasets (≤32 samples) use full-batch training;
• Large datasets automatically switch to mini-batch training; when the batch size exceeds 32, batch processing is enabled to balance memory and efficiency.

Project Structure

• brain/: Neural network core (layer definitions, activation functions, forward/backward propagation);
• data/: Data processing and loading;
• main/: Main program entry and training scripts;
• ui/: User interface-related code.

Educational Value

• Understand matrix operations, mathematical principles of backpropagation, the impact of activation functions, and the importance of weight initialization;
• Master numerical calculation techniques (avoiding overflow/underflow, gradient clipping);
• Learn engineering practices such as modular code design and clear project structure.

Application Scenarios

1. Teaching demonstration: Assist deep learning courses in understanding internal mechanisms;
2. Interview preparation: Demonstrate in-depth mastery of deep learning principles;
3. Algorithm research: Test new activation functions or optimization strategies;
4. Lightweight applications: Deploy simple neural networks in environments without heavy frameworks.

The MLP-From-Scratch project proves that core deep learning concepts are based on solid mathematical foundations rather than relying on complex frameworks. Through NumPy, one can fully control the details of neural networks; implementing from scratch is the best path to understanding complex systems, as it not only deepens theoretical understanding but also cultivates engineering problem-solving abilities. This open-source project provides a concise and complete reference for teaching, research, and application development; it is recommended that learners try practicing it to deepen their understanding of deep learning.