Implementing a Multilayer Perceptron from Scratch: A Complete Practice with Pure NumPy Neural Networks

A project that implements a Multilayer Perceptron (MLP) from scratch using pure NumPy, including a complete neural network architecture, forward/backward propagation, SGD and Adam optimizers, and a practical case of breast cancer classification.

• Original Author/Maintainer: Donghan5
• Source Platform: GitHub
• Original Title: multilayer_perceptron
• Original Link: https://github.com/Donghan5/multilayer_perceptron
• Release Time: 2025 (project under active maintenance)

This project is a teaching-oriented neural network implementation aimed at helping learners understand the basic principles of artificial neural networks. Built entirely from scratch using NumPy without relying on any deep learning frameworks, it implements a complete MLP architecture and applies it to the Wisconsin Breast Cancer Dataset for a binary classification task (malignant vs. benign tumors).

The dataset contains 569 samples and 30 features, making it a classic small-scale machine learning dataset ideal for verifying the correctness of neural network implementations.

The project uses a modular design with clear responsibilities for each component:

Responsible for splitting raw data into training and validation sets in an 80/20 ratio, and supports fixing random seeds to ensure reproducible results. This step is crucial for preventing data leakage and obtaining reliable model evaluations.

Implements the forward and backward propagation algorithms of the neural network. It supports configurable hidden layer structures, with a default configuration of three hidden layers each containing 24 neurons. Weight initialization supports two strategies: He Uniform and Xavier Uniform, which are suitable for ReLU and Sigmoid activation functions respectively.

Contains implementations of activation functions such as Sigmoid, ReLU, and Softmax, as well as cross-entropy loss function and one-hot encoding tools. These basic components are the cornerstones of neural network operations.

Implements two classic optimization algorithms:

SGD (Stochastic Gradient Descent) : The most basic optimization method, which updates all parameters using a single global learning rate.

Adam (Adaptive Moment Estimation) : Combines the advantages of Momentum and RMSProp, maintains first-moment (gradient mean) and second-moment (mean of squared gradients) estimates for each parameter, and achieves adaptive learning rate adjustment through bias correction.