Handwritten Neural Network: Building a Multilayer Perceptron Step-by-Step with a Calculator

Clippie released the project Walkthrough_MLP_from_Scratch on GitHub. Its core is to use a digital notepad and TI-84 calculator to manually perform the forward and backward propagation calculations of a Multilayer Perceptron (MLP), unveiling the mystery of neural network internal mechanisms from first principles and addressing the vague understanding of underlying principles caused by over-reliance on deep learning framework APIs.

Modern deep learning frameworks (such as PyTorch and TensorFlow) are convenient, but they often lead practitioners to only know how to call APIs without understanding the internal logic of models. The original author adopted an extreme manual calculation approach (without Python/NumPy) to gain a solid understanding of neural network mechanisms and form a reliable intuitive foundation when dealing with complex models. The author said: 'This process helped me unveil the mystery of AI and also made me have deeper respect for the computing power of modern computers.'

Basic Concepts

Artificial neural networks are inspired by biological neurons. A single perceptron is the basic unit (weighted sum of inputs + bias + activation function), and stacking multiple layers forms an MLP.

Architecture Selection

Due to manual calculation constraints, the project uses an extremely simple but complete architecture: 2 neurons in the input layer, 2 in the hidden layer, and 1 in the output layer (functionally a two-layer trainable network). Weights and biases are initialized to random values between 0 and 1 to simplify calculations.

Forward Propagation

1. Hidden layer: Input × weight + bias → Sigmoid activation;
2. Output layer: Weighted sum of hidden layer outputs + bias → Sigmoid activation to get the predicted value.

Backward Propagation

1. Loss calculation: Mean Squared Error (MSE) = (predicted value - true value)² / 2;
2. Gradient calculation: Use the chain rule to find the gradient of the loss with respect to each parameter. The derivative of Sigmoid can be directly calculated from the output (σ’(x) = σ(x) × (1 - σ(x)));
3. Parameter update: New weight = old weight - learning rate × gradient.

1. Demystify Frameworks: Automatic differentiation is essentially a systematic implementation of the chain rule, and GPU acceleration comes from parallelization of matrix operations;
2. Cultivate Numerical Intuition: Observe the impact of weight initialization, learning rate, and activation functions on the model;
3. Understand Computational Complexity: A minimal 2-2-1 network requires dozens of manual steps, while modern models have millions/billions of parameters, leading to enormous computational load.

Tool Selection

• TI-84 calculator: Supports scientific computing and basic programming;
• Digital notepad: Records intermediate results and parameter states.

Training Loop Process

Initialization → Forward propagation → Loss calculation → Backward propagation → Parameter update → Repeat.

Visualization and Documentation

The repository contains detailed calculation screenshots showing each step, making it easy for learners to follow and practice.

Suitable Scenarios

Building intuition in the introductory stage, interview preparation (handwriting backward propagation), improving understanding in teaching scenarios, and returning to basics when debugging model anomalies.

Learning Path

First understand the principles (manual calculation) → then use framework tools → finally debug and optimize (principles + tools). It is opposed to skipping principles and directly using frameworks.

The value of the project is not in solving practical problems, but in demonstrating the attitude of pursuing a thorough understanding of principles. It reminds us to think about the mechanisms behind multiplication, addition, weight updates, etc., when calling model.fit(), to become better deep learning practitioners. In today's era of rapid AI development, this learning attitude that is not satisfied with API calls is particularly precious.