hdrnn: A Practical Introduction to Neural Networks for Handwritten Digit Recognition

Project Overview

The hdrnn project (maintained by author adnlv, hosted on GitHub, released on 2026-05-26) is a practical entry point for learning neural networks through handwritten digit recognition. It demonstrates building a model to identify 0-9 digits using the MNIST dataset, covering core steps like data preprocessing, network architecture design, training flow, and evaluation metrics. This project provides a clear path for beginners to grasp deep learning fundamentals.

Source: GitHub repository

Handwritten digit recognition is a classic machine learning entry problem. Since Yann LeCun's LeNet-5 in 1998, the MNIST dataset (60,000 training images and 10,000 test images) has become a standard benchmark.

Reasons for choosing this task:

1. Clear problem definition: Input is 28×28 grayscale images, output is 0-9 class labels.
2. Easy data access: MNIST is public and preprocessed.
3. Resource-friendly: Runs efficiently on modern CPUs.
4. Visualizable: Input, output, and intermediate features are intuitive.
5. Rich benchmarks: Compare results with existing work.

It's an ideal way to understand neural network principles.

Dataset Processing

• Normalization: Scale pixel values from [0,255] to [0,1] or [-1,1] to aid gradient convergence.
• Flattening: Convert 2D 28×28 images to 1D 784-dimensional vectors.
• Label encoding: One-hot encode integer labels (0-9).

Network Architecture

Typical structure: Input (784 neurons) → Hidden (128) → Hidden (64) → Output (10 neurons).

• Input layer: Receives flattened image vectors.
• Hidden layers: Use ReLU for nonlinearity.
• Output layer: Softmax for probability distribution over 10 digits.

Training Flow

1. Forward propagation: Compute predictions.
2. Loss calculation: Cross-entropy loss between predictions and true labels. 3 Backward propagation: Calculate gradients.
3. Parameter update: Gradient descent or variants (Adam, SGD with momentum).

Evaluation Metrics

• Accuracy: Ratio of correct predictions.
• Confusion matrix: Show class-wise prediction performance.
• Loss curve: Monitor training progress and detect overfitting.

Activation Functions

• ReLU: Simple, mitigates gradient vanishing (default choice).
• Sigmoid/Tanh: Traditional but have gradient saturation issues.
• Softmax: For output layer, converts logits to probabilities.

Loss Function

Categorical cross-entropy