Deep Learning Practice: Implementing MNIST Handwritten Digit Recognition with Convolutional Neural Networks

This project focuses on implementing MNIST handwritten digit recognition using Convolutional Neural Networks (CNN), covering dataset characteristics, model architecture design, Adam vs. SGD optimizer comparison experiments, and the complete training process, providing a practical reference case for deep learning beginners. Key content includes data preprocessing, CNN hierarchical feature extraction, optimizer performance analysis, and result visualization, among other critical steps.

Handwritten digit recognition is a classic computer vision problem and an introductory deep learning practice project. Since its release by Yann LeCun et al. in 1998, the MNIST dataset has become a standard benchmark for validating algorithms: it contains 60,000 training images and 10,000 test images, all 28×28 pixel grayscale images corresponding to 10 categories (0-9). The data sources are handwritten samples from U.S. Census Bureau employees and high school students, and have undergone standardization (centered digits, fixed size).

Data Preprocessing

1. Pixel value normalization: Convert 0-255 grayscale values to 0-1 (formula: normalized value = original value / 255.0) to accelerate model convergence.
2. Dimension reshaping: Convert images to (28,28,1) 3D tensors to fit CNN input (1 represents single-channel grayscale).
3. Label one-hot encoding: For example, digit 3 is encoded as [0,0,0,1,0,0,0,0,0,0], which is used with Softmax to calculate cross-entropy loss.

CNN Architecture

• First Conv2D layer (32 filters, ReLU): Extract low-level features like edges
• First MaxPooling layer: Reduce dimensionality, decrease computation, enhance translation invariance
• Second Conv2D layer (64 filters, ReLU): Extract complex patterns
• Second MaxPooling layer: Further dimensionality reduction
• Flatten layer: Convert 2D features to 1D vector
• Dense layer (128 neurons, ReLU): Integrate features
• Dropout layer (dropout rate 0.5): Prevent overfitting
• Output layer (10 neurons, Softmax): Output class probability distribution

Training Configuration

• Training epochs: 5
• Batch size: 64 (balance GPU parallelism and memory usage)
• Validation set ratio: 20% (split from training set)

Optimizer Comparison

• Adam: Combines momentum method and RMSProp, adaptive learning rate, moderate memory requirement, good performance with default parameters
• SGD: Basic optimization algorithm, can accelerate convergence with momentum, requires careful learning rate tuning, better generalization in some scenarios

Model Configuration	Accuracy	Loss Value
CNN+Adam	99.04%	0.0294
CNN+SGD	97.02%	0.0969

Key Findings:

1. Adam's accuracy is about 2 percentage points higher than SGD, showing significant improvement on the MNIST dataset
2. Adam's loss value is much lower than SGD, with higher prediction confidence
3. Adam converges faster, suitable for resource-limited or fast iteration scenarios

Visualization: Monitor overfitting via accuracy/loss curves (e.g., if validation accuracy is lower than training accuracy and validation loss increases while training loss decreases, it signals overfitting)

Tech Stack

Use TensorFlow/Keras to build models, NumPy for numerical processing, Matplotlib for visualization, Pandas for result analysis, supporting Google Colab cloud execution.

Extension Directions

1. Apply to Fashion-MNIST (fashion item classification), CIFAR-10/100 (color image classification)
2. Real-world scenarios: Bank check recognition, postal code recognition

Summary

This project covers the entire workflow of a deep learning classification system. Key takeaways: CNN can extract hierarchical features, preprocessing affects performance, Adam optimizer is better in most scenarios, Dropout regularization prevents overfitting, visualization helps model debugging. The MNIST project is an ideal starting point for beginners; after mastering core concepts at low cost, one can move to complex computer vision tasks.