Hands-On Practice of CNN-Based Face Recognition System: From Olivetti Dataset to Complete Project

This article provides an in-depth analysis of an open-source face recognition project based on Convolutional Neural Networks (CNN). Using the classic Olivetti face dataset as the foundation, it fully demonstrates the complete workflow from data preprocessing to model optimization, covering data preprocessing, baseline model construction, architecture tuning, feature map visualization, and performance comparison. It is suitable for deep learning beginners and advanced developers as a reference.

Project Background and Introduction to the Olivetti Dataset

The Olivetti dataset is a classic benchmark dataset in face recognition research, collected by the Cambridge branch of AT&T Laboratories in the 1990s. It contains 400 grayscale face images of 40 different individuals, with 10 photos per person taken under different times, lighting conditions, and facial expressions. All images are uniformly sized at 64×64 pixels.

Compared to modern large-scale face datasets, the Olivetti dataset has a smaller volume, but the challenge lies in extracting effective features to achieve accurate recognition under limited sample conditions, which aligns with the real needs of data-scarce scenarios.

Data Preprocessing: Preparing High-Quality Input for the Model

Data preprocessing is the first critical step to project success:

1. Image Normalization: Scale pixel values from 0-255 to the 0-1 range to accelerate convergence and improve training stability;
2. Data Augmentation: Expand sample diversity through random rotation, horizontal flipping, and slight scaling to reduce overfitting risk;
3. Face Detection and Alignment: Ensure consistent face position and angle through key point localization to minimize pose interference.

Baseline CNN Model Architecture

The baseline model adopts a LeNet-style architecture: it includes two convolutional layers (32 3×3 filters + ReLU, 64 3×3 filters + ReLU), two 2×2 max-pooling layers, and two fully connected layers (128 neurons + Softmax output for 40-class probabilities).

Architecture Tuning and Hyperparameter Optimization

Tuning measures:

• Network Depth: A 6-layer convolutional structure achieves the best balance on the Olivetti dataset;
• Regularization: Dropout (0.5 probability of discarding fully connected layer neurons) + Batch Normalization to accelerate training;
• Learning Rate Scheduling: Cosine annealing algorithm + Early stopping mechanism (terminate training if validation set performance does not improve for 5 rounds).

Feature Map Visualization: Understanding the Model's "Vision"

Feature map visualization shows the internal working of the model:

• First Convolutional Layer: Responds to horizontal, vertical, and diagonal edges;
• Second Convolutional Layer: Captures curved contours and texture patches;
• Deep Network: Activation regions focus on key identity-discriminating parts such as eyes and eyebrows.

Visualization can assist in model diagnosis: If the convolutional kernel activation map shows random noise, initialization needs to be adjusted or regularization strength increased.

Performance Evaluation and Model Comparison

Evaluation results:

• The optimized CNN model achieves a test set accuracy of 97.5% (baseline 92%);
• The confusion matrix shows balanced performance for most categories, with misjudgments only in extreme lighting samples;
• Comparison with traditional methods: PCA Eigenfaces (85%), LBP (89%)—deep learning's automatic feature learning is more robust;
• Inference Speed: 12 milliseconds per image on CPU, and the volume is ≤2MB after ONNX quantization, suitable for mobile deployment.

Practical Insights and Future Outlook

Practical insights: An effective system can be built through data augmentation and regularization under limited data conditions, and a systematic optimization process is crucial.

Future directions:

• Introduce attention mechanisms to focus on key facial regions;
• Try metric learning to extract discriminative feature embeddings;
• Explore lightweight architectures like MobileFaceNet to reduce computational overhead.

Face recognition technology is expanding from 2D to 3D and from static to video streams. Mastering CNN fundamentals and optimization techniques is the foundation for tackling complex visual tasks.