From Logistic Regression to ResNet18: A Complete Experimental Comparison of Image Classification on CIFAR-10

Title: From Logistic Regression to ResNet18: A Complete Experimental Comparison of Image Classification on CIFAR-10 Abstract: A systematic deep learning experiment project starting from simple logistic regression, gradually comparing the performance of various models (fully connected neural networks, convolutional neural networks, batch normalization, data augmentation, and ResNet18) on the CIFAR-10 dataset, and providing complete learning curves and performance analysis. Original Author and Source:

• Original Author/Maintainer: uchidaGHub
• Source Platform: GitHub
• Original Project Title: cifarClassificationExperiment
• Project Link: https://github.com/uchidaGHub/cifarClassificationExperiment
• Publication Date: 2026-06-17

Project Background and Objectives

In the field of deep learning, understanding the performance differences between different model architectures is key to mastering core machine learning concepts. This project helps learners understand the evolution path of image classification tasks from basic to advanced levels by systematically comparing various classic and modern neural network models on the CIFAR-10 dataset. As a standard benchmark dataset in computer vision, CIFAR-10 contains 60,000 32x32 color images across 10 categories, making it an ideal choice for testing model capabilities.

Experimental Design and Model Evolution

The project adopts a progressive experimental design, moving from simple models to complex architectures:

Phase 1: Basic Models

Starting with logistic regression (as a baseline comparison), then introducing fully connected neural networks to demonstrate the advantages of multi-layer perceptrons and the problem of overfitting.

Phase 2: Introduction of Convolutional Neural Networks

Implementing two-layer convolutional networks (local receptive field, weight sharing) and three-layer convolutional networks to show the ability of convolutional layers to capture spatial structures of images.

Phase 3: Optimization Techniques

Applying Adam optimizer (accelerating convergence), batch normalization (solving internal covariate shift), and data augmentation (random flipping/cropping to improve generalization).

Phase 4: ResNet18

Implementing the ResNet18 architecture, which solves the gradient vanishing problem in deep networks through residual connections.

Experimental Results and Performance Comparison

The project provides training/test learning curves for each model, showing:

• The relationship between model complexity and overfitting risk
• The impact of different optimization strategies on convergence speed
• The improvement of generalization performance by data augmentation
• The help of residual connections for training deep networks Hyperparameter configurations are compared through cross-validation to provide references for practical applications.

Code Structure and Usage

The project code is modular:

• dataset.py: Loads and preprocesses CIFAR-10, implements data augmentation
• model.py: Defines all model architectures (from linear layers to ResNet blocks)
• utils.py: General functions for training, evaluation, plotting, and cross-validation
• main.py: Experiment entry point with an interactive selection interface Model checkpoints are saved in models/checkpoints, and final models are stored in the models directory.

Learning Value and Practical Significance

This project is a deep learning learning guide. Reproducing the experiments allows you to:

1. Understand the evolution of models (from linear to residual networks)
2. Master PyTorch practice (model definition, training loop, evaluation)
3. Cultivate experimental thinking (hyperparameter tuning, model selection)
4. Establish a cognitive baseline for CIFAR-10 model performance.

Summary and Recommendations

For beginners: Run the experiments in order, observe changes in learning curves, and think about the principles of technical improvements; refer to the project report cifarClassificationReport.pdf for in-depth understanding. For experienced developers: Draw on the code structure (modular design, logging, experiment management).