In-depth Analysis of EfficientNet: Rethinking Model Scaling Strategies for Convolutional Neural Networks

This article provides an in-depth interpretation of the EfficientNet paper and its PyTorch implementation. The core lies in the proposal of the Compound Scaling strategy, which collaboratively optimizes the three dimensions of convolutional neural networks—depth, width, and input resolution—to balance accuracy and computational efficiency. It also introduces its baseline architecture EfficientNet-B0 (including MBConv and SE modules), performance, application scenarios, and future development directions.

Since AlexNet, CNNs have improved performance by increasing depth/width, but there are three major issues: 1. Explosive computational resources (high training and inference costs, difficult to deploy on mobile/edge devices); 2. Diminishing marginal returns (limited gains from increased depth, with sharply rising overhead); 3. Dimension imbalance (scaling a single dimension makes it hard to reach the optimal balance). In 2019, Google Research proposed EfficientNet to address these problems.

Compound Scaling core: Depth, width, and resolution are interrelated and need to be optimized collaboratively. Formula: d' = α·d, w' = β·w, r' = γ·r, with the constraint α·β²·γ² ≈2 (scale proportionally when the budget doubles). The optimal coefficients are α=1.2, β=1.1, γ=1.15. Baseline architecture B0: Centered on MBConv (Mobile Inverted Bottleneck Convolution, including three stages of expansion→depthwise separable convolution→compression + linear bottleneck + skip connection), integrated with SE attention module (Squeeze→Excitation→Recalibration) to enhance feature expression.

Open-source projects adopt a modular design: ConvBNReLU (basic unit), MBConv (configurable expansion ratio/kernel/SE), and the EfficientNet class (integrating blocks of each stage). The compound scaling code is implemented via round_filters (adjusting channel count) and round_repeats (adjusting repetition count). It supports loading ImageNet pre-trained weights (EfficientNet.from_pretrained('efficientnet-b0')) to accelerate transfer learning.

On ImageNet: B0 achieves 77.3% Top-1 accuracy with 5.3M parameters and 0.39B FLOPs, outperforming ResNet-50 (25.6M parameters, 4.1B FLOPs, 76.0% accuracy); B7 achieves 84.3% accuracy with 66M parameters and 37B FLOPs, surpassing GPipe (557M parameters) while having 8.4x fewer parameters. Transfer learning performs excellently on datasets like CIFAR-10/100 and Flowers, with strong generalization.

EfficientNet is widely used due to its efficiency: 1. Mobile vision (B0/B1 deployed in apps for real-time classification and detection); 2. Edge computing (balancing accuracy and latency on embedded/IoT devices); 3. Cloud inference (B7 as a high-precision API backend); 4. Medical image analysis (strong feature extraction capability, easy for local deployment).

Limitations: Deep models (e.g., B7) are difficult to train and require fine tuning; inference latency on some hardware exceeds expectations. Future improvements: EfficientNetV2 introduces Fused-MBConv and progressive learning; Noisy Student training enhances performance.

EfficientNet conveys the efficiency-first philosophy: pursuing the optimal balance between accuracy and computational cost, rather than mere metric breakthroughs. Compound scaling is highly versatile and can be extended to architectures like Transformers (e.g., ViT draws on similar ideas). Insights for practitioners: Start from the core problem, propose a concise solution and validate it, practicing the first-principles research approach.