A Review of Deep Learning Applications in Hyperspectral Image Classification

This article reviews the latest advances of deep learning techniques in the field of hyperspectral image classification, covering convolutional neural networks (CNN), attention mechanisms, and Transformer-based methods, and discusses the challenges and future development directions in this field. Hyperspectral images (HSI) are three-dimensional data cubes containing hundreds of consecutive narrow bands, which have unique advantages in precision agriculture, environmental monitoring, and other fields, but face challenges such as the curse of dimensionality, spectral variability, and spatial-spectral joint modeling. Deep learning brings breakthroughs to solve these problems by automatically extracting hierarchical features.

Characteristics and Applications of Hyperspectral Images

Hyperspectral images are three-dimensional data cubes containing hundreds of consecutive narrow bands, which can capture the reflection characteristics of ground objects at different wavelengths. Compared with RGB images, they provide richer spectral information and are applied in precision agriculture, environmental monitoring, mineral exploration, military reconnaissance, and other fields.

Core Challenges

1. Curse of Dimensionality: High-dimensional data (hundreds of bands) and limited labeled samples easily lead to overfitting;
2. Spectral Variability: The spectral characteristics of the same ground object vary greatly under different conditions;
3. Spatial-Spectral Joint Modeling: Need to effectively fuse spectral and spatial texture information.

Traditional methods (such as SVM and random forests) are difficult to automatically extract hierarchical abstract features, and deep learning provides new ideas.

CNN is the most widely used deep learning architecture in hyperspectral classification, with the core advantage of capturing local spatial correlation and spectral continuity simultaneously.

• 2D CNN: Early processing of single-band or dimensionality-reduced data, losing spectral information;
• 3D CNN: Using 3D convolution kernels to extract spatial and spectral features simultaneously, which has higher accuracy than traditional methods on datasets like Indian Pines, but has problems of high computational complexity and overfitting;
• Improvement Strategies: Dilated convolution (expanding the receptive field), separable convolution (reducing computational overhead), residual connections (alleviating gradient vanishing).

Attention mechanisms allow the network to dynamically focus on important parts, and are divided into three levels in hyperspectral classification:

• Spectral Attention: Focus on the contribution differences of different bands, such as the SE module which suppresses noise bands and enhances discriminative bands;
• Spatial Attention: Focus on key spatial regions (such as category boundaries, texture-rich areas);
• Hybrid Attention: Simultaneously model spectral and spatial dependencies, such as dual-branch attention networks and Transformer self-attention mechanisms (capturing long-distance dependencies).

Transformer establishes global dependencies through self-attention mechanisms and has sparked a boom in the hyperspectral field since 2020.

• Adaptation Schemes:
  1. Spectral group Transformer: Group adjacent bands into spectral tokens to reduce computational complexity;
  2. 3D patch embedding: Directly divide 3D patches to retain the complete structure;
  3. Pyramid structure: Draw on Swin Transformer to build a multi-scale feature pyramid.
• Pre-training Capability: Pre-training on large-scale unlabeled data (such as masked autoencoders) to learn general representations, suitable for scenarios with scarce annotations.

Multimodal Fusion

• Hyperspectral-LiDAR Fusion: Combine spectral (material identification) and elevation information (geometric structure);
• Hyperspectral-SAR Fusion: Complement optical and all-weather observations, using multi-branch structures for fusion;
• Cross-domain Transfer: Domain adaptation techniques transfer knowledge from the source domain to the target domain, reducing annotation dependence.

Lightweight Strategies

• Knowledge distillation: Transfer knowledge from the teacher network to the student network;
• Network pruning: Remove redundant connections;
• NAS: Automatically search for the optimal structure;
• Quantization/binarization: Low-precision representation reduces storage and computational overhead, supporting real-time processing on embedded devices.

Future Development Directions

1. Self-supervised/Unsupervised Learning: Reduce annotation dependence and use unlabeled data to learn general representations;
2. Physical Interpretability: Integrate physical prior knowledge to improve generalization and interpretability;
3. Open World Recognition: Handle unknown categories and realize open-set/incremental learning;
4. Edge Intelligence: Design ultra-low-power architectures to achieve integrated sensing and computing.

Summary

Deep learning has changed the paradigm of hyperspectral classification, with significant performance improvements from CNN to Transformer, but there is still a gap between academia and applications (such as efficient deployment and scarce annotations). With the progress of new sensors (EnMAP, PRISMA, etc.) and deep learning, hyperspectral analysis will play a more important role in earth observation and other fields.