Multimodal Disaster Detection System: Intelligent Building Damage Assessment Combining Optical and SAR Imagery

After a natural disaster, quickly and accurately assessing building damage is crucial for rescue decision-making and post-disaster reconstruction. The disaster-detection project proposes an innovative multimodal AI solution that combines Sentinel satellite optical and SAR imagery, using the Segment Anything Model (SAM) segmentation model and dual-encoder ResNet network to achieve automated intelligent assessment of building damage before and after disasters, addressing issues such as low efficiency and limitations of single data sources in traditional assessment methods.

Traditional disaster assessment relies on manual surveys or expert visual interpretation, which is inefficient and difficult to cover large areas. While satellite remote sensing provides a macro perspective, single data sources have limitations: optical imagery is restricted by weather and lighting conditions, and SAR imagery has strong cloud-penetrating capabilities but is difficult to interpret. In addition, accurately identifying building outlines and classifying damage levels requires combining advances in computer vision; core challenges include multi-source heterogeneous data fusion, pre- and post-disaster image registration, and few-shot/zero-shot learning.

The project adopts a multimodal fusion architecture, integrating two complementary satellite data sources:

• Pre-disaster optical imagery: From Sentinel-2 L2A satellites, providing high-resolution multispectral imagery that clearly shows surface features such as building appearance and vegetation, serving as an important source of pre-disaster baseline data.
• Post-disaster SAR imagery: From Sentinel-1 GRD satellites, not restricted by clouds or lighting, enabling continuous data acquisition under severe weather conditions, which is crucial for disaster response. Through registration and fusion, the high spatial resolution of optical imagery and the all-weather capability of SAR are comprehensively utilized to improve the accuracy and robustness of damage detection.

The project integrates two key AI components:

1. Zero-shot building extraction: SAM model Meta's SAM model has strong zero-shot generalization capabilities, enabling high-quality segmentation without training on specific scenarios. It is used to extract building footprints from pre-disaster optical imagery, defining the spatial scope for subsequent assessment, and can be applied to any region globally without pre-labeled data.
2. Damage classification: Dual-encoder ResNet network A customized dual-encoder architecture processes pre- and post-disaster image features separately, integrating spatiotemporal information through a fusion module to output damage levels. Its advantage lies in learning baseline and change features separately, focusing on the change patterns of building areas, and being better able to capture complex damage features than simple difference or single-encoder methods.

The complete workflow of the system:

1. Data preprocessing: Perform atmospheric correction, radiometric calibration, geometric correction, etc., on Sentinel-2 and Sentinel-1 data to ensure precise spatial registration.
2. Building footprint extraction: Input the preprocessed pre-disaster optical imagery into the SAM model to obtain building area segmentation masks and define regions of interest.
3. Feature extraction and fusion: The dual-encoder network processes the registered pre- and post-disaster images separately, extracting multi-scale features (including residual connections, pyramid pooling, and other structures).
4. Damage classification: The fused features are passed through a classification head to output damage levels (no damage, mild, moderate, severe, complete damage, etc.).

The system has a wide range of potential application scenarios:

• Emergency response: Quickly generate damage maps after disasters to guide rescue deployment and priority ranking.
• Insurance claims: Provide objective assessment basis to accelerate the claims process and reduce subjective delays.
• Urban planning and disaster prevention: Accumulate historical data, analyze building vulnerability, and support planning and regulation revisions.
• Humanitarian aid: International organizations quickly assess disaster-stricken areas to optimize the distribution of relief supplies.

The disaster-detection project demonstrates the potential of AI to address global challenges. By combining multimodal remote sensing data, zero-shot segmentation models, and customized deep learning architectures, it provides a promising technical route for automated disaster assessment. With the enrichment of satellite data and advances in AI models, such systems will play a more important role in disaster response and contribute technical strength to protecting life and property safety.