CoralReef: Multimodal Deep Learning Guards Marine Coral Ecosystems

CoralReef is a multimodal system integrating underwater image processing, fine-tuning of NOAA scientific models, XGBoost environmental modeling, and Groq large language model interpretation. It aims to address the pain points in coral bleaching monitoring, enable intelligent detection and natural language explanations, and empower marine conservation workers to efficiently assess coral health status.

Coral reefs support 25% of marine species but cover only 0.1% of the ocean area. Global warming has caused large-scale bleaching (three global events since 2014, with the Great Barrier Reef hit consecutively). Traditional manual diving surveys are time-consuming, labor-intensive, and costly; remote sensing technology has limited resolution and struggles to identify early signs. AI technology has become a key solution to this problem.

1. Image Preprocessing: Use OpenCV/scikit-image to address issues like blue-green tint and scattering in underwater images, generating standardized images;
2. Visual Classification: Fine-tune YOLO11n-cls based on NOAA pre-trained models, using 922 labeled images (437 healthy / 485 bleached) to train for identifying visual feature changes;
3. Environmental Modeling: XGBoost integrates 6 parameters including seawater temperature, DHW, SSTA, etc., and learns nonlinear relationships from data at 1252 stations;
4. Fusion and Interpretation: Weighted fusion of visual and environmental model results, and call the Groq API to generate natural language explanations.

• Backend: FastAPI asynchronous API, supporting two modes: image-only / full multimodal;
• Frontend: React+TypeScript following Stitch specifications, providing upload and result display interfaces;
• Model Service: Ultralytics for YOLO deployment, pickle for loading XGBoost weights;
• Data Sources: NOAA Coral Reef Watch indicators, NMFS-OSI pre-trained models, global coral bleaching database.

Applicable to regular monitoring of marine protected areas, distributed data collection for citizen science projects, spatiotemporal analysis for climate change research, and data support for policy formulation, helping conservation workers take timely measures.

• Multimodal fusion improves prediction robustness;
• Explainable AI transparently presents the decision-making process via LLM;
• Transfer learning uses NOAA models to reduce annotation costs;
• Integration of physical priors with oceanographic knowledge makes the model more scientific.

Limitations: Dependence on Groq API (offline access restricted), small training data scale (922 images), high threshold for obtaining environmental parameters; Future: Develop offline interpretation models, expand datasets to cover more coral species, integrate satellite remote sensing, and lower the threshold for mobile usage.