PODS-AI: An AI-Based Programmatic Orca Recognition System

PODS-AI is a bimodal AI system integrating audio signal processing and computer vision. It aims to address the limitations of traditional monitoring methods against the backdrop of orca endangerment, enabling automatic detection and recognition of orcas, and providing an efficient technical tool for marine biodiversity conservation. The system includes two core modules: audio recognition and image recognition, using deep learning technology to help expand monitoring scope and improve data quality.

As top predators in the marine ecosystem, orcas face survival crises due to threats such as marine pollution, food shortages, ship noise, and climate change (e.g., the southern resident orcas in the waters of Washington State, USA, and British Columbia, Canada are already endangered). Traditional monitoring relies on manual visual observation or audio analysis, which is time-consuming and limited by manpower and weather conditions. The PODS-AI project uses artificial intelligence technology to achieve automatic monitoring, providing a key tool for orca conservation.

PODS-AI adopts a bimodal design, combining audio and image data to improve detection accuracy. The audio module uses the unique "dialects" of orcas, builds datasets through underwater microphone collection, preprocessing (filtering, time-frequency conversion), and expert annotation, and uses a deep learning model of CNN+RNN/GRU+attention mechanism. The image module is based on fastai and PyTorch, identifies individuals through features such as dorsal fin shape and saddle patch patterns, and uses transfer learning strategies (fine-tuning pre-trained ResNet/EfficientNet) to address challenges in shooting conditions.

The project relies on toolchains such as fastai, PyTorch, and librosa. The code structure is divided into ModelTraining (data preprocessing, training, evaluation) and PictureRecognition (image download, training, inference) modules. Performance evaluation covers multi-dimensional metrics: detection (precision, recall, F1 score), recognition (population classification accuracy, individual Top5 accuracy), and real-time performance (inference latency, throughput).

PODS-AI can be deployed in coastal real-time monitoring stations (24/7 monitoring, real-time alerts, population tracking); combined with drone/ship assistance (aerial image analysis, photo database matching); and used for historical data mining (long-term trend analysis, call library construction, cross-population comparison).

PODS-AI empowers ecological conservation: expanding monitoring scope, improving data quality, responding to threats in a timely manner, and supporting policy formulation. Future directions include expanding to multi-species recognition, integrating satellite data, edge computing deployment, and incorporating crowdsourced data.

PODS-AI demonstrates the application value of artificial intelligence in biodiversity conservation. It solves ecological monitoring challenges through bimodal technology, provides strong support for orca conservation, and offers a reference technical framework for wildlife monitoring, making it a typical case of AI for Good.