DisasterBench: A Multimodal Reasoning Benchmark and Lightweight Model for UAV Disaster Rescue

The research team released DisasterBench, the first multi-stage multimodal reasoning benchmark for disaster rescue, and introduced DisasterVL, a lightweight model with only 2B parameters that achieves reasoning capabilities close to GPT-4o on edge devices. DisasterBench covers 14 disaster scenarios, 9 key tasks, and 4 types of reasoning. DisasterVL achieves high efficiency through a three-stage optimization strategy, providing important support for AI systems in disaster rescue.

When a disaster occurs, rescue teams need to solve problems such as causal attribution, spread prediction, and decision-making reasoning, and must obtain results in real time with limited on-site computing resources. Existing multimodal benchmarks mostly focus on perception tasks, have limited coverage of disaster types, and lack systematic evaluation of multi-stage reasoning capabilities, making them difficult to meet the needs of actual emergency responses.

DisasterBench Benchmark Design

DisasterBench is the first multi-stage multimodal reasoning benchmark for UAV disaster rescue, covering 14 disaster scenarios, 9 key tasks (full stages of pre-disaster, during disaster, and post-disaster), and 4 types of reasoning.

DisasterVL Model Optimization

DisasterVL adopts three-stage optimization:

1. Domain instruction fine-tuning: Establish understanding of disaster scenarios and terminology;
2. Chain-of-thought guided multimodal alignment: Enhance information fusion and complex reasoning;
3. Reinforcement learning strategy optimization: Optimize for decision-making tasks.

In a comparison with 21 mainstream multimodal large language models, DisasterVL achieved:

• Ranked first among open-source models;
• Significantly narrowed the gap with top closed-source models like GPT-4o;
• With only 2B parameters, it can run in real time on edge devices without relying on network connections.

Technical Contributions

1. Fill the gap in the field: The first multi-stage multimodal reasoning benchmark for disaster rescue;
2. Provide a unified evaluation framework;
3. Open-source code and data.

Practical Value

1. Real-time decision support: Help commanders quickly understand the disaster situation and formulate plans;
2. Edge deployment capability: Can work in offline environments;
3. Multimodal fusion: Integrate UAV images, voice, text, and other information. Overall, this work marks the advancement of disaster AI from perception to multi-stage reasoning, laying the foundation for practical rescue AI systems.

Limitations

1. Limited coverage of disaster scenarios (14 types);
2. There is a gap between benchmark data and the complexity of real disasters;
3. The issue of ethical responsibility attribution for AI decisions.

Future Directions

• Expand more disaster types and geographic regions;
• Verify with real rescue data;
• Explore human-machine collaborative decision-making models;
• Develop more efficient edge reasoning architectures.