SafetyALFRED: A Safety-Aware Planning Evaluation Framework for Multimodal Large Language Models

SafetyALFRED is an open-source evaluation framework accepted by ACL 2026 Findings, developed by the SLED Lab at the University of Michigan. It extends the classic ALFRED benchmark to fill the gap where traditional benchmarks ignore safety constraints, providing a standardized evaluation platform for the planning capabilities of multimodal large language models (MLLMs) in safety-sensitive scenarios.

With the widespread application of MLLMs in embodied intelligence, robot control, and other fields, model safety evaluation has become increasingly important. Traditional benchmarks mainly focus on task completion rates but ignore the consideration of safety constraints during the planning process. Thus, the SafetyALFRED project was born to provide a standardized platform specifically for evaluating models' safety-aware capabilities.

SafetyALFRED extends the ALFRED benchmark and introduces rich safety-constrained scenarios (physical safety, social norms, privacy protection, environmental safety, etc.). Its core components include:

• dataset/: Multimodal task data with safety annotations (visual scenes, instructions, constraints)
• pddl_trajs/: Planning trajectories in PDDL format, facilitating comparison with classic methods
• scripts/: Data preprocessing and evaluation scripts that support automated testing.

The project adopts a comprehensive evaluation system with metrics including safety compliance rate, constraint understanding ability, trade-off decision quality, and error recovery ability. It supports comparison of multiple models, such as closed-source commercial models (GPT-4V, Claude), open-source models (LLaVA, Qwen-VL), and methods combining traditional planning with neural networks.

Academic Value: Provides standardized safety evaluation protocols, fine-grained error analysis tools, public datasets and code to promote community collaboration; Industrial Prospects: Helps product teams identify safety risks, regulatory agencies establish certification standards, and developers optimize model safety behaviors (applicable to smart home, service robot, and other fields).

Technically, it uses a modular architecture. The src/ directory contains a scene parser (visual-language to structured), a constraint checker (real-time compliance verification), and an evaluator (score calculation and report generation). Components provide APIs for easy integration. The community can contribute to expanding safety scenarios, multi-language support, visualization tools, and establishing safety leaderboards.

SafetyALFRED represents an important advancement in the field of AI safety evaluation, conveying the core concept that "a truly intelligent system must first be safe". It is an indispensable evaluation resource for teams developing or deploying multimodal AI applications.