EST-Bench: A Safety Evaluation Benchmark for Large Language Models in Extreme Survival Scenarios

EST-Bench is an open-source deterministic evaluation framework specifically designed to test the safety, policy compliance, and tactical reasoning capabilities of large language models (LLMs) in extreme survival scenarios such as harsh conditions, power outages, and resource scarcity. It fills the gap in traditional safety evaluations regarding the assessment of decision-making capabilities in extreme environments, providing researchers and developers with standardized tools.

With the widespread deployment of large language models (LLMs) in practical applications, model safety and reliability have become critical issues. Traditional safety evaluations mostly focus on regular scenarios such as content moderation and bias detection, but lack systematic assessment of decision-making capabilities in extreme environments. The EST-Bench project was born to fill this gap.

EST-Bench (Extreme Survival Test Benchmark), developed by the AryanGold team, is an open-source deterministic evaluation framework that tests the performance of large language models in harsh, power-outage, and resource-scarce survival scenarios. Its goal is to provide researchers and developers with standardized tools to evaluate models' safety, policy compliance, and tactical reasoning capabilities under high-pressure environments.

Deterministic Evaluation

Unlike traditional non-deterministic evaluations, EST-Bench emphasizes 'deterministic' evaluation—models should produce predictable and reproducible outputs under the same input conditions, which is crucial for safety-critical applications.

Coverage of Extreme Scenarios

It focuses on survival scenarios with resource scarcity and infrastructure breakdown, requiring models to make rational decisions under conditions of incomplete information, time constraints, and limited resources.

Multi-Dimensional Evaluation Metrics

Evaluation is conducted from three core dimensions:

1. Safety: Whether the model outputs harmful, dangerous, or unethical suggestions
2. Policy Compliance: Whether the model adheres to predefined behavioral guidelines and safety policies
3. Tactical Reasoning Capability: Whether the model can perform logical reasoning and formulate effective strategies in complex situations

EST-Bench adopts a modular design, with core components including:

• Scenario Generator: Generates diverse survival scenarios based on predefined templates
• Evaluation Engine: Executes model interactions and records responses
• Scoring System: Quantitatively scores model outputs based on preset standards
• Report Generator: Outputs detailed evaluation reports and analysis results

Safety Research

Provides AI safety researchers with a standardized experimental platform to systematically study model behavior patterns under extreme pressure and identify potential safety vulnerabilities.

Model Development

Model developers can use it for regression testing to ensure that the safety of new versions does not degrade and to identify weak points that need improvement.

Enterprise Deployment

Before enterprises deploy LLMs to critical businesses, they can understand the model's performance under abnormal working conditions through pre-evaluation, providing data support for risk management and control.

EST-Bench is an open-source project with a permissive license that allows free use, modification, and extension. The community can contribute new test scenarios, improve evaluation metrics, or develop dedicated evaluation suites for different domains.

EST-Bench represents an important direction for LLM safety evaluation to extend from regular scenarios to extreme scenarios. As AI is implemented in key domains, boundary condition evaluation will become more important. This framework not only provides a stress testing tool for current models but also serves as a reference benchmark for the design of more robust and safer AI systems in the future. Researchers and practitioners concerned with AI safety should pay attention to this project to better understand the behavioral boundaries of models under extreme conditions and build more reliable AI systems.