NucBench: The First Multimodal Large Model Evaluation Benchmark for Nuclear Engineering

NucBench is the first open-source multimodal large language model evaluation benchmark designed specifically for nuclear engineering application scenarios, filling the gap in AI application evaluation in the nuclear energy field. Developed by the NS3G-UoS team, it aims to establish a comprehensive and authoritative evaluation framework to test the performance of models on nuclear engineering-related tasks, covering dimensions such as basic nuclear physics, technical document parsing, multimodal fusion, and safety decision-making, thereby promoting the safe and effective implementation of AI in the nuclear energy field.

With the widespread application of large language models (LLMs) in various industries, the nuclear energy field—a highly specialized area with extremely high safety requirements—has also explored the possibility of integrating AI. However, there is a lack of systematic evaluation standards to determine whether general AI models can understand complex concepts, technical specifications, and operational scenarios in nuclear engineering. The birth of NucBench fills this gap, becoming the first open-source multimodal large model evaluation benchmark for nuclear engineering scenarios.

Developed and open-sourced by the NS3G-UoS team, NucBench's core goal is to establish a comprehensive and authoritative evaluation framework to test the performance of multimodal large models on nuclear engineering-related tasks. It not only focuses on text comprehension capabilities but also emphasizes the comprehensive processing ability of images, charts, and technical documents in the nuclear engineering field, reflecting the application potential of multimodal AI in professional vertical domains.

NucBench's evaluation system covers several key dimensions:

• Basic nuclear physics concept understanding: Assessing the mastery of basic theories such as nuclear reactions, radiation protection, and reactor physics
• Technical document parsing: Testing the ability to read and understand nuclear engineering design specifications, operation manuals, and safety reports
• Multimodal information fusion: Examining the ability to conduct comprehensive analysis by combining text descriptions with engineering drawings and system schematics
• Safety decision support: Verifying the accuracy of reasoning and judgment in nuclear safety-related scenarios The task design fully considers the special characteristics of nuclear engineering—high professionalism, high risk, and strict regulation—to ensure that the results reflect practical application availability.

As an open-source project, NucBench provides standardized evaluation datasets, assessment scripts, and an extensible framework, making it convenient for the community to contribute more nuclear engineering-related evaluation scenarios. The open collaboration model helps:

1. Establish industry benchmarks, providing objective references for the nuclear industry to select and deploy AI solutions
2. Promote model improvement, helping developers identify weak links in the nuclear engineering field
3. Facilitate interdisciplinary communication, building a bridge between AI researchers and nuclear engineers.

NucBench is expected to play a key role in the digital transformation of nuclear energy:

• Intelligent operation and maintenance assistance: Assessing the potential of models in nuclear power plant operation data analysis and anomaly detection
• Training and knowledge management: Testing the feasibility of models as nuclear engineering knowledge bases and training assistants
• Safety supervision support: Exploring the application boundaries of AI in nuclear safety review and compliance checks In terms of challenges, the special characteristics of nuclear engineering lead to severe consequences of model hallucination, so NucBench pays special attention to the reliability and traceability of outputs.

NucBench represents the trend of AI evaluation deepening from general capabilities to professional vertical domains. As the capabilities of multimodal large models improve, similar domain-specific evaluation benchmarks will emerge in more high-risk and high-precision industries, promoting the safe and effective implementation of AI in fields that truly need it.