MiMo Reasoning Bench: A Comprehensive Evaluation Toolkit for Reasoning Ability Assessment

MiMo Reasoning Bench is an open-source comprehensive reasoning evaluation toolkit developed by the AXA3743 team. It is designed specifically for MiMo models and compatible with mainstream large language models (LLMs). Covering three core areas—mathematical reasoning, code generation, and logical reasoning—it provides a standardized evaluation process, multi-dimensional assessment metrics, and an extensible framework. It supports model development iteration, model selection comparison, academic research, and educational applications, serving as a key infrastructure for reasoning ability assessment.

With the rapid improvement of large language model (LLM) capabilities, scientifically and comprehensively evaluating their reasoning abilities has become a key challenge in AI research. Traditional evaluation benchmarks are limited to single task types and struggle to reflect real performance in complex scenarios. Especially for reasoning-intensive tasks such as mathematical proof, code generation, and logical deduction, existing tools lack sufficient coverage and depth. MiMo Reasoning Bench was created to fill this gap.

Evaluation System Architecture

• Mathematical Reasoning Module: Covers multi-level problems from basic arithmetic to advanced mathematics (algebraic equation solving, geometric proof, calculus operations, probability and statistics, etc.). Each problem is accompanied by a standard answer and detailed explanation, supporting automatic grading and error pattern analysis.
• Code Generation Module: Includes multi-language programming tasks (Python, JavaScript, C++, etc.), ranging from simple algorithm implementation to complex system design. Evaluation metrics include compilation success rate, unit test pass rate, code complexity, etc.
• Logical Reasoning Module: Contains question types such as deductive reasoning, inductive reasoning, and analogical reasoning, involving classic logic problems like Boolean logic, temporal reasoning, and causal inference. It can distinguish between models that truly understand logical rules and those that rely on surface pattern matching.

Technical Implementation Features

• Standardized Evaluation Process: Unified interface—users only need to provide a model reasoning function to run a complete evaluation. Built-in batch processing mechanism supports large-scale evaluation, with progress logs and intermediate result saving functions.
• Multi-dimensional Assessment Metrics: In addition to accuracy, innovative dimensions such as reasoning chain completeness, answer confidence, error type distribution, and time efficiency are introduced.
• Extensible Evaluation Framework: Modular design supports adding custom evaluation tasks. Standardized data formats and interfaces facilitate rapid integration of community datasets.

MiMo Reasoning Bench适用于以下场景：

1. Model Development Iteration: Provides fine-grained capability diagnosis for model training, helping developers identify weak points.
2. Model Selection Comparison: Offers an objective basis for enterprises and research institutions to compare model capabilities.
3. Academic Research: Provides a standardized experimental environment and benchmark data for papers related to reasoning abilities.
4. Educational Applications: Serves as an AI teaching aid to help students understand the capability boundaries of large language models.

Compared to single-domain evaluation benchmarks such as HumanEval (code domain) and GSM8K/MATH (math domain), MiMo Reasoning Bench's advantages lie in its comprehensiveness and consistency: the unified evaluation framework ensures fairness in cross-task comparisons, and fine-grained error analysis provides clear directions for model improvement.

The project team plans to add the following features in future versions:

• Multi-modal reasoning evaluation (combining text, image, and table data)
• Long-context reasoning ability testing
• Multi-language reasoning evaluation support
• Deep integration with mainstream training frameworks (e.g., Hugging Face, DeepSpeed)

MiMo Reasoning Bench provides a professional and comprehensive solution for reasoning ability evaluation, and is an open-source project worth attention for developers engaged in large language model research and applications. Through standardized evaluation processes and rich analysis dimensions, it is expected to become a key infrastructure in the field of reasoning ability research.