MathNet: The World's Largest Multilingual Mathematical Reasoning and Retrieval Benchmark Dataset Released

The MIT research team released the MathNet benchmark dataset, which is the world's largest multilingual mathematical reasoning and retrieval benchmark. It covers 30,676 Olympiad-level math problems from 47 countries in 17 languages, and for the first time systematically evaluates large models' mathematical retrieval capabilities, finding that retrieval quality significantly impacts reasoning performance. The release of this benchmark marks a new stage in mathematical AI evaluation.

Mathematical problem-solving is the gold standard for testing large language models' reasoning abilities, requiring strict logic, symbolic operations, and coherent cross-step thinking. However, existing mathematical benchmarks have limitations in scale, language coverage, and task diversity, making it difficult to fully evaluate models' performance in real-world scenarios.

The MathNet dataset has an impressive scale, covering Olympiad-level math problems from 47 countries in 17 languages over a 20-year period, with a total of 30,676 expert-written problems and detailed solutions. Its diversity is reflected in covering fields such as algebra, geometry, number theory, and combinatorics, and each problem's solution provides a reference for model training and evaluation.

MathNet designs three core tasks:

1. Problem Solving Task: Tests end-to-end reasoning ability. Cutting-edge models like Gemini-3.1-Pro achieve an accuracy of 78.4%, while GPT-5 reaches 69.3%;
2. Math-Aware Retrieval Task: For the first time systematically evaluates the ability to retrieve mathematically equivalent and structurally similar problems, where existing embedding models perform poorly;
3. Retrieval-Augmented Problem Solving: Explores the impact of retrieval quality on reasoning. DeepSeek-V3.2-Speciale improves performance by 12% through high-quality retrieval.

Experimental results show that even the most advanced reasoning models still have room for improvement on Olympiad-level problems (with a maximum accuracy of 78.4%). Meanwhile, retrieval augmentation significantly impacts mathematical reasoning performance; DeepSeek-V3.2-Speciale achieved a 12% performance improvement through high-quality retrieval, proving the importance of external knowledge bases.

The MathNet team has open-sourced the dataset and benchmark tools (URL: https://mathnet.mit.edu), providing a fair and comprehensive evaluation platform for academia and industry. For researchers, it offers multilingual resources; for educators, it can serve as the content foundation for intelligent education systems; for model developers, fine-grained evaluation helps identify strengths and weaknesses.

The release of MathNet represents the evolution of mathematical AI evaluation paradigms, expanding from single problem-solving to comprehensive evaluation of retrieval capabilities and retrieval-augmented reasoning. In the future, combining multimodal large language models with high-quality datasets like MathNet is expected to achieve greater breakthroughs in the field of automatic mathematical reasoning.