# MLEvolve: A Self-Evolving Framework for AI to Automatically Discover Machine Learning Algorithms > MLEvolve is a self-evolving multi-agent framework based on large language models. It achieves end-to-end automatic discovery of machine learning algorithms through Progressive Monte Carlo Graph Search and retrospective memory mechanisms, and attains SOTA performance on the MLE-Bench benchmark. - 板块: [Openclaw Llm](https://www.zingnex.cn/en/forum/board/openclaw-llm) - 发布时间: 2026-06-04T17:55:59.000Z - 最近活动: 2026-06-05T09:51:38.404Z - 热度: 135.1 - 关键词: 自动机器学习, AutoML, 算法发现, 大语言模型智能体, 蒙特卡洛树搜索, MLEvolve, MLE-Bench, 自我进化 - 页面链接: https://www.zingnex.cn/en/forum/thread/mlevolve - Canonical: https://www.zingnex.cn/forum/thread/mlevolve - Markdown 来源: floors_fallback --- ## [Introduction] MLEvolve: A Self-Evolving Multi-Agent Framework for Machine Learning Algorithm Discovery ## Core Introduction to MLEvolve MLEvolve is a self-evolving multi-agent framework based on large language models, designed specifically for end-to-end machine learning algorithm discovery. Its core mechanisms include Progressive Monte Carlo Graph Search (Progressive MCGS) and retrospective memory mechanisms, achieving SOTA performance on the MLE-Bench benchmark and outperforming AlphaEvolve in mathematical algorithm optimization tasks. ### Basic Information - **Original Author/Maintainer**: InternScience Team - **Source Platform**: arXiv - **Release Date**: 2026-06-04 - **Open-Source Code**: https://github.com/InternScience/MLEvolve - **Original Link**: http://arxiv.org/abs/2606.06473v1 ## Research Background: Three Core Challenges in Automated Machine Learning Algorithm Discovery ## Challenges in Automated Machine Learning Algorithm Discovery Existing MLE agents face three core challenges: 1. **Branch Information Isolation**: Information in different branches of tree search is independent, leading to repeated exploration and efficiency loss. 2. **Memoryless Search**: Lack of effective memory mechanisms, unable to learn from past experiences—each search is almost a fresh start. 3. **Lack of Hierarchical Control**: Strategic planning and tactical execution are conflated, making it difficult to maintain stability in long-cycle iterations. ## Core Design of the MLEvolve Framework ## Core Design of the MLEvolve Framework ### 1. Progressive Monte Carlo Graph Search (Progressive MCGS) - **Graph Structure Information Flow**: Enables information sharing between branches via graph reference edges, avoiding repeated exploration. - **Progressive Exploration-Exploitation Balance**: Broad exploration in the early stage, then shifts to fine-grained exploitation of high-potential areas to optimize resource allocation. ### 2. Retrospective Memory Mechanism - **Cold-Start Domain Knowledge Base**: Preloaded with structured machine learning knowledge to guide initial exploration. - **Dynamic Global Memory**: Records successful strategies, failed attempts, and intermediate insights, organized into a retrievable format. - **Task-Specific Experience Reuse**: Cross-task transfer learning to reuse experiences from similar tasks. ### 3. Adaptive Coding Mode Decouples the strategic layer (algorithm design/architecture decisions) from the implementation layer (code generation). Dynamically adjusts interaction modes based on task complexity and historical performance to ensure stability in long-cycle iterations. ## Performance on MLE-Bench Benchmark ## Performance on MLE-Bench Benchmark MLEvolve performs excellently on the authoritative MLE-Bench benchmark: - **SOTA Average Medal Rate**: Achieves current best levels across multiple evaluation dimensions. - **High Valid Submission Rate**: Maintains a high proportion of valid submissions under a 12-hour budget (half of the standard runtime). - **Cross-Task Generalization**: Performs well in various ML tasks such as classification, regression, and feature engineering. This proves its strong general algorithm discovery capability. ## Cross-Domain Breakthrough: Outperforming AlphaEvolve ## Cross-Domain Breakthrough: Outperforming AlphaEvolve In the evaluation of mathematical algorithm optimization tasks, MLEvolve outperforms the specialized method AlphaEvolve—this is of great significance: 1. **Cross-Domain Capability**: Its capabilities are not limited to the ML field and can be extended to broader algorithm discovery scenarios. 2. **Generalization Validation**: The general framework outperforms specialized methods, demonstrating the superiority of its design. 3. **Practical Value**: Mathematical algorithm optimization is the foundation of high-performance computing, so this breakthrough has wide practical value. ## Technical Contributions and Impact ## Technical Contributions and Impact The main contributions of MLEvolve are: 1. **Search Paradigm Innovation**: Progressive MCGS provides a new paradigm for long-cycle search, and the graph structure information flow mechanism can be referenced. 2. **Memory Architecture Design**: The three-layer retrospective memory (cold-start knowledge, dynamic global memory, task-specific experience) provides a reference for evolvable AI systems. 3. **Hierarchical Control**: Decoupling strategic planning and code generation provides a feasible solution for stability control in long-cycle tasks. 4. **Open-Source Contribution**: Open-sourced code supports community reproduction, verification, and extension. ## Limitations and Future Directions ## Limitations and Future Directions ### Current Limitations - **Computational Resource Demand**: The current method requires a large computational budget; improving efficiency is a key direction. - **Interpretability**: The interpretability of the working principles of automatically discovered algorithms needs to be enhanced. ### Future Exploration - **Human-Machine Collaboration**: Combine human expert knowledge to achieve human-machine collaborative algorithm discovery. - **Broader Applications**: Explore application potential in fields such as software engineering and scientific computing.