FragileML: A Deterministic Agent Training Environment for Machine Learning Debugging Workflows

FragileML is a lightweight, fully deterministic environment designed specifically for training and evaluating agents capable of handling real-world machine learning debugging workflows, with a particular focus on modeling common failure scenarios in Hugging Face pipelines. It addresses the problem of oversimplification in existing training environments, providing a reliable training foundation for AI to automatically debug ML pipelines.

Debugging machine learning pipelines involves multiple stages such as data preprocessing, model configuration, training execution, and result validation, where various errors can easily occur. Common failures on the Hugging Face platform provide research materials, but existing training environments are too simplified to reflect the complexity of production environments. FragileML aims to create a lightweight yet fully functional deterministic environment to support the training and evaluation of agents' debugging capabilities.

FragileML follows three core design principles:

1. Full determinism (predictable behavior under the same initial state and input, ensuring experimental reproducibility);
2. Real-scenario modeling (abstracting common Hugging Face failures such as configuration errors, dependency conflicts, data format issues, etc.);
3. Lightweight architecture (lowering the barrier to use, facilitating participation from more researchers).

FragileML includes core modules:

• Environmental state management (maintaining pipeline configurations, dependencies, and execution states);
• Action space (agents can perform operations such as modifying configurations, installing dependencies, adjusting parameters, etc.);
• Multi-dimensional reward mechanism (evaluating repair success, efficiency, and whether new issues are introduced);
• Observation interface (supporting integration of agent architectures like rule-based systems, reinforcement learning, and large language models).

In academia, FragileML provides a standardized benchmark platform to facilitate comparison of results across different teams; in industry, trained agents can be integrated into CI/CD workflows to enable automated fault detection and repair. Additionally, its scenario library and data help understand the fragility of ML systems and drive improvements in upstream tools.

In the future, we can expect FragileML to integrate more real-world scenarios, support multi-agent collaboration, and deeply integrate with mainstream ML platforms. Developers can contribute to the development of automated ML engineering by improving the environment or training agents.