Practices for Machine Learning Reproducibility: A Research Methodology from 'Runnable' to 'Trustworthy'

The Scientific Computing Team at Aalto University has launched the 'Machine Learning Reproducibility Examples' open-source project. Addressing the reproducibility crisis in the machine learning field, it proposes a four-phase framework—planning, execution, review, and publication—emphasizing the integration of research integrity and engineering practices to help researchers develop reproducible research habits and enhance the credibility of their studies.

Beneath the prosperity of the machine learning field lies a reproducibility crisis: many paper experiment results are hard to reproduce, code fails to run, hyperparameters are missing, and preprocessing procedures are unclear—wasting resources and undermining research credibility. The Scientific Computing Team at Aalto University has launched an open-source project to address this issue and provide a complete research methodology.

The project proposes a four-phase framework:

1. Planning: Use model cards to record environment, code structure, data descriptions, etc.
2. Execution: Environment version control, modular code, reusable pipelines, experiment tracking.
3. Review: Code review, independent reproduction, document improvement, result verification.
4. Publication: Open sharing of code/data/models, preprint sharing, obtaining a DOI.

Key practical techniques for reproducibility include:

• Environment management: Virtual environments, dependency records, Docker containers;
• Code organization: Standardized style, centralized configuration, unit tests;
• Experiment recording: Random seeds, training logs, version control;
• Documentation: README files, code comments, run examples.

The project provides rich learning resources (reproducibility concepts, environment management, etc.) and practical cases (data processing, experiment tracking, etc.). Aalto University regularly holds workshops; the project is open-source and welcomes community contributions, supporting continuous updates and customization.

Future trends in reproducibility:

• Maturation of the tool ecosystem (MLflow, DVC, etc.);
• Journals and conferences requiring code and data submission, and setting reproducibility awards;
• Integration of reproducibility education into curricula to cultivate a rigorous attitude among the next generation of researchers.

This project conveys the attitude that 'scientific value lies in verifiability and extensibility'. In the era of rapid AI development, maintaining research rigor is crucial. It is recommended that all machine learning researchers study and practice these methods, respect others' and their own work, and promote scientific progress.