Biomedical Data Synthesizer: A Reproducible Benchmarking Tool for Feature Selection in High-Dimensional Machine Learning

The open-source tool biomedical-data-generator introduced in this article is specifically designed for reproducible benchmarking of feature selection methods in high-dimensional machine learning scenarios. It aims to address the research dilemmas caused by the scarcity of real medical data and privacy constraints. This tool supports the generation of controllable and reproducible synthetic biomedical data, providing a fair testing platform for related research.

In the interdisciplinary field of machine learning and biomedicine, high-quality medical data is scarce and protected by privacy regulations. Data such as genomics often exhibits the characteristics of "high dimensionality and small sample size". This poses challenges to the development and validation of feature selection algorithms—real data is hard to obtain, simple random data cannot simulate the complex structure of biomedical data, and researchers urgently need controllable and reproducible data to fairly compare method performance.

biomedical-data-generator is a Python tool with core features including:

1. High-dimensional data simulation: Generate datasets with thousands to tens of thousands of features to simulate the dimensionality of high-throughput biological data, with precise control over truly relevant features and noise;
2. Configurable signal-to-noise ratio: Flexibly set the proportion of relevant features, effect sizes, and correlation structures between features to simulate different biomedical scenarios;
3. Reproducibility guarantee: Ensure precise reproducibility of datasets through fixed random seeds and clear parameter configurations.

This tool does not generate data randomly; instead, it simulates the statistical characteristics of real biological data:

• Correlation structure between features: Model through covariance matrices to generate features with specific correlation structures (e.g., gene co-expression modules);
• Class imbalance: Support custom sample proportions for each class to simulate scenarios like rare diseases;
• Nonlinear relationships: Generate data containing nonlinear and interaction effects;
• Noise injection: Simulate measurement errors and biological variations in real data through a multi-level noise model.

This tool is suitable for:

1. Feature selection algorithm developers: Provide a fair testing platform to systematically analyze the pros and cons of algorithms;
2. Bioinformatics researchers: Use synthetic data to explore methods and validate workflows before obtaining real data;
3. ML educators: Generate teaching cases to help students understand concepts like overfitting and the curse of dimensionality;
4. Privacy computing researchers: Use as a substitute for real data in the development and validation of technologies like federated learning and differential privacy.

This project helps address the reproducibility crisis:

• Transparency: All data generation parameters are public, allowing review and reproduction;
• Fair comparison: Provide a unified testing benchmark for different feature selection methods;
• Lowering barriers: Enable institutions with limited resources to conduct high-quality methodological research.

Although biomedical-data-generator is niche, it solves real problems. In the development of AI in healthcare, data bottlenecks restrict innovation, and this tool opens up new possibilities for algorithm research through high-quality synthetic data. For researchers in high-dimensional data analysis, feature selection, or bioinformatics, it is worth paying attention to and trying—it is not just a code repository, but also a practical implementation of the concepts of open science and reproducible research.