OSS-Threat-Data: ML-Powered Open Source Supply Chain Threat Detection

This project aims to automatically detect and classify open source software (OSS) supply chain threats using machine learning. Key components include:

• Annotated dataset (data/oss_threat_dataset.csv)
• Python scripts for evaluation (scripts/evaluate.py, scripts/evaluate_with_predictions.py)
• Automated evaluation via GitHub Actions workflow

Source Info:

• Author/Maintainer: Mdniloykhan
• Platform: GitHub
• Original Link: https://github.com/Mdniloykhan/oss-threat-data
• Release Date: 2026-06-03

It addresses the gap in traditional security tools by proactively identifying suspicious patterns instead of relying on known signatures.

The Urgency of OSS Supply Chain Security

OSS is the backbone of modern digital infrastructure (e.g., Linux, React, TensorFlow). However, supply chain attacks are increasing in frequency and impact—examples include SolarWinds, Log4j, XZ Utils backdoor, and malicious npm packages.

Traditional tools like vulnerability scans and dependency checks are reactive (only detect known threats). This project explores proactive detection of suspicious behaviors in the OSS ecosystem.

Project Method & Core Components

Core Components

1. Annotated Dataset: data/oss_threat_dataset.csv (for model training/evaluation)
2. Python Scripts:
   • evaluate.py: Shows label statistics
   • evaluate_with_predictions.py: Checks prediction accuracy
3. Automation: GitHub Actions workflow (.github/workflows/evaluate.yml) runs evaluation on code changes, generating evaluation_report.md.

ML's Role

Unlike traditional signature-based tools, ML learns normal vs abnormal patterns to detect unknown threats. Possible features include:

• Code: Complexity, sensitive API calls, obfuscation
• Behavior: Abnormal release frequency, version jumps
• Metadata: Maintainer activity, commit patterns
• Network: Package name similarity (typosquatting), download anomalies

Covered Threat Types & Tool Complementarity

Threat Classification

The dataset covers 5 key supply chain threats:

1. Malicious code injection (backdoors, data theft)
2. Dependency confusion (public vs private package name conflicts)
3. Version poisoning (trusted maintainers adding malware later)
4. Build system attacks (contaminating CI/CD pipelines)
5. Metadata manipulation (fake download counts/ratings)

Comparison with Other Tools

Tool Type	Representative Products	Detection Method	OSS-Threat-Data's Position
Vulnerability Scan	Snyk, Dependabot	Known CVE matching	Complement: detects unknown threats
Static Analysis	SonarQube, CodeQL	Code rule matching	Complement: ML-driven anomaly detection
Dependency Check	OWASP Dependency-Check	Signature/hash comparison	Complement: behavior pattern analysis
Threat Intelligence	GitHub Advisory Database	Manual curation	Complement: automated pattern learning

It does not replace existing tools but adds an ML layer to catch missed patterns.

Limitations & Key Challenges

The project faces several challenges:

1. Data Scarcity: Few supply chain attack events lead to class imbalance, affecting model performance.
2. Adversarial Attacks: Attackers may design bypasses if they know the model's logic.
3. High False Positive Cost: Mislabeling legitimate projects harms reputation/operations.
4. Dynamic Threats: Attack methods evolve, requiring continuous model updates.

Real-World Applications & Future Directions

Applications

• Enterprise Teams: Risk assessment before adopting new OSS dependencies.
• Package Platforms: Real-time scanning of new packages (npm, PyPI, Maven).
• Security Research: Benchmark dataset for supply chain security models.
• OSS Maintainers: Monitor dependency trees for risk propagation.

Future Directions

1. Multi-modal feature fusion (code, commit history, community data)
2. Graph Neural Networks (model dependency graphs)
3. Temporal modeling (detect evolving threats over time)
4. Federated learning (aggregate intelligence without sharing sensitive data)

Summary & Significance

OSS-Threat-Data offers a novel approach to supply chain security—using ML to learn threat patterns instead of relying on known signatures. While still in early stages, it addresses a critical need as OSS becomes more integral to key infrastructure.

This open source project provides a collaborative platform for the community to enhance OSS ecosystem resilience. Security practitioners, maintainers, and developers are encouraged to engage with the project.