Data-Driven Penetration Testing Framework: Cybersecurity Practice Integrating Machine Learning and Network Traffic Analysis

An end-to-end penetration testing framework integrating real network traffic capture, hybrid dataset construction, machine learning classification, and SHAP interpretability analysis, providing an intelligent analysis tool for cybersecurity defense.

• Original Author/Maintainer: meryem-ZRIOUIL
• Source Platform: GitHub
• Original Title: pfe-pentesting-framework
• Original Link: https://github.com/meryem-ZRIOUIL/pfe-pentesting-framework
• Publication Date: May 31, 2026

---

In today's environment where digital threats are increasingly complex, traditional penetration testing methods face dual challenges of efficiency and depth. Cybersecurity professionals need to process massive amounts of network traffic data to identify potential attack patterns and abnormal behaviors. However, manual analysis is not only time-consuming and labor-intensive but also prone to missing hidden attack signs.

The rise of machine learning has brought new possibilities to the field of cybersecurity. By training models to automatically identify malicious traffic patterns, security teams can significantly improve detection efficiency. However, the application of machine learning in cybersecurity also faces many challenges: the need for high-quality training data, lack of transparency in model decision-making processes, and how to translate research results into practical tools.

Against this background, this data-driven penetration testing framework project came into being. It attempts to build an end-to-end solution, starting from real network traffic capture, going through data engineering processing, and finally providing actionable intelligence to security analysts through machine learning models and interactive visualization interfaces.

---

The project adopts a multi-component collaborative architecture design, integrating mainstream tools in the cybersecurity field and machine learning technologies:

The project uses Kali Linux with the Metasploitable2 target machine to capture traffic in real attack scenarios. As a professional penetration testing distribution, Kali provides a rich set of attack tools; Metasploitable2 is a deliberately vulnerable virtual machine that provides a safe experimental environment for security research and testing.

To balance the authenticity and diversity of data, the project adopts a hybrid dataset strategy. The base layer uses the CICIDS2017 dataset, which is a standard dataset widely used in intrusion detection research and contains various common network attack types. On this basis, the project also integrates real traffic data captured by the team itself, making the training data more close to actual application scenarios.

The project implements three mainstream machine learning algorithms for traffic classification:

• Random Forest: An ensemble learning method that builds multiple decision trees and combines their prediction results, with good accuracy and anti-overfitting ability
• XGBoost: An efficient implementation of gradient-boosted decision trees, which has performed well in many machine learning competitions and is particularly suitable for processing tabular network traffic features
• Multi-Layer Perceptron (MLP): A feedforward neural network that can learn complex nonlinear relationships between features

In high-risk fields such as cybersecurity, model interpretability is crucial. Security analysts need to understand why a model classifies a certain traffic as malicious, not just a black-box prediction result. The project uses SHAP (SHapley Additive exPlanations) values to explain model decisions, assigning importance scores to each feature to help analysts understand which network features have the greatest impact on classification results.