AI-Powered SIEM Security Operations Center: Practical Construction from Passive Response to Active Defense

This project is a practical AI-integrated SIEM system based on the Random Forest algorithm, demonstrating how to build a home SOC lab with early file anomaly detection capabilities and achieve a complete security operation closed loop from log collection, behavior analysis to automatic isolation. The project is maintained by Willem476 and open-sourced on GitHub (link: https://github.com/Willem476/AI-integrated-SIEM-system-for-early-file-abnormally-detection).

Traditional SOC faces three major problems: alarm fatigue, system fragmentation, and static rule lag. Signature-based detection is ineffective against zero-day vulnerabilities and new ransomware. This project proposes integrating AI into SIEM, using machine learning models to learn normal file behavior patterns and warn at the early stage of malicious behavior, realizing the transformation from post-event accountability to pre-event prevention.

The project is deployed in a virtualized environment, with core tech stack including: virtualization platform (VMware ESXi/Workstation/Proxmox), terminal system (Windows Server 2022), log management (ELK Stack), automation orchestration (Shuffle SOAR), threat intelligence and case management (TheHive + Cortex), network isolation (pfSense), and AI engine (Python + Random Forest). All components collaborate via APIs, embodying the concepts of integration, automation, and intelligence.

The intelligent core of the system is a Random Forest-based model, trained on 185,845 records (half benign and half malicious). Unlike hash comparison, the model identifies anomalies by analyzing file behavior features, extracting 27 features in three categories: 1. Extension features (5 dimensions: script type, executable file, double extension, etc.); 2. Path and size features (15 dimensions: directory credibility, file size anomaly, etc.); 3. Pattern and trust features (7 dimensions: whitelisted location activity, suspicious pattern combinations, etc.).

After AI detects an anomaly, the system calculates a dynamic risk score from 0 to 100 using the formula: 50% × ML probability + 30% × MITRE ATT&CK score + 20% × IOC score. When the score exceeds the threshold (>20 points), it triggers responses according to LOW/MEDIUM/HIGH/CRITICAL levels, ensuring the score is accurate and interpretable.

The project implements a fully automatic response loop: 1. Anomaly detection; 2. Risk calculation; 3. SOAR execution (Shuffle); 4. Network isolation (pfSense API); 5. Case creation (Cortex/OpenCTI + TheHive). This process reduces MTTR from hours to minutes/seconds, providing a significant advantage in dealing with fast-spreading threats.

The project provides a guide for security practitioners and students to build an enterprise-level SOC from scratch, covering: AI integration module (feature engineering, model training), system architecture module (installation and configuration of ELK/TheHive/Cortex/Shuffle), and SOAR playbook module (workflow JSON and API examples). Readers can build a home SOC lab to understand the modern security operation tech stack and processes.

This project demonstrates the practical value of AI in cybersecurity and is a deployable complete solution. Its open-source nature allows the community to contribute new models, playbooks, and integration schemes. As threats become more complex, integrating AI into security operations is an inevitable trend, and this project is a high-quality resource to enhance security operation capabilities.