AI-Enhanced Intrusion Detection System: Using LLM to Convert 13GB of Network Traffic into Actionable Threat Intelligence

This article presents a four-layer hybrid intrusion detection pipeline combining Snort, XGBoost, and large language models, designed to address the alert fatigue issue of traditional IDS. The system converts 13GB of network traffic into precise threat intelligence, achieving 99% accuracy and zero-cost local deployment, effectively improving security operation efficiency.

In modern enterprise networks, IDS is the first line of defense, but traditional tools like Snort suffer from severe alert fatigue—over half of the alerts are false positives, and analysts waste a lot of time investigating false threats. This project targets this pain point and uses the CICIDS2017 dataset (13GB of real traffic, 692,703 labeled records) to build a solution.

Layer 1: Snort Full Capture

Responsible for monitoring all suspicious traffic, with zero missed alerts but a false positive rate of 54.9% (130,133 out of 237,240 alerts are noise).

Layer 2: XGBoost Intelligent Noise Reduction

Using a threshold-optimized binary classification model, it compresses alerts to 108,172, reduces false positive rate to 1%, increases accuracy to 99%, and maintains a recall rate of 100%.

Layer 3: GPT-4o mini Threat Briefing

Generates structured briefings (including threat type, severity, and disposal recommendations) at a cost of only $0.000603.

Layer 4: Locally Fine-Tuned LLaMA 3 8B

Trains a local model using LoRA technology; three metrics are on par with GPT-4o mini, enabling zero-cost local deployment and data privacy protection.

Dataset

Uses the CICIDS2017 dataset, which includes various attack types such as DoS Hulk (33.4%) and GoldenEye (1.5%).

Performance Comparison

Metric	Snort Alone	Full Pipeline
Alerts to Review	237,240	108,172
False Positive Rate	54.9%	1.0%
Accuracy	0.45	0.99
Recall	1.00	1.00
The experiment uses a three-layer matching strategy to solve NAT translation issues, ensuring data accuracy.

1. Hybrid architecture balances coverage and accuracy: Pure rule engines have high false positives, pure ML easily misses new attacks, and layered filtering balances both.
2. LLM bridges the gap between technology and decision-making: Converts raw logs into human-readable intelligence to support operational decisions.
3. Local models have production availability: LLaMA 3 8B is comparable to commercial APIs in specific tasks and meets privacy compliance requirements.

This project provides a practical solution for the cybersecurity field: Hybrid architecture is the current optimal choice, LLM has great potential in the threat intelligence field, and locally fine-tuned models can replace commercial APIs. As attack methods evolve, this AI-enhanced detection pipeline is expected to become a standard configuration for enterprise security infrastructure.