AI Triage Agent: RAG-based Automated Vulnerability Ticket Processing System

Project Basic Information

• Original Author/Maintainer: adaka-security
• Source Platform: GitHub
• Project Repository: applied-ai-projects
• Original Title: AI Triage Agent
• Release Time: Created on June 15, 2026; updated on June 16, 2026

Core Content

This project is an AI agent system for security operation teams. By combining LLM (Claude API), Retrieval-Augmented Generation (RAG), and tool calling technology, it automates the classification, prioritization, and repair recommendation generation for vulnerability tickets. It addresses the pain point of time-consuming and error-prone manual evaluation of massive vulnerability alerts, improving security operation efficiency.

Common problems faced by security operation teams: Massive vulnerability alerts require manual evaluation one by one, which is not only time-consuming (occupying a lot of analysts' energy) but also prone to misjudgment or omission due to human factors. This project aims to alleviate this pain point by using AI automated processing to convert unstructured vulnerability reports into structured decision-making recommendations.

The project adopts a modular architecture, including four core components:

1. Data Ingestion Module (ingest_cves.py)：Retrieves CVE data from NVD API and stores it in a standardized way; supports replacement with data sources like enterprise internal ticket systems;
2. Vector Storage Module (vector_store.py)：Built on ChromaDB, stores vector embeddings of vulnerability descriptions, providing similar case retrieval capabilities for RAG;
3. Triage Agent (triage_agent.py)：The core brain, uses Claude API's tool calling feature to output structured evaluation results (priority P0-P3, reasoning process, repair recommendations, etc.);
4. Service Interface Module (api.py)：RESTful API built with FastAPI, exposing /triage and /rebuild-index endpoints for easy integration into existing security toolchains.

1. Structured Tool Calling

Using Claude's tool calling feature to force output of strictly typed evaluation results, including priority enumeration (P0-P3), reasoning process, recommended actions, and impact assessment. These results can be directly consumed by downstream systems without additional parsing.

2. Retrieval-Augmented Generation (RAG)

When processing new vulnerabilities:

• Convert vulnerability descriptions into vector embeddings;
• Query the most similar historical vulnerabilities in ChromaDB;
• Input historical cases as context into LLM to improve judgment accuracy and consistency.

3. Context-Aware Reasoning

Analyze vulnerability impact by combining enterprise asset inventory: For example, the same RCE vulnerability will have an elevated priority if it affects core business systems, and a reduced priority if it affects test environments, making the assessment more aligned with actual business scenarios.

Input Information:

• Vulnerability ID: CVE-2024-21413
• CVSS Score: 9.8
• Type: Network-based, no authentication required
• Affected Product: Microsoft Outlook

RAG Context: Retrieved historical similar high-risk Outlook RCE vulnerabilities and exploitation patterns

Output Results:

• Priority: P0 (highest level, need immediate handling)
• Impact Assessment: Remote Code Execution (RCE) may lead to ransomware, lateral movement, and credential theft
• Reasoning Basis: CVSS score 9.8 + network attack vector + no authentication required + matches known high-risk patterns
• Repair Recommendations: Apply Microsoft's February 2024 patch, push updates, temporarily disable preview pane, search for attack indicators, block outbound SMB/WebDAV.

Extensibility

The architecture can adapt to various workflows (IT support ticket routing, content moderation, alert triage, etc.). Adaptation steps only require: replacing data sources, adjusting priority modes, updating prompt templates—other components remain unchanged.

Tech Stack

• LLM: Anthropic Claude API (supports tool calling)
• Vector Database: ChromaDB
• Web Framework: FastAPI
• Machine Learning: scikit-learn (vectorization and similarity calculation) Dependencies are managed in a standardized way via requirements.txt and .env.example.

Future Extensions

• Slack/Email Alerts: Automatically notify designated channels for P0-level vulnerabilities;
• Consistency Evaluation: Compare the stability of agent judgments through multiple runs;
• Multi-agent Architecture: Introduce CMDB real-time query agents to replace static asset inventories.

Summary

This project is a practical application of LLM technology in enterprise security operation scenarios. It is not just a simple chatbot but an integrable production-level agent. For security teams: it reduces manual workload and allows focus on high-value tasks; for AI developers: it provides best practice references for RAG, tool calling, and structured output.