AI Cybersecurity Range: A Large Language Model Offensive and Defensive Practice Platform Based on OWASP Top 10

The AI Cybersecurity Range (AI-cyber-range) is an automated offensive and defensive practice platform for large language models (LLMs). Its core goal is to cover the OWASP LLM Top10 threat list, providing a safe and controllable practice environment for AI security research, talent development, and enterprise security assessment. This platform bridges theory and practice, helping security practitioners master LLM attack and defense skills and promoting the development of AI security capabilities.

As LLMs move from laboratories to production environments, security issues have become real threats (such as prompt injection, training data poisoning, model theft, etc.). The OWASP LLM Top10 released in 2023 provides a systematic risk framework for the industry, but the lack of practical experience restricts the improvement of defense capabilities. Traditional cybersecurity ranges simulate scenarios like network infrastructure, while the AI-cyber-range innovatively extends to the LLM ecosystem (including API interfaces, RAG systems, Agent frameworks, etc.), filling the gap in practical training for AI systems.

AI-cyber-range strictly aligns with the OWASP LLM Top10 threats, covering:

• LLM01: Prompt Injection (directly/indirectly overriding system prompts)
• LLM02: Unsafe Output Handling (unverified outputs leading to downstream risks)
• LLM03: Training Data Poisoning (manipulating data to implant backdoors/biases)
• LLM04: Model Denial of Service (resource-exhausting inputs)
• LLM05: Supply Chain Vulnerabilities (security risks from dependent components)
• LLM06: Sensitive Information Disclosure (privacy/confidentiality in training data)
• LLM07: Unsafe Plugin Design (insufficient plugin permissions/validation)
• LLM08: Excessive Agency (exceeding necessary permission capabilities)
• LLM09: Overreliance (lack of human review)
• LLM10: Model Theft (extracting parameter/architecture information)

The platform adopts a modular architecture with core components including:

1. Vulnerability Environment Pool: Preconfigured LLM scenarios with known vulnerabilities (e.g., customer service robots, code assistants) to realistically reproduce security weaknesses;
2. Attack Script Library: Contains standardized attack processes (e.g., prompt injection, multi-turn dialogue attacks) with step-by-step instructions and expected results;
3. Defense Toolbox: Integrates mainstream protection measures like input filtering and output review, supporting comparative testing;
4. Evaluation and Scoring System: Automatically records attack/defense effects and generates quantitative evaluation reports.

Core Drill Scenarios

• Jailbreak Challenge: Bypass safety guardrails to generate harmful content (testing skills like role-playing and coding bypass);
• Data Extraction: Extract sensitive information from training data via APIs (simulating memory/membership inference attacks);
• RAG Poisoning: Control external knowledge bases to spread misinformation (revealing supply chain risks);
• Agent Hijacking: Manipulate AI Agents to call external APIs for unauthorized operations;
• Model Reverse Engineering: Infer details like model architecture and training data distribution.

Automated Testing Capabilities

• Fuzz Testing Engine: Generate mutated inputs to detect abnormal behaviors;
• Adversarial Example Generation: Construct adversarial prompts that bypass security measures;
• Red Team Drills: Simulate multi-stage penetration by real attackers;
• Regression Testing: Re-run historical cases after model/protection updates to verify patch effectiveness.

Education and Research Value

• Security Training: Provide a safe and controllable environment for trainees to experience LLM attack techniques;
• Academic Research: Standardize evaluation conditions to promote fair comparison of protection schemes and benchmark establishment;
• Enterprise Security: Support red-blue team exercises, evaluate the security posture of LLM applications, and identify protection blind spots.

Compatibility with Existing Tools

• Integrate with OWASP ZAP and Burp Suite to cover non-AI attack surfaces;
• Connect to MLflow and Weights & Biases to track experiment configurations and results;
• Integrate with SIEM systems for unified security monitoring;
• Compatible with Kubernetes to support elastic scaling for large-scale concurrent testing.

Limitations and Usage Notes

• Attack techniques are only for authorized testing/education; actual attacks are strictly prohibited;
• Advanced scenarios require LLM background knowledge to fully understand;
• Automated tools may generate a large number of API calls; cost control is necessary;
• Content needs regular updates to keep up with the latest attack techniques.

Future Development Directions

• Expand to multi-modal models (image, audio, video security testing);
• Integrate automatic red team AI Agents to enable autonomous evolution of attack strategies;
• Build industry-specific scenarios for vertical fields like finance and healthcare;
• Develop a security certification system to issue qualification certificates to those who pass the assessment.