aegisRT: A Practical Framework for Large Language Model Security Testing with Full Coverage of OWASP LLM Risks

This article introduces the open-source project aegisRT—a Python-native security testing framework for large language models (LLMs). Its design goal is to systematically cover the OWASP LLM Top10 security risks, providing practical tools and methodologies for AI system security audits. As LLM applications become widespread, they face new threats such as prompt injection, data leakage, and hallucinations. aegisRT helps organizations effectively manage LLM security risks through capabilities like modular architecture, adversarial sample generation, and response analysis.

LLM security threats are unique and different from traditional software security: prompt injection can manipulate model behavior, training data contamination may plant backdoors, model hallucinations generate false information, and supply chain risks involve links like pre-trained models. OWASP has released a Top10 risk list for LLMs, including 10 categories such as prompt injection, training data poisoning, and sensitive information leakage. aegisRT aims to cover these risk categories.

aegisRT adopts a Python-native modular design: test cases are hierarchically organized by risk category and support selective execution; it has built-in prompt construction techniques to generate adversarial samples (e.g., DAN prompts, delimiter bypass); it integrates lightweight response analysis components (content classification, sensitive detection); and it provides an extensible evaluation interface for custom rules. Core testing capabilities include prompt injection attack testing, sensitive information leakage detection, unsafe output handling verification, and hallucination and factuality assessment.

Best practices for using aegisRT: establish a security baseline and conduct regression testing; implement layered testing (unit, integration, adversarial); continuously monitor and update test suites; perform red team exercises to verify defenses. Technical implementation needs to balance performance and coverage, weigh false positives and false negatives, and maintain a model-agnostic design to adapt to different LLMs.

LLM security governance trends include the EU AI Act (strict risk management required for high-risk AI), NIST AI Risk Management Framework, and MLCommons security benchmarks. As an open-source tool, aegisRT evolves continuously through community collaboration, supporting contributions of new test cases and improvements to detection logic to respond to emerging threats.

aegisRT is an important step in the tooling of LLM security testing, covering the OWASP LLM Top10 risks and helping organizations manage LLM security risks. As LLM applications expand and regulations tighten, security testing frameworks have become standard components of AI governance infrastructure. Establishing systematic security testing capabilities is a necessary condition for responsible AI deployment.