Nebula Shield: An Automated Security Assessment Framework for Localized Large Language Models

Nebula Shield is an automated vulnerability assessment framework for locally deployed large language model (LLM) APIs, initiated by security researcher edgerunner85. It combines a Flask defense layer with the NVIDIA Garak scanner to provide prompt injection attack detection and input validation mechanisms, aiming to build a complete experimental environment for local LLM security assessment and systematically detect and evaluate vulnerability risks of locally deployed LLMs.

With the development of LLM technology, local deployment has become favored due to data privacy and reduced cloud dependency, but prompt injection attacks have emerged as a severe threat. Attackers can bypass security restrictions by constructing inputs to obtain sensitive information or induce unintended operations. The Nebula Shield project was developed to build an experimental environment for local LLM security assessment, combining a defense layer with automated scanning tools to systematically detect vulnerability risks.

Nebula Shield adopts a design combining layered defense and active testing, including three core components:

1. Defensive Application Layer: A Flask-based proxy service defensive_app.py that acts as a front-end gateway for LLM APIs, performing multi-layer security checks;
2. Target LLM Service: A local model deployed using the Ollama framework, providing an OpenAI-compatible API and supporting flexible model replacement;
3. Automated Scanning Engine: Integrates the NVIDIA Garak vulnerability scanning tool (v0.15.1), launching test attacks from a Kali Linux virtual machine.

The defense layer implements multiple security checks to form in-depth defense:

1. Input Length Anomaly Detection: Rejects inputs exceeding 4000 characters to prevent token flooding and hidden malicious injections;
2. Heuristic Signature Matching: Detects common prompt injection patterns (e.g., instruction overriding, role-playing, privilege escalation) via regular expressions; returns 403 and logs if a match is found;
3. Secure Forwarding Mechanism: Validated inputs are repackaged into standardized API requests and forwarded to the backend LLM, reducing the risk of raw input exposure.

Nebula Shield seamlessly integrates with NVIDIA Garak to enhance deep security assessment capabilities:

1. Scan Configuration: Automatically executed via run_scan.py, using the promptinject detector suite (containing hundreds of attack templates);
2. REST API Adaptation: Garak communicates with the defense endpoint via a REST generator, defining the target URI, HTTP method, and request template to inject attack payloads;
3. Isolated Test Environment: The scanner is deployed on an independent Kali Linux virtual machine to simulate real attack scenarios and ensure host stability.

Complete assessment workflow:

1. Launch the Ollama service to load the target model;
2. Launch defensive_app.py to listen on port 5000;
3. Execute run_scan.py in the Kali virtual machine, where Garak sends prompt injection attack payloads;
4. Garak generates a detailed report (including detection success rate, bypass cases, response analysis), and nebula_shield_report.html in the repository presents the standard format.

Nebula Shield applies to multiple scenarios:

• Security Researchers: Standardized LLM vulnerability assessment benchmark to compare model security performance or verify the effectiveness of defense technologies;
• Enterprise Developers: Demonstrates methods to add a security layer to production LLM applications; the defense layer code can be directly integrated into Flask applications;
• Model Developers: Identify security blind spots left from training via Garak scans to guide model alignment and fine-tuning.

Nebula Shield has limitations and improvement directions:

1. Heuristic signature matching cannot cover all prompt injection variants; semantic analysis needs to be introduced to detect deformed attacks;
2. The input length threshold is statically configured; it can be dynamically adjusted according to the model;
3. Currently focuses on prompt injection; detectors need to be expanded to cover other LLM security threats like data leakage and model theft.