Nebula-Shield: Practical Security Assessment of Local LLM APIs — Offensive and Defensive Drills Based on Garak

This article focuses on the Nebula-Shield project, detailing the complete process of using NVIDIA's open-source Garak scanner to conduct security assessments on locally deployed Ollama+Flask LLM APIs. The assessment covers the detection and defense of key attack vectors such as prompt injection, data leakage, and harmful content generation, aiming to help organizations identify and mitigate security risks in local LLM deployments.

With the popularization of LLM technology, more and more organizations choose to deploy models locally to meet data privacy and compliance requirements. Tools like Ollama simplify the deployment process, but the security responsibility for local deployments falls entirely on the deployer, facing threats such as prompt injection, data leakage, and harmful content generation. The Nebula-Shield project presents a complete local LLM security assessment solution, using the Garak scanner to conduct comprehensive tests on Ollama+Flask APIs.

Target System: Ollama (local LLM runtime supporting models like Llama and Mistral, providing CLI and REST API) + Flask encapsulation layer (lightweight API gateway that may include logic such as authentication and logging, introducing new attack surfaces), deployed in a locally network-isolated environment.

Attack Platform: Kali Linux (professional penetration testing distribution, deployed as a virtual machine to isolate the attack environment) + Garak v0.15.1 (NVIDIA's open-source LLM vulnerability scanner with preset attack payloads and probes).

Design Philosophy: Systematic testing (testing attack vectors according to threat models), repeatability (standardized use cases), extensibility (support for custom probes).

Core Detection Modules:

• Prompt injection: direct injection (executing malicious commands), indirect injection (third-party content injection), jailbreak attacks (bypassing safety alignment);
• Data leakage: training data extraction, system prompt leakage, conversation history leakage;
• Harmful content: toxicity generation, dangerous behavior guidance, misinformation;
• Others: adversarial robustness, encoder attacks, context manipulation.

Scan Configuration: Specify target API endpoint, authentication method, model type, detection modules, generation parameters (temperature, maximum token count).

Scan Execution: Send test requests in parallel, collect responses (generated text + metadata), classify results using heuristic rules.

Result Analysis: Generate vulnerability reports (type, severity, reproduction steps), risk rating, and remediation recommendations.

Prompt injection vulnerabilities: Manifest as executing malicious commands; Defenses include input filtering, instruction isolation, output review, and least privilege.

Data leakage vulnerabilities: Manifest as outputting sensitive training data or system configurations; Defenses include data cleaning, differential privacy, output filtering, and access control.

Harmful content generation: Manifest as generating hate speech, dangerous guidance, etc.; Defenses include safety alignment (RLHF), input classification, output review, and rate limiting.

Architecture level: Network isolation, API gateway (unified authentication/rate limiting/logging), microservice splitting.

Application level: Input validation, context management (limiting history length), tool call control (strictly limiting callable tools).

Operation level: Log monitoring (anomaly detection), regular scanning (incorporating into CI/CD), emergency response plan.

The Nebula-Shield project demonstrates the complete process of local LLM security assessment. Security assessment should become a necessary part of the LLM application lifecycle, and tools like Garak promote security left-shift. Future trends include: automated security testing, adversarial training, standardized assessment (e.g., MLCommons AI Safety benchmarks), and red team serviceization. It is recommended that local LLM deployment teams incorporate security assessment into their standard processes and continuously harden their systems.