LLM Secret Guard: A Localized LLM Sensitive Information Leakage Assessment Tool Based on the OWASP Framework

This article introduces LLM Secret Guard, a sensitive information leakage and defense assessment system for large language models (LLMs). It adheres to the OWASP LLM Application Security Framework, supports testing of multiple attack types and comparison of defense strategies, and specifically addresses the lack of targeted testing for locally deployed open-source models in existing tools.

With the widespread adoption of LLMs in enterprise and personal applications, sensitive information leakage has become a critical security risk. The 2023 OWASP Top 10 for LLM Applications lists sensitive information leakage as one of the top risks, along with prompt injection and system prompt leakage. However, existing security assessment tools mostly focus on cloud API models and lack a targeted testing framework for locally deployed open-source models. Researchers and developers need tools that can be repeatedly executed locally, support quantitative comparisons, and validate multiple defense strategies.

LLM Secret Guard is a localized security assessment tool based on the OWASP LLM Application Security Framework, specifically designed to test whether LLMs leak sensitive information under attack prompts. Its core design philosophy is to establish a repeatable, quantifiable, and comparable testing process to help researchers systematically evaluate the effectiveness of different models and defense strategies. The term "Secret Guard" in the name means information guardian; it identifies the model's vulnerabilities to malicious prompts through pre-set attack sets and scoring mechanisms, and is designed to target the generative nature and context understanding capabilities of LLMs.

LLM Secret Guard's core functions include:

1. Fixed Attack Set Testing: Built-in multiple standardized attack scripts to ensure consistent test input conditions and comparable results;
2. Leakage Level Determination Mechanism: Uses hierarchical assessment, scoring based on the sensitivity and completeness of sensitive information to more accurately reflect risks;
3. Valid Sample Filtering: Automatically identifies and filters valid samples containing sensitive information to reduce manual review;
4. Defense Score Calculation: Provides a standardized method for calculating defense scores to intuitively compare the defense effects of different models/configurations.

The tool currently supports testing of the following common attack types:

• Prompt Injection Attack: Tests the model's ability to resist prompt injection and prevent system instructions from being overwritten;
• Cross-Language Attack: Verifies the model's behavior when faced with unexpected language inputs;
• Role-Playing Attack: Tests whether the model overshares sensitive information in role-playing scenarios;
• System Prompt Leakage: Attempts to extract the model's system prompts to understand the model's behavioral boundaries and potential attack surfaces.

The main application scenarios of LLM Secret Guard include: model security assessment in academic research, security review before internal LLM deployment in enterprises, and effect verification during defense strategy development. Future plans include extending the testing scope to Web LLM Apps and Agent architectures, with the potential to develop into a more comprehensive LLM application security assessment solution.

The emergence of LLM Secret Guard reflects the trend in the LLM security field from cloud API security to local deployment and open-source model security, providing organizations that independently control model deployment with necessary tools for risk management. At the same time, the design adhering to the OWASP framework demonstrates the importance of security standardization, helping the industry form consensus and promote the progress of defense technologies.