Multi-Layer Adversarial Prompt Detection System: Protecting Large Language Models from Malicious Attacks

The Abinesh092 team proposes a multi-layer cascaded adversarial prompt detection system. Using a three-layer architecture of rule filtering, machine learning (TF-IDF + LightGBM), and semantic analysis (Sentence-BERT), it defends large language models against prompt injection and jailbreak attacks, addressing the limitations of single protection methods while balancing detection accuracy and real-time response.

With the widespread application of Large Language Models (LLMs) in production environments, prompt injection and jailbreak attacks have become serious security threats—attackers can bypass security restrictions to obtain harmful content or manipulate model behavior. Traditional single protection methods have flaws: rule-based approaches are easy to bypass, and pure machine learning solutions have incomplete training data coverage and high inference latency. How to balance detection accuracy and real-time response is a topic of industry concern.

The solution is a three-layer cascaded "gated pipeline" architecture: early layers quickly filter obviously harmless/harmful inputs, while boundary cases enter subsequent complex analysis layers. The first layer is rule-based filtering (predefined pattern matching + keyword detection), pursuing high throughput and low latency; the second layer uses machine learning (TF-IDF feature extraction + LightGBM classifier) to identify complex attacks that rules cannot cover; the third layer uses Sentence-BERT semantic analysis to calculate semantic similarity with known malicious prompts to detect rewritten/encoded samples.

In terms of feature engineering: the TF-IDF layer converts text into high-dimensional sparse vectors, LightGBM learns decision boundaries, and its inference speed is faster than deep neural networks; the Sentence-BERT layer generates dense sentence vectors, calculates semantic proximity via cosine similarity, and may be fine-tuned to adapt to adversarial prompt datasets; the three-layer gating mechanism: input is passed to the next layer only if the confidence of the previous layer is below the threshold, balancing efficiency and in-depth detection.

Although there is no publicly available detailed experimental data, performance can be inferred from the architecture: in terms of latency, most normal requests pass quickly through the first layer with an average response time in milliseconds; attack samples benefit from multi-layer collaboration to improve coverage. In terms of accuracy: the rule layer may have false positives, and subsequent layers perform secondary verification to reduce false alarms; the multi-layer architecture reduces the false negative risk of a single model—attackers need to bypass all three layers to succeed.

The system design considers production needs: the modular architecture allows adjustment of thresholds for each layer to balance security and user experience (e.g., relaxed rules for internal tools, strict policies for public services); it has strong interpretability, can track the input processing path and provide interception reasons, which is beneficial for security audits and user communication.

The current system faces challenges: adversarial attack methods are evolving, requiring continuous updates to the rule base and retraining of models; multi-language support and multi-modal input detection need to be explored. Future directions: collaboration between detection and generation models, real-time monitoring during inference, to achieve more comprehensive security protection.

This multi-layer system demonstrates a pragmatic security engineering approach: building a reliable protection system through layered collaboration and complementary advantages, which has important reference value for teams deploying LLMs.