Prompt Security Engine: A Hybrid Prompt Attack Detection Framework Based on DistilBERT

Prompt Security Engine is an open-source hybrid machine learning framework designed to protect large language models (LLMs) from prompt injection attacks. Combining traditional machine learning techniques with the DistilBERT deep learning model, it can detect threats like jailbreak attacks, harmful requests, copyright infringement, and policy bypasses. It features explainable AI (XAI), drift detection capabilities, and supports FastAPI deployment and Docker containerization. The project is maintained by rahu7biju and was released on GitHub on June 2, 2026.

LLMs face various prompt injection threats: jailbreak attacks bypass security restrictions via techniques like role-playing; harmful requests directly demand violent/hateful content generation; copyright infringement induces the creation of protected materials; policy bypasses obtain restricted information through indirect questions. Among existing solutions, simple keyword filtering lacks semantic understanding, cloud-based security services cannot be deployed locally and have limited interpretability, while Prompt Security Engine provides solutions to these shortcomings.

The framework adopts a hybrid machine learning architecture: traditional ML models provide lightweight, interpretable, and fast filtering; the DistilBERT model uses strong semantic understanding to capture complex attack patterns. Compared to the original BERT, DistilBERT is 40% smaller in size, 60% faster in inference, retains 97% of language capabilities, and is suitable for production deployment. The model fusion strategy integrates outputs from both via ensemble learning to generate the final security score.

Core features include: 1. Explainable AI: Highlight text segments that trigger detection, provide confidence scores, and generate security reports; 2. Drift detection: Monitor changes in data distribution, detect new attacks, and trigger model updates; 3. FastAPI deployment: RESTful API interface, asynchronous processing, automatic documentation, and Docker containerization support, facilitating integration into existing systems.

Applicable scenarios: 1. Enterprise-level LLM deployment (filtering inputs for OpenAI/Azure or self-hosted models); 2. Content platform security (compliance for chatbots and customer service systems); 3. Red team testing (evaluating vulnerabilities in LLM applications); 4. Compliance auditing (generating logs to meet GDPR and AI Act requirements).

Prompt Security Engine is an important advancement in the field of LLM security, with its hybrid architecture balancing efficiency and interpretability. As LLM applications become more widespread, prompt attacks will grow more complex, and the framework can address emerging threats through continuous learning. This open-source framework provides enterprises and developers with a technical foundation for secure LLM deployment, helping to build trustworthy AI systems.