Meta Open-Sources Prompt-Siren: A Research Platform for LLM Prompt Injection Offense and Defense

Meta's latest open-source project, Prompt-Siren, is an experimental platform dedicated to researching prompt injection attacks and defenses for large language models (LLMs). This platform supports the AgentDojo and SWE-bench benchmarks, and features fine-grained state machine control, Hydra configuration management, and an extensible plugin architecture, providing AI security researchers with a systematic experimental sandbox.

With the widespread deployment of LLMs in various applications, prompt injection attacks have become one of the most pressing challenges in the AI security field. As a research-grade workbench, Prompt-Siren focuses on prompt injection as a specific attack vector, aiming to help researchers simulate attack scenarios and test defense mechanisms in a controlled environment, positioning itself as a "sandbox laboratory" for AI security research.

Prompt-Siren's core architecture includes the following features:

1. Fine-grained state machine control: Precisely tracks the decision-making process of AI agents and supports simulation of complex attack scenarios;
2. Multi-benchmark support: Natively integrates AgentDojo (AI agent security testing) and SWE-bench (real code editing task evaluation);
3. Hydra configuration management: Enables parameter scanning and complex experiment orchestration via YAML configurations;
4. Extensible plugin architecture: Allows customization of attack vectors, defense mechanisms, evaluation environments, and AI agent types.

Prompt-Siren supports two operating modes:

• Benign evaluation: Establishes a baseline for the normal task performance of AI agents, providing a reference for attack evaluation;
• Attack simulation testing: Injects prompt attack templates (built-in or custom) to observe model responses; Experimental result analysis uses the pass@k metric, which measures the probability of successfully completing a task at least once in k attempts, better reflecting reliability in adversarial environments.

To install Prompt-Siren, the following requirements must be met:

• Python 3.10+;
• Linux/macOS (Windows is not supported temporarily);
• Docker environment (for SWE-bench integration and sandbox isolation);
• Valid LLM API keys (supports multiple providers). The platform uses a modular design, allowing users to choose to install components such as core functions, benchmark support, and Docker sandbox. Using the uv package manager is recommended.

The open-source release of Prompt-Siren is of great significance for AI security research:

1. Establishes a standardized evaluation benchmark for prompt injection defense solutions;
2. Reduces the cost of experimental setup and accelerates research iteration;
3. The open-source architecture promotes community sharing of attack patterns and defense strategies;
4. Helps developers understand the potential security risks of LLM applications.

With the development of multimodal models and embodied intelligence, the attack surface of prompt injection will further expand. Prompt-Siren's extensible architecture reserves space to address emerging threats, and the community expects more attack simulations and defense mechanisms for specific scenarios to be validated on this platform. For AI security enthusiasts, it is an important entry point to participate in building a more secure AI ecosystem.