Red Set ProtoCell: Open-Source Dual-Agent Red Team Testing Platform for Automatically Discovering Unknown Failure Modes of Large Language Models

Project Introduction

Red Set ProtoCell (RSP for short) is an open-source AI red team testing engine developed and maintained by Arnoldlarry15, released on GitHub on June 9, 2026. It uses a Sniper/Spotter dual-agent architecture, combining evolutionary algorithms and adaptive attack strategies to focus on proactively detecting unknown failure modes of large language models (LLMs), providing reproducible and auditable vulnerability discovery capabilities for AI security research.

Core Value

Unlike traditional static testing or manual red teaming, RSP can run autonomously 24/7, continuously discovering emerging unknown vulnerabilities through evolutionary strategies, helping organizations shift from passive compliance to proactive risk prevention.

Project Positioning

RSP is not a compliance tool or content filter; it is a proactive offensive AI security platform specifically designed to discover LLM failure modes.

Problems Solved

Traditional static testing suites only cover known issues, while manual red team testing is inefficient and unsustainable. RSP fills the gap in detecting unknown failure modes, discovering emerging risks through autonomous evolutionary strategies, and providing forward-looking security guarantees for AI deployments.

Dual-Agent Architecture

• Sniper Agent: Responsible for generating adversarial prompts, using 6 mutation strategies (vocabulary, encoding, structure, role-playing, context, obfuscation).
• Spotter Agent: Evaluates model responses through a three-layer scoring system (L1 Language Security Layer: 35%, L2 Security Exploitability Layer: 45%, L3 Cognitive Stability Layer: 20%).

Evolutionary Intelligence Process

1. Generation: Sniper constructs adversarial prompts
2. Execution: Send to target LLM API
3. Evaluation: Spotter quantifies failures
4. Evolution: Successful patterns guide the next generation of attacks

Fitness Function

Three-dimensional evaluation (effectiveness: 60%, consistency: 20%, novelty: 20%) drives strategy optimization.

Modern Web Interface

Provides real-time attack flow visualization, interactive dashboards, attack configuration, cost management, and custom input functions.

Multi-Platform API Support

Compatible with OpenAI (GPT series), Anthropic (Claude series), custom HTTP endpoints, and experimental local models.

Deployment Flexibility

Supports multiple deployment methods such as Firebase Hosting+Cloud Run, Docker Compose, Render/Vercel, etc.

Ethical Guardrails (EGG)

Prevents the generation of non-compliant content such as CSAM, bioweapon information, and exploitable attack code.

Strategy Locking and Reproducibility

Attack strategies are versioned and immutable, ensuring results are reproducible and auditable.

Execution Security

Default target isolation, limits on iteration count/token budget, and non-persistent storage of sensitive data.

Applicable Scenarios

1. Pre-release security assessment of models
2. Continuous monitoring of deployed models
3. Compliance verification (providing auditable evidence)
4. Adversarial research (exploring LLM security boundaries)
5. Enterprise red team capability building

Enterprise-Level Value

• Discover unknown failure modes and reduce AI deployment risks
• Shift from passive response to proactive prevention
• Provide defensible risk assessment results
• Support systematic vulnerability identification rather than single attacks

Project Significance

RSP represents a significant advancement in the field of AI security testing, realizing a mindset shift from static testing to evolutionary attack strategies, and providing a systematic risk quantification method for LLM security.

Open-Source Community and Future

The open-source nature promotes community collaboration to improve strategies. In the future, we will continue to develop multi-agent systems, knowledge systems, and autonomous workflows, laying the foundation for AI security research.