Prompt Injection Testing Framework for Reasoning Large Language Models

This article introduces an open-source testing framework developed by sysingleton, focusing on chain-of-thought prompt injection attack testing for reasoning large language models (LLMs). It helps developers evaluate the security performance of models under adversarial inputs. The framework is implemented in pure Python, includes core modules, and is suitable for security research, development testing, and educational scenarios. Note the ethical boundaries when using it.

With the rise of reasoning LLMs such as OpenAI's o-series and DeepSeek-R1, AI security has faced new dimensions. Reasoning models generate chain-of-thought to enhance answer quality, but they introduce unique risks: attackers can manipulate the internal reasoning process through prompt injection. These attacks are stealthy and difficult to detect by conventional filtering, as malicious instructions may not directly appear in the final output.

This open-source project, developed by sysingleton, is a prompt injection testing framework for reasoning LLMs, focusing on chain-of-thought characteristics. Core modules include: harness.py (testing engine), payloads.py (injection payload library), probe_model.py (model probing), analyze.py (result analysis), run_campaign.py (batch testing), and apps.py (example scenarios).

Framework workflow: 1. Load various injection payloads (command overriding, role-playing, semantic manipulation, etc.); 2. The probe_model module interacts with the target LLM to collect final outputs and chain-of-thought content; 3. The analyze module compares behavioral differences between normal and injected inputs and outputs a structured analysis report.

The framework is valuable for multiple user groups: security researchers can use it to systematically study attack impacts and develop defense mechanisms; developers can evaluate model vulnerabilities before deployment; educational scenarios can help students understand the principles of prompt injection.

Technical features: Modular design, components can be independently replaced or customized. Usage recommendations: Use only in authorized scenarios (own models, authorized environments, public datasets), comply with ethical and legal boundaries, and do not test on unauthorized third-party services.

Future expansion directions: Support injection testing for multi-turn dialogue scenarios, integrate automated defense strategy evaluation, expand to multimodal reasoning models, etc. Paying attention to such open-source projects helps improve AI security protection capabilities.