Activation Consistency Training: A New Defense Line to Protect Reasoning Models from Adaptive Attacks

The study proposes the Activation Consistency Training (ACT) method, which effectively defends against adversarial jailbreak attacks and prompt injection attacks by supervising the internal representations of large language models, with minimal impact on benign inputs. This research comes from the arXiv paper published in May 2026 titled 'Mitigating Adaptive Attacks against Reasoning Models with Activation Consistency Training'. Its core is to embed consistency constraints into the model's internal activation level, outperforming output-level consistency training (BCT) and having strong interpretability.

As the reasoning capabilities of Large Language Models (LLMs) improve, the chain of thought becomes longer and more complex, opening up new attack surfaces for adversarial attacks. The main threats include adversarial jailbreak attacks (bypassing safety alignment to generate harmful content) and prompt injection attacks. Traditional defenses target the final output, but attacks on reasoning models can occur at any stage of the chain of thought, which is a limitation.

Consistency training forces the model to behave consistently for clean prompts and adversarially modified prompts. The main variants are:

1. Output-level Consistency Training (BCT): Requires consistent final outputs but ignores internal reasoning differences;
2. Activation-level Consistency Training (ACT): Constrains the internal neuron activation patterns to be consistent, with advantages including needing only self-supervised data, directly supervising internal states, and strong interpretability.

After ACT training, the model has a linear shift in the activation space at the boundary of assistant turns, encoding the defense mechanism; a single 'steering direction' can be extracted to adjust activations to control the rejection of harmful requests; it is robust to pre-filling attacks—even if the chain of thought is replaced with a compliant trajectory from an undefended model, it can still identify the attack.

Evaluated on 5 reasoning models:

• ACT's defense against prompt injection attacks is competitive with other training methods;
• More robust than BCT in jailbreak attack scenarios;
• Outstanding resistance to adaptive attacks;
• Minimal impact on benign inputs, high deployment feasibility.

Activation Consistency Training (ACT) provides a new method based on internal representations for reasoning model security, effectively defending against adversarial attacks while maintaining interpretability and minimal impact on benign inputs; it proves that supervising internal representations is an effective and interpretable direction, which is of great significance for the safe deployment of reasoning models.

Current limitations: Specific experimental scenarios, training costs and performance impacts need further evaluation, and steering direction extraction can be optimized; future research can explore combining ACT with other defense mechanisms and verify its effectiveness on more models and tasks.