Limited Impact of Valuation Awareness on Language Model Behavior: An Empirical Study Re-examining VEA Safety Risks

This study systematically assesses the real impact of "Verbalized Evaluation Awareness (VEA)" in the chain of thought of language models through on-policy and off-policy experiments. It finds that VEA has an extremely limited effect on model outputs, challenging the existing view that a high VEA rate equals strategic behavior or alignment tampering.

Large Reasoning Models (LRMs) explicitly express metacognitive behaviors of "being possibly evaluated" (VEA) in their chain of thought, raising concerns such as false safety performance, alignment tampering, and threats to evaluation validity. However, existing studies only focus on the incidence and correlation of VEA, lacking causal intervention experiments to verify whether VEA actually changes behavior.

On-Policy Analysis: Generate chain of thought naturally, compare answer distributions between spontaneous VEA and non-VEA samples (reflects real behavior but has selection bias); Off-Policy Analysis: Inject/remove VEA statements via pre-filling technology to actively manipulate variables for causal inference; Evaluation dimensions cover safety, alignment, moral reasoning, and political views.

1. VEA injection effect size ω ≤0.06 (negligible); 2. VEA removal effect size ω ≤0.12 (small effect); 3. Maximum effect size of spontaneous VEA ω ≤0.31 (small to medium, distribution shift up to 3.7%); 4. Results are consistent across different models and tasks.

1. Reassess VEA risks—VEA may just be a pattern in training data rather than strategic reasoning; 2. Distinguish between what models "say" and what they "do"; 3. Focus on more real risks like training data contamination and reward hacking; 4. Improve evaluation methods (multi-scenario testing, behavior measurement, long-term monitoring).

Conclusions: VEA's impact on model behavior is far lower than assumed, not constituting a major safety threat; Limitations: VEA detection relies on keywords, intervention methods are single, and proprietary models and high-risk scenarios are not covered; Future research needs more refined detection, expanded model scope, and studies on long-term effects.