Representation-Guided Driving Mechanism: A Case Study Based on Rejection Behavior

This study addresses the black-box problem of the internal mechanism of representation-guided technology, using rejection behavior as a case study. Through a multi-token activation patching framework, it reveals that representation-guided vectors primarily interact with the attention mechanism via the OV circuit, enabling 90-99% sparsification while maintaining performance. The research provides mechanistic understanding and practical guidance for model alignment.

Representation guidance is an efficient technique in the field of large language model alignment. It can change model behavior (e.g., honesty, usefulness) by adding directional vectors to intermediate layer activations without fine-tuning. However, its black-box nature limits optimization and brings reliability risks and security hazards. Rejection behavior, due to its duality, clear neural basis, and alignment significance, becomes an ideal research object.

The research team extended traditional activation patching to multi-token scenarios. Core operations include: hierarchical intervention (applying vectors at different model depths), component decomposition (separating QK and OV circuits to test their impacts), and cross-model validation (repeating experiments on models like Llama/Qwen), systematically probing the impact of representation guidance on internal computations.

Experiments show that freezing the QK circuit (attention weights) only reduces performance by 8.75%, while intervening in the OV circuit (information aggregation) significantly affects the effect. This indicates that representation guidance works by changing "how to process the attended information" rather than "what to attend to". Additionally, different steering methods utilize functionally interchangeable circuits at the same layer.

Mathematical decomposition of the OV circuit reveals semantic components (e.g., safety considerations, user intent recognition). Maintaining the steering effect only requires retaining 1-10% of key dimensions, enabling 90-99% sparsification, which improves storage/computational efficiency and interpretability. Different methods show high consistency in identifying key dimensions.

The study emphasizes the engineering value of mechanistic understanding (e.g., sparsification). The circuit perspective provides a methodology for alignment research, and understanding component interactions paves the way for fine-grained control (e.g., targeted optimization of OV circuits).

Limitations: Focuses only on rejection behavior and relies on the activation patching method. Future work needs to verify mechanisms of other behaviors, combine methods like probe learning/causal abstraction, and explore optimal sparsification strategies.