MCircKE: Mechanism Circuit-Based Knowledge Editing for Large Language Models

Large language models face challenges in knowledge update in a dynamic world. Existing knowledge editing methods have a "reasoning gap" (can recall edited facts but fail to apply them in multi-step reasoning). MCircKE achieves precise knowledge editing through the "mapping-adaptation" framework by identifying complete causal circuits related to target knowledge, effectively bridging the reasoning gap. It outperforms existing methods significantly in multi-hop reasoning tasks while minimizing interference with other model knowledge.

Demand for Knowledge Update in a Dynamic World

Pre-trained knowledge of large language models is static and cannot adapt to real-world changes (e.g., CEO replacements, scientific progress). Traditional retraining is costly, making knowledge editing a research hotspot.

Limitations of Existing Methods

Existing methods can fix isolated facts but have a "reasoning gap": For example, after updating Apple's CEO to John Smith, the model can answer direct questions correctly, but multi-hop questions (e.g., the CEO's university) still rely on old knowledge. This "can recall but not reason" issue restricts the practical application of knowledge editing.

Core Idea

MCircKE adopts the "mapping-adaptation" paradigm:

1. Mapping: Identify the complete causal circuit related to target knowledge (including fact storage locations and reasoning routing paths);
2. Adaptation: Update parameters only within the mapped circuit to achieve precise local editing.

Technical Implementation

• Causal Circuit Identification: Locate fact storage layers/neurons via causal intervention, analyze attention heads, track reasoning paths, and generate circuit masks;
• Precise Parameter Update: Modify only the parameters marked by the mask. Advantages include minimal interference, high interpretability, and low computational overhead. Constraint optimization is used to ensure accurate encoding of new knowledge.

Benchmark Test Results

On MQuAKE-3K (a multi-hop reasoning test benchmark), MCircKE performs as follows:

1. Single-hop question answering accuracy is comparable to the best baseline;
2. Multi-hop reasoning accuracy is significantly higher than other methods, effectively bridging the reasoning gap;
3. Minimal interference with unrelated knowledge, maintaining overall model stability.

Ablation Experiment Conclusions

• Circuit mapping is necessary: Methods that only edit fact storage have poor multi-hop performance;
• Value of precise update: Full parameter fine-tuning sometimes improves multi-hop performance but impairs other capabilities;
• Complete circuit mapping is more effective than partial components.

Real-Time Knowledge Update

Suitable for fast-changing fields such as news, finance, and healthcare, enabling continuous knowledge update while maintaining reasoning integrity.

Personalization and Privacy

Precise editing is suitable for fine-grained customization (e.g., user-specific knowledge) or injection of privacy-sensitive information.

Interpretability and Safety

Explicitly mapping knowledge to corresponding circuits enhances model interpretability, facilitating AI safety (e.g., detecting and eliminating harmful knowledge).

Current Limitations

1. High cost of circuit identification, leading to large overhead for large-scale updates;
2. Single circuit editing may be insufficient for complex reasoning chains involving multiple knowledge points;
3. Handling conflicts between new and existing knowledge requires further research.

Future Directions

1. Incremental editing strategy: Efficiently handle continuous knowledge updates;
2. Hierarchical circuits: Achieve multi-granularity (concept to instance) editing;
3. Cross-language transfer: Research knowledge transfer in multilingual models;
4. Integration with RAG: Flexible knowledge management.