Mechanistic Validity: Establishing a Scientific Validation Framework for Neural Network Interpretability

This article introduces the Mechanistic Validity framework, a methodological system integrating philosophy of science, neuroscience, pharmacology, and measurement theory. It aims to address the core problem in Mechanistic Interpretability (MI) research: "how to verify that discoveries correspond to real mechanisms". The framework includes five-dimensional validation lenses, six-tier validation levels, a claim taxonomy, and an open-source ecosystem, providing a rigorous evaluation benchmark for MI research. It推动 the field from the "discovery" phase to the "validation" phase, which is of great significance for AI safety.

Mechanistic Interpretability focuses on identifying "circuits" (minimal computational units performing specific functions) in neural networks, using techniques like activation patching and ablation experiments. However, the field faces four major challenges:

1. Correlation ≠ causation: A neuron’s correlation with behavior does not imply a causal relationship;
2. Overfitted explanations: Explanations for specific inputs may fail on out-of-distribution data;
3. Vague description levels: Definitions and levels of "mechanism" are inconsistent across studies;
4. Questionable measurement reliability: Validation metrics themselves may have issues. The Mechanistic Validity framework is designed to address these challenges.

The framework integrates validation perspectives ("lenses") from five disciplines:

• Construct Lens (philosophy of science): Are claims falsifiable and well-defined? Clear definitions of "circuit" and "function" are needed, along with falsifiable experiments;
• Internal Lens (neuroscience): Is causal evidence sufficient? Both necessity (removing X leads to Y failure) and sufficiency (only X is enough to produce Y) need to be verified;
• External Lens (pharmacology): Can conclusions generalize? Mechanisms should be stable across different input distributions, model scales, and architectures;
• Measurement Lens (measurement theory): Are metrics reliably calibrated? Tools like Logit Lens and attention weights need reliability and validity tests;
• Explanatory Lens (MI itself): Are description levels clear and consistent? Consistency must be maintained across levels like neurons, attention heads, and modules.

Based on the five-dimensional framework, six validation levels are established:

Tier	Name	Meaning
Tier1	Proposed	Only structural alignment, no causal evidence
Tier2	Causally Suggestive	Necessity established (ablation degrades behavior)
Tier3	Mechanistically Supported	Necessity + sufficiency
Tier4	Triangulated	Convergence of multiple independent indicators
Tier5	Validated	Passes all five lens tests

Additionally, the framework provides six types of mechanistic claims: causal, structural, information-theoretic, behavioral, representational, and measurement-theoretic. Each type corresponds to different validation standards, avoiding one-size-fits-all evaluation.

The framework was applied to published MI studies, with the following results:

• High tier: IOI Circuit (Wang et al.2022), Othello World Model (Li et al.2023) reached Tier4 (triangulation);
• Mid tier: Induction Heads (Olsson et al.2022), Greater-Than (Hanna et al.2023), Copy Suppression (McDougall et al.2023) reached Tier3 (mechanistic support);
• Needs improvement: Grokking (Nanda et al.2023) Tier2 (causal suggestion), Knowledge Neurons (Dai et al.2022) Tier1 (proposed), Superposition (Elhage et al.2022) Tier1. These assessments help identify directions for further validation.

The Mechanistic Validity project consists of three modular code libraries:

1. mechanistic-validity: Core framework, including metrics, calibration tools, claim specifications, and documentation;
2. mechanistic-validity-lab: Infrastructure, providing experiment runners, result tracking, and cloud deployment (Modal/RunPod);
3. mechanistic-validity-experiments: Applied research, containing a collection of experiments using the framework. This separation allows different users (theoretical, experimental, and applied researchers) to choose appropriate entry points.

Mechanistic Validity marks the evolution of the MI field from "discovery" to "validation", with significant implications for AI safety:

1. Elevate research standards: Clear validation levels and multi-dimensional criteria reduce false discoveries;
2. Facilitate cross-study comparison: A unified framework enables comparison between different studies and identifies robust findings;
3. Guide future research: Point out the transition direction from Tier2 to Tier3;
4. Connect to academic traditions: Introduce mature disciplinary methodologies to avoid reinventing the wheel. Conclusion: This framework is an important milestone in the MI field. It emphasizes that understanding neural networks is not only a technical challenge but also a scientific methodology challenge, providing a rigorous guarantee for opening the AI black box.

The current framework is still in active development, with its main contribution being theoretical, and scripts serving as examples rather than production tools. Future directions:

• Develop an automated validation toolchain;
• Establish community-consensus calibration benchmarks;
• Extend to multimodal models and reinforcement learning agents;
• Integrate with other branches of alignment research (e.g., red teaming, scalable oversight).