MoRFI: Using Sparse Autoencoders to Identify the 'Culprit' Behind Large Model Hallucinations

Core Idea: Large models are prone to hallucinations when fine-tuned on new knowledge, and the mechanism has long been unclear. The MoRFI method uses sparse autoencoders to analyze residual stream activations, identifies latent directions causally related to hallucinations, and can restore the model's knowledge retrieval ability through single-dimensional intervention, providing a new path to mitigate hallucinations.

The hallucination problem of Large Language Models (LLMs) is a core obstacle restricting their applications, but the neural mechanism remains unclear. LLMs acquire factual knowledge during pre-training, but post-training (such as SFT, RLHF) with new knowledge tends to trigger hallucinations. Existing studies show that SFT exacerbates hallucinations, but the mechanism is not clear. The MoRFI research aims to locate the internal representation changes of hallucinations, and explore reversibility and repair methods without retraining.

Experimental Design: Three models—Llama3.1 8B, Gemma2 9B, and Mistral7B v0.3—were selected and fine-tuned on 7 closed-book QA datasets. By controlling the proportion of new knowledge and training epochs, it was found that the hallucination rate increases with the proportion of new knowledge and the number of training epochs.

Tools and Methods: Use Sparse Autoencoders (SAE) to decompose residual stream activations into sparse features; MoRFI filters SAE features that change monotonically with the proportion of new knowledge, and identifies latent directions causally related to hallucinations.

Single latent variable intervention (adjusting the activation value of specific SAE features) on the latent directions identified by MoRFI can effectively restore the model's knowledge retrieval ability and reduce hallucinatory fabrications. This intervention is effective across different model architectures such as Llama, Gemma, and Mistral, showing universality.

1. Provides a mechanistic explanation for hallucinations: Reveals that abnormal activation of specific latent directions is the neural basis of hallucinations;
2. Demonstrates the potential of SAE in LLM interpretability, which can be extended to AI safety issues such as bias and toxicity;
3. Suggests lightweight model editing solutions: Local intervention on specific features is more efficient than retraining.

Limitations: The experiments focus on closed-book QA tasks, with a single form of hallucination; the identified features are correlational rather than strictly causal; there is a lack of systematic repair strategies.

Future Directions: Extend to tasks such as open-ended generation and multi-turn dialogue; strengthen causal inference verification; explore repair solutions such as joint adjustment of multiple features and dynamic intervention thresholds.