ActLCD: Active Layer-Contrastive Decoding Significantly Reduces Hallucination in Large Language Models

A research team from Purdue University and the University of California, Davis proposed ActLCD (Active Layer-Contrastive Decoding), a new decoding method that dynamically activates layer contrast mechanisms using reinforcement learning strategies. This method outperforms existing SOTA methods across five benchmark tests including TruthfulQA, LongFact, and StrategyQA, with a maximum improvement of 19.81%, and has been accepted by the main conference of EMNLP 2025.

Large Language Models (LLMs) tend to produce "hallucinations" when generating text—outputs that seem reasonable but are incorrect—which restricts their application in critical tasks. Most existing decoding methods operate at the token level and struggle to handle cumulative errors in long contexts; some layer contrast methods may cause a "misunderstanding snowball effect" by forcing early interpretation of long sentences, amplifying early biases.

ActLCD models decoding as a Markov Decision Process (MDP), with core mechanisms including: 1. Dynamic layer contrast activation: Determine whether to activate layer contrast and which layers to compare based on the current context to avoid over-intervention; 2. Reward-aware classifier: A lightweight policy network predicts the expected return of applying layer contrast based on hidden states to achieve global optimization.

ActLCD performed excellently in five benchmark tests:

• TruthfulQA: 19.81% improvement in factuality
• LongFact F1@128: 3.30% improvement in long-text factuality
• StrategyQA: 7.51% improvement in reasoning ability
• GSM8K:7.21% improvement in mathematical reasoning
• Package Hallucination:9.23% improvement in code generation reliability

A math word problem case shows: Greedy decoding forgets initial values leading to errors; SLED/DoLa's early misunderstanding builds an incorrect reasoning chain; ActLCD selectively activates layer contrast, uses deep knowledge to build a coherent logical chain, arrives at the correct answer, and avoids the fundamental misunderstanding risk of traditional layer contrast methods.

Advantages of ActLCD: Global optimization perspective (beyond single-token optimization), dynamic adaptability (intelligent strategy adjustment), wide applicability (excellent performance across multiple tasks), lightweight and efficient (easy to integrate). This method has been open-sourced, providing a feasible solution to improve LLM reliability, and has been recognized by the main conference of EMNLP 2025.

ActLCD, through a reinforcement learning-driven dynamic layer contrast mechanism, demonstrates the possibility of improving factual accuracy while maintaining generation quality. With technological development, it is expected to promote more reliable and trustworthy AI assistants to be applied in real-world scenarios.