RAMM: A New Retrieval-Augmented Multimodal Framework for Fake News Detection

This paper proposes the RAMM (Retrieval-Augmented Multimodal Model for Fake News Detection) framework, which aims to address the shortcomings of existing fake news detection models in cross-instance narrative consistency and domain-specific knowledge reasoning. Through two core modules—abstract narrative alignment and semantic representation alignment—combined with a retrieval-augmented mechanism, the framework has been validated on three public datasets, providing new insights for fake news detection.

In the era of social media, fake news spreads rapidly, but traditional detection methods have limitations:

1. Isolated Processing Flaw: Treating each news item as an independent entity, making it difficult to capture the cross-instance narrative consistency of fake news spread in clusters;
2. Knowledge Dependency Issue: Over-reliance on fixed knowledge in pre-trained parameters, leading to a significant decline in generalization ability when facing emerging events or niche domains.

The abstract narrative alignment module of RAMM can adaptively extract abstract narrative consistency from diverse instances across different domains, aggregating relevant knowledge to model high-level narrative information. By analyzing semantic connections between news samples, this module identifies cross-instance narrative patterns and effectively detects fake news that changes its expression but retains the core structure.

The semantic representation alignment module is inspired by the human news verification process (analogical reasoning based on past experience). It transforms the model's decision-making paradigm from direct multimodal feature inference to instantiated analogical reasoning, making the model's reasoning approach closer to human cognitive patterns.

RAMM uses a Multimodal Large Language Model (MLLM) as its backbone, which can process multimodal information such as text and images simultaneously and capture cross-modal semantic connections. By dynamically retrieving relevant instances and knowledge, it supplements the fixed knowledge in model parameters, significantly improving domain adaptation capabilities.

RAMM performs excellently on three public datasets:

1. Cross-domain Generalization: Outperforms traditional methods, solving the problem of insufficient knowledge in emerging domains;
2. Cluster Fake News Detection: Effectively identifies fake news campaigns spread collaboratively by multiple accounts, with important practical value.

The research team has open-sourced the RAMM code on GitHub to promote domain research and industry applications. In the future, they will combine the development of multimodal large language models and retrieval technologies to expand RAMM's application in more scenarios, helping to build a clean online information environment.