CoALFake: A New Cross-Domain Fake News Detection Method Combining Human-Machine Collaborative Annotation and Active Learning

This article proposes the CoALFake framework, which combines human-machine collaborative annotation with domain-aware active learning to solve the core problems of scarce labeled data and loss of domain features in cross-domain fake news detection, achieving efficient and accurate fake news identification.

Practical Challenges

1. Overly Strong Domain Specificity: Most models are optimized for specific domains, leading to a sharp drop in cross-domain performance and limiting practical applications.
2. Difficulty in Obtaining Labeled Data: High-quality annotation requires professional fact-checking, which is costly and time-consuming, making it hard to scale.

Core Cross-Domain Difficulties

• Scarcity of Labeled Data: Supervised learning relies on large amounts of labeled samples, but fake news annotation is resource-intensive and difficult to obtain.
• Loss of Domain Feature Information: Existing methods either rigidly classify or ignore domain features, leading to loss of key information and impairing detection accuracy.

Human-Machine Collaborative Annotation Mechanism

LLMs are responsible for initial annotation (scalable, low-cost), while human experts supervise and validate, balancing cost-effectiveness, quality assurance, and scalability.

Domain Embedding Technology

Dynamically captures domain-specific details and cross-domain patterns, helping to train truly domain-agnostic detection models.

Domain-Aware Sampling Strategy

Active learning prioritizes samples with diverse domain coverage, avoiding over-representation or under-representation of certain domains and improving generalization ability.

Cross-Dataset Consistency

Performs excellently on multiple datasets with strong generalization ability, adapting to different data distributions and domain characteristics.

Cost-Benefit Analysis

Even with minimal human supervision, it maintains excellent performance and effectively utilizes limited expert resources.

Comparison with Existing Methods

Significantly improves key metrics such as accuracy, recall, and F1 score, thanks to the effective use of domain information and intelligent management of the annotation process.

Fact-Checking Support

Helps news agencies and fact-checking organizations prioritize the identification of high-risk fake news and optimize resource allocation.

Social Media Implications

Its cross-domain capability and cost-effectiveness make it a potential solution for platform-level detection, balancing freedom of speech and fake information containment.

Contribution to the Research Community

Provides a reference for the idea of combining human-machine collaboration and active learning, which can be extended to other NLP tasks requiring large-scale annotation.

Limitations

The current framework mainly targets English news and does not support multilingualism; inference efficiency needs to be optimized for real-time detection; adversarial robustness needs to be improved to deal with malicious evasion.

Future Directions

• Expand multilingual support
• Optimize inference efficiency for real-time detection
• Enhance adversarial robustness to deal with malicious evasion strategies