Arabic Fact-Checking Open-Source Tool: Evidence Retrieval and Claim Verification Based on Large Language Models

Arabic-Fact-Checking is an open-source project for Arabic fact-checking, providing a complete pipeline from evidence retrieval, QA pair generation to claim verification. It enables researchers to quickly build and evaluate fact-checking systems, aiming to fill the gap of scarce high-quality Arabic fact-checking tools while serving as a research platform to explore best practices of Large Language Models (LLMs) in fact-checking.

In the era of information explosion, misinformation spreads far faster than the truth, and Arabic users lack high-quality fact-checking tools. This project emerged to fill this gap, providing a complete solution for the Arabic community. It also serves as a research platform, allowing developers to quickly experiment with different large language models in fact-checking tasks and explore best practices for Retrieval-Augmented Generation (RAG) and claim verification.

The project covers three core modules throughout the full lifecycle of fact-checking:

1. Evidence Retrieval Module: Retrieves relevant evidence fragments from large-scale text corpora, supporting keyword matching, semantic similarity search, and hybrid retrieval strategies, combined with large language models to understand the deep semantics of claims;
2. QA Pair Generation Module: Automatically generates QA pairs based on retrieved evidence, helping verifiers quickly understand evidence content and providing training data for model fine-tuning. The generated results undergo quality control to ensure relevance and accuracy;
3. Claim Verification Module: Receives claims to be verified and evidence, outputs verification results of support, refutation, or insufficient information. It supports multiple strategies from rule-based methods to chain-of-thought reasoning, allowing developers to flexibly choose the one that fits their scenario needs.

The project adopts a modular design, with components decoupled through clear interfaces, offering significant advantages: easy scalability (replacing a single module does not require system reconstruction) and convenient evaluation (independently evaluating each module's output to identify bottlenecks). It fully leverages the semantic understanding and reasoning capabilities of large language models while properly handling the uniqueness of Arabic—linguistic features such as right-to-left writing system, rich morphological changes, and dialect diversity.

Applicable to multiple scenarios:

• News agencies: Assisting editors in quickly verifying the authenticity of Arabic news;
• Social media platforms: Serving as an automated content moderation component;
• Academic researchers: Providing a standardized benchmark framework to facilitate comparison of different methods' effectiveness;
• Regions with limited educational resources: Open-source and free to access, promoting the democratization of fact-checking technology.

The project documentation details environment configuration, data preparation, and operation processes, allowing even NLP beginners to build a runnable prototype in a short time. Community contributions are welcome: code improvements, document translation, new evaluation datasets, etc., to jointly create value for the Arabic NLP community.

Arabic-Fact-Checking represents an important advancement in fact-checking technology for low-resource languages, providing both a practical tool and an open research platform. With the progress of large language model technology, we look forward to more language communities benefiting from similar open-source projects to build a truthful and trustworthy information environment together.