Agentic News Verifier: A Multi-step Reasoning Fact-checking Mechanism for Meta Hackathon Winning Project

Agentic News Verifier is a winning project from the joint event of Meta PyTorch Hackathon and Scaler Technology Institute, designed as an agent-based fact-checking system with an OpenAI Gym-style environment. Its core design philosophy is to enable AI to proactively search for evidence before forming conclusions, just like human fact-checkers. This behavior is incentivized through reward mechanisms, which effectively reduce model hallucinations and improve accuracy. The tech stack includes FastAPI, Docker, Qwen2.5-72B, etc., supporting reinforcement learning framework integration and containerized deployment.

In the era of information explosion, fake news spreads faster than the truth. Traditional rule-based fact-checking struggles to handle complex environments, and relying solely on large language models is prone to hallucinations. This project was born in this context, aiming to improve fact-checking accuracy and reduce hallucination risks through a multi-step reasoning process (searching for evidence first, then making judgments), serving the joint event of Meta PyTorch Hackathon and Scaler.

Adopting a modular architecture, the core components include:

1. Environment Engine: Similar to OpenAI Gym environment, it maintains a news database and progressive reward logic to encourage reasonable fact-checking strategies;
2. FastAPI Backend: Provides RESTful APIs compliant with OpenEnv specifications (/reset to reset state, /step to execute actions), supporting reinforcement learning framework integration;
3. Docker Deployment: Complete configuration supports one-click deployment, ensuring environment consistency and result reproducibility.

The carefully designed reward structure guides agent behavior:

• Search Action: +0.15 reward, encouraging investigation before conclusion;
• Correct Verification: +0.95 reward (meeting Meta's evaluation requirement of (0,1) interval);
• Incorrect Verification: +0.05 reward (to avoid harsh punishment);
• Default Step: +0.05 base reward to maintain interaction motivation. The design embodies the positive incentive principle of reinforcement learning, guiding a rational fact-checking mode.

Multi-task Evaluation: Built-in 3 independent tasks (task-1 to task-3) covering fake information types such as fabricated titles, out-of-context quotes, and misleading data, ensuring comprehensive and robust evaluation; Technology Selection: FastAPI (high-performance asynchronous API), Pydantic (data validation), Docker (containerization), Qwen2.5-72B (underlying large language model).

Local Testing: Clone the repository → Install dependencies (pip install fastapi uvicorn pydantic) → Start the service (python -m uvicorn server.app:app --host 0.0.0.0 --port 7860); Production Deployment: Build via Docker image, supporting operation on platforms like Hugging Face Spaces, AWS, GCP, etc.

Social Value: Can serve as an assistant to human fact-checkers to improve efficiency, or be integrated into social media to provide real-time fact-checking; Research Value: Provides an experimental platform for AI safety and alignment research, supporting testing of reward design, reasoning strategies, and model performance; Insight: Proves that reasonable incentive mechanisms can guide AI to form rational and reliable behaviors, providing references for AI application development.