HalluCase: An Open-Source Tracking and Governance Platform for Legal AI Hallucination Incidents

HalluCase is an open-source registry for AI hallucination incidents in the legal field. It systematically records, tracks, and prevents hallucinatory outputs of generative AI in legal scenarios using a CVE-style system, helping the legal industry establish accountability mechanisms and detection tools. Its core objectives include: establishing systematic records, promoting industry learning, driving the development of detection tools, and pressuring AI vendors, etc.

Generative AI has rapidly penetrated the legal field since 2023. While improving work efficiency, AI hallucinations (generating seemingly reasonable but false content) have become a fatal hidden danger. In 2025, there were over 729 publicly reported legal AI hallucination incidents, with impacts including: fines exceeding $31,000, sanctions from the American Bar Association, contract risks, and erroneous case judgment bases, etc. Core issues: Lack of accountability for AI false information and a systematic tracking and prevention mechanism.

HalluCase's core mechanisms include:

1. Incident Classification System: 9 major types (fake_citation, misquoted_statute, etc.);
2. Severity Grading: 5 levels: critical/high/medium/low/info;
3. Unique Identification System: HC-XXXXXX format numbering;
4. Standardized Report Fields: id, hc_id, title, description, hallucination type, severity, etc.

CLI Tool

Supports commands for searching, retrieving reports, submitting incidents, etc. (e.g., hallucase search --query "fake citation" --severity critical).

API Service

RESTful API provides functions like report CRUD and statistics (e.g., GET /api/reports, POST /api/reports).

Local Deployment

Can be run directly via npx or deployed by cloning the source code, default port is 3457.

For Legal Practitioners

Risk education, due diligence checklists, compliance references;

For AI Developers

Test datasets, adversarial training, industry benchmarks;

For Regulatory Authorities

Basis for policy formulation, industry supervision tools, foundation for international coordination.

HalluCase uses the Apache 2.0 open-source license, and community contributions are welcome:

1. Fork the repository and create a feature branch;
2. Submit a Pull Request;
3. Submit new incidents via CLI or GitHub Issue;
4. Improve documentation.

HalluCase represents a pragmatic approach to addressing AI risks: systematically recording, analyzing, and solving problems. In the legal field, transparent accountability mechanisms and shared knowledge bases are crucial. As AI is deeply applied in the legal industry, its value will become increasingly prominent, and it is a collective effort to maintain professional standards and public trust.