HalluciScope: An Automated Framework for Detecting LLM Hallucinations Using NLI and Explainable AI

HalluciScope is an open-source research framework developed by Niharika Banothu. It automatically detects, classifies, and explains hallucinations in large language model (LLM) outputs using Natural Language Inference (NLI) and Explainable AI technologies, providing reliability assurance for LLM deployment in high-risk domains such as healthcare, law, and finance. The framework supports REST API integration for easy practical application.

With the widespread application of LLMs like ChatGPT and Claude in high-risk domains, hallucination phenomena (generating content that seems plausible but is incorrect or factually inconsistent) have become increasingly prominent. For example, medical AI may give wrong medication advice, and legal assistants may cite non-existent precedents, severely restricting LLM deployment in critical tasks. Hallucination detection has become a cutting-edge topic in AI safety research.

The framework includes four core capabilities:

1. Detection Layer: Based on the Microsoft DeBERTa cross-encoder model, it determines hallucinations by comparing the logical relationship between the premise (true answer) and hypothesis (LLM response) (a contradiction score >0.5 indicates a hallucination);
2. Classification Layer: Uses GPT to classify hallucinations into four categories: factual, inferential, contextual, and fictional;
3. Explanation Layer: Uses sentence-transformers to calculate phrase semantic similarity (cosine similarity <0.3 marks suspicious phrases);
4. Service Layer: FastAPI REST interface supports real-time/batch integration. In terms of technical implementation, the code structure is clear, including detector, categorizer, explainer, and api modules. The usage process is simple: environment preparation → configure keys → run evaluation → API service.

Benchmark tests show that HalluciScope performs well on a 10-sample test set: the hallucination detection rate is about 40% (varies by model), with an average contradiction score of 0.43 and an average entailment score of 0.38. The study found that hallucination rates vary significantly among different LLMs (GPT-4 has a lower hallucination rate than GPT-3.5-turbo), verifying the effectiveness of the detection method.

HalluciScope aligns with the LLM reliability research of top AI labs and is a practical open-source solution for hallucination detection. Application scenarios include: model evaluation (comparing hallucination tendencies of different LLMs), prompt engineering optimization (improving prompts through analysis), production monitoring (real-time detection of hallucinatory outputs), and safety audits (validation before launching high-risk applications). Key takeaways: NLI technology effectively captures semantic inconsistencies, fine-grained explanations are more valuable, and hallucination detection should be a standard feature for high-risk LLM applications.