HalluciScope: A Practical Framework for Automated Detection and Explanation of Large Language Model Hallucinations

HalluciScope is a research-grade open-source framework designed to address LLM hallucination issues in high-risk fields like healthcare, law, and finance. It integrates natural language reasoning (NLI) and explainable AI (XAI) to detect, classify, and explain hallucinations in LLM outputs. Key features include NLI-based detection using DeBERTa, four-category hallucination classification, token-level explainability, FastAPI integration, and batch evaluation. Developed by Niharika Banothu, it's available on GitHub (https://github.com/niharikabanothu/HalluciScope).

LLM hallucination—generating plausible but false information—poses severe risks in high-stakes applications (e.g., medical diagnosis, legal advice). This issue hinders reliable LLM deployment in production. HalluciScope was created to fill the gap of an automated toolchain for hallucination analysis, addressing the need for systems that not only detect but also explain hallucinations to support mitigation.

Detection: Uses DeBERTa cross-encoder (cross-encoder/nli-deberta-v3-base) to compute contradiction scores between LLM responses (hypothesis) and ground truth (premise); a score >0.5 indicates hallucination. Classification: Categorizes hallucinations into four types: factual (wrong facts like dates), reasoning (logical errors), context-ignoring (contradicts given context), fabrication (completely made-up info). Explainability: Uses sentence-transformers to calculate semantic similarity of phrases in LLM responses to ground truth; phrases with cosine similarity <0.3 are marked as suspicious.

Architecture: Modular design with components for detection (detector), classification (categorizer), explainability (explainer), API (FastAPI), and evaluation. Key files: hallucination_detector.py, hallucination_categorizer.py, hallucination_explainer.py, api/main.py, evaluate.py. Quick Start:

1. Clone repo: git clone https://github.com/niharikabanothu/HalluciScope.git
2. Set up venv and install dependencies: pip install -r requirements.txt
3. Configure .env with OPENAI_API_KEY. API Example: Use curl to send POST request to /analyze endpoint with prompt, ground truth, and LLM response; returns hallucination status, category, explanation, etc.

Preliminary evaluation on 10 samples shows:

• Hallucination rate ~40% (varies by model).
• GPT-4 has lower hallucination rate than GPT-3.5-turbo (consistent with industry observations).
• Average contradiction score:0.43, average entailment score:0.38. The framework demonstrates ability to systematically evaluate hallucination tendencies across models.

Value: Provides an out-of-the-box solution for hallucination detection in high-risk fields; supports both research (benchmarking models) and production (quality assurance). Emphasizes explainability to enable targeted mitigation. Limitations: Dependence on OpenAI API for classification; small evaluation dataset (10 samples); English-only support. Future: Replace OpenAI API with open-source models for offline use; expand dataset size; add multi-language support.