MidstreamAI Extractive Q&A Bot: Practice of a Zero-Hallucination Semantic Retrieval System

MidstreamAI Extractive Q&A Bot is an extractive Q&A system based on FastAPI and React. It achieves millisecond-level document retrieval via sentence-transformers and FAISS, uses SOLID architecture to ensure code maintainability, and its core feature is completely eliminating hallucination issues of generative AI—only returning original text fragments actually present in the documents.

Project Background and Core Issues

In enterprise knowledge management and customer service scenarios, traditional generative AI chatbots have the pain point of hallucinations and may fabricate incorrect information. MidstreamAI adopts a pure retrieval-based architecture with the core design concept of 'zero hallucination', sacrificing some conversational flexibility in exchange for accuracy and credibility.

Technical Architecture Overview

• Separation of front-end and back-end: Backend based on FastAPI, front-end using React 18+ with TypeScript
• Backend core components: Document loading service (supports multiple formats via factory pattern), text chunk processor (default 200 words per chunk, 30-word overlap), embedding vector generator, FAISS vector storage
• Text chunking strategy: Balances retrieval accuracy and context integrity; CHUNK_SIZE parameter can be adjusted.

Embedding Model and Vector Retrieval

• Embedding model: Selected sentence-transformers/all-MiniLM-L6-v2, 384-dimensional vector, balancing efficiency and effectiveness
• Vector retrieval: FAISS engine provides millisecond-level approximate nearest neighbor search; measured response time is below 200ms
• Confidence threshold: Default 0.4, filters low-relevance results, adjustable according to business needs.

SOLID Principles Practice

• Single Responsibility: Each service has clear responsibilities (DocumentService, QueryService, etc.)
• Open/Closed Principle: Document loaders extend new formats via factory pattern
• Liskov Substitution: IVectorStore interface supports interchange of different storage implementations
• Interface Segregation: Fine-grained interface design (IDocumentLoader only defines the load method)
• Dependency Inversion: High-level modules depend on interfaces rather than concrete implementations.

Front-end Interaction and Deployment Configuration

• Front-end: React+Material-UI, with intelligent formatting (bold headings, line breaks, etc.), fragment extraction, comparative query, professional content filtering
• Deployment: Backend Python 3.9+ (run via uvicorn), front-end Node.js 16+ (built with Vite)
• Configuration: Adjustable parameters include CHUNK_SIZE, CONFIDENCE_THRESHOLD, TOP_K_RESULTS, etc., with document hot-update mechanism.

Applicable Scenarios and Value Proposition

• Applicable scenarios: Industries with high accuracy requirements such as medical, legal, finance; technical document query; enterprise knowledge base
• Value: Zero hallucination feature, low AI application entry threshold (no need to train dedicated models).

Limitations and Summary Insights

• Limitations: Cannot answer uncovered questions, no reasoning ability, does not support multi-turn context
• Summary: Choose appropriate solutions based on business needs; software engineering practices like SOLID principles ensure project maintainability, providing reference for enterprise knowledge base Q&A systems.