AI RAG Document Chatbot: PDF Intelligent Q&A System Based on Local LLM

This project is a complete open-source RAG (Retrieval-Augmented Generation) chatbot implementation that supports multi-PDF uploads and intelligent conversational Q&A. The core tech stack includes Streamlit, LangChain, ChromaDB, MongoDB, and the Ollama local Phi model. Its biggest advantage is fully local operation without relying on external APIs, which both protects data privacy and reduces usage costs.

Original Author/Maintainer: Karan3710 Source Platform: GitHub Original Link: https://github.com/Karan3710/AI-RAG-DOCUMENT-CHATBOT Release Time: May 2026

This project is built on the RAG architecture, allowing users to upload multiple PDF documents and interact via natural language. Its core feature is local operation using the Phi model deployed via Ollama, with data never leaving the local environment—ideal for privacy-focused scenarios.

1. User Authentication: Supports registration and login, password hash protection, multi-user data isolation.
2. Multi-PDF Processing: Automatic text extraction, intelligent chunking, preserves document structure.
3. RAG Pipeline: Document vectorization → vector storage → semantic retrieval → context generation.
4. Conversation Management: MongoDB stores chat history, supports multi-turn context and history review.
5. Local Inference: Ollama runs the Phi model, no external API calls, zero-cost high-frequency usage.
6. Responsive Interface: Built with Streamlit, simple and intuitive, adapts to multiple devices.

• Python: Core development language, provides rich AI/ML library support.
• Streamlit: Quickly build interactive web apps, pure Python development without front-end knowledge.
• LangChain: Framework connecting components, simplifies RAG pipeline implementation.
• ChromaDB: Open-source vector database, efficient storage and retrieval of embedding vectors.
• MongoDB: Document-oriented database, suitable for storing flexible conversation data.
• Ollama: Local LLM running tool, deploy models with one command.
• Phi Model: Microsoft's small-sized high-performance model, suitable for consumer-grade hardware.

Document Processing Phase:

1. Upload PDF → extract text → intelligent chunking → generate embedding vectors → store in ChromaDB to build index.

Conversation Phase:

1. User query → vectorize query → retrieve relevant fragments from ChromaDB → construct prompt → Phi model generates answer → store conversation in MongoDB → display result.

1. AI Document Assistant: Quick query for internal enterprise documents.
2. Research Paper Chatbot: Assists in understanding paper content and comparative research.
3. PDF Knowledge Retrieval: Legal/financial/medical literature analysis.
4. Enterprise Knowledge Base: Integrate scattered documents, provide unified Q&A entry.
5. AI Learning Assistant: Personalized learning support for student textbooks/lecture notes.

1. Install dependencies: pip install -r requirements.txt
2. Download Phi model: ollama pull phi
3. Start Ollama service: ollama serve
4. Run the app: streamlit run app.py

No complex configuration needed, suitable for quick deployment and personal use.

This project is a complete example of RAG technology practice, providing learning resources for developers. It demonstrates the integration of modern AI tech stacks, and its local design represents the direction of intelligence under privacy protection. For enterprises, it can be extended as a customized framework—such as adding more document formats or integrating enterprise authentication.