Intelligent Medical Diagnosis Support System Based on Large Language Models and RAG

Original Author and Source

• Original Author/Maintainer: afnanzia97
• Source Platform: GitHub
• Original Title: My_Dissertation
• Original Link: https://github.com/afnanzia97/My_Dissertation
• Publication Time: 2026-06-13

This study constructs an intelligent medical diagnosis support system based on large language models (LLMs) and Retrieval-Augmented Generation (RAG) technology, addressing issues such as high maintenance costs of traditional Clinical Decision Support Systems (CDSS) and knowledge hallucinations in LLMs. The system enhances generation through retrieval from external medical knowledge bases, improving diagnostic accuracy and interpretability. Experiments show that compared to the pure LLM baseline, the accuracy on MedQA increased by 13% and the hallucination rate decreased by 12%.

Medical diagnosis relies on professional knowledge and experience, but doctors are prone to cognitive biases. The explosive growth of medical knowledge makes traditional CDSS (rule-based engines) difficult to adapt. Although LLMs have potential, they face challenges such as knowledge hallucinations, poor timeliness, and lack of interpretability, requiring reliable technical solutions.

Advantages of RAG Architecture

• Knowledge Updatable: Independently maintain medical knowledge bases
• Reduce Hallucinations: Generate based on real literature
• Interpretability: Display original literature sources

Core Modules

• Knowledge Base: Integrate guidelines, cases, knowledge graphs, mixed storage (vector database/graph database/document library)
• Retrieval: Multi-path recall (semantic/keyword/graph) + re-ranking (cross-encoder/time decay)
• Generation: Structured prompt, output diagnosis list + evidence + examination suggestions + references
• Safety: Disclaimer, confidence threshold, sensitive content filtering

Evaluation Datasets

• MedQA (USMLE multiple-choice questions), PubMedQA (Q&A), self-built clinical cases

Key Results

• MedQA Accuracy: 65%→78%
• Hallucination Rate:15%→3%
• Expert Score:3.2/5→4.1/5
• Medical-specific encoder outperforms general encoder by 8 percentage points

This system represents an important direction for AI-assisted healthcare, combining the language capabilities of LLMs with structured knowledge bases to provide interpretable recommendations. Although it cannot replace doctors, as an intelligent assistant, it has demonstrated value in improving efficiency, reducing knowledge gaps, and promoting evidence-based practice, with expectations for future clinical applications.

Application Scenarios

• Outpatient Pre-diagnosis: Preliminary diagnosis suggestions
• Difficult Cases: Differential diagnosis ideas
• Medical Education: Clinical thinking practice

Deployment Challenges

• Privacy: Desensitization/local deployment/compliance
• Regulation: Clinical validation/auxiliary positioning/manual review

Future Directions

• Multimodal fusion (imaging/laboratory data)
• Personalized adaptation (patient medical history/preferences)
• Continuous learning (doctor feedback/knowledge updates)