Exploration of the Application of Large Language Models in Clinical Diagnosis and Treatment Recommendations

Nina-Voj's bachelor's degree project uses the MIMIC-IV database as a benchmark to explore the performance of large language models (LLMs) in clinical diagnosis support and treatment recommendations, analyze their advantages, limitations, and significance for smart healthcare, and provide practical experience for AI-assisted medical decision-making.

Clinical diagnosis requires integrating multi-source information such as patient symptoms, test results, and medical history. However, the uneven distribution of medical resources and excessive workload of doctors are common global issues. With their strong natural language understanding and knowledge integration capabilities, LLMs are expected to become intelligent assistants for doctors. The core questions of this project are: How do current LLMs perform in real clinical scenarios? Can they reliably assist in diagnostic decisions? The MIMIC-IV (an authoritative public intensive care dataset) was selected as the evaluation benchmark.

MIMIC-IV is maintained by the MIT Laboratory for Computational Physiology and contains hundreds of thousands of de-identified health records of real patients, covering demographic data, diagnostic codes, laboratory results, medication records, etc. Advantages: The data comes from real clinical environments, with strong representativeness and complexity; de-identification protects privacy, supports academic research, and allows model performance evaluation close to real scenarios.

A systematic evaluation framework was adopted:

1. Prompt Engineering Strategies: Role setting (experienced clinician), structured prompts (organizing patient information), chain-of-thought prompts (guiding step-by-step reasoning);
2. Diagnosis Support Task: Generate diagnostic recommendations based on patient information, evaluate accuracy, completeness, and consistency with clinical guidelines, and compare with actual diagnoses in MIMIC-IV;
3. Treatment Recommendation Task: Evaluate the model's performance in recommending drug therapy, surgical intervention, nursing measures, etc.

The code structure is clear and includes multiple modules:

• Data Preprocessing: Text cleaning and standardization, numerical feature normalization, time series alignment and interpolation, diagnostic code mapping and conversion;
• Model Interface: A unified layer supports access to different LLMs (e.g., GPT, LLaMA series);
• Clinical Accuracy Evaluation: Combines expert manual review and automatic verification with medical knowledge bases, not relying solely on NLP metrics like BLEU.

Advantages: LLMs can understand complex clinical descriptions, integrate multi-source information, generate structured diagnostic thinking, and perform stably in common disease scenarios; Limitations: Insufficient recognition of rare diseases, prone to errors in numerical reasoning, medical knowledge has timeliness issues, and occasionally generates dangerous recommendations that require strict manual review.

Significance: LLMs cannot completely replace doctors, but can serve as auxiliary tools; promote human-machine collaboration (AI undertakes tasks like information integration to free up doctors' energy); establish multi-dimensional evaluation standards (clinical practicality, safety, interpretability); Outlook: Explore multi-modal fusion (imaging + laboratory tests + text), personalized diagnosis and treatment recommendations, and stricter clinical verification processes.