Practical Guide to Fine-Tuning Medical Large Models: Open-Source Solution for Coronary Artery Disease (CAD) Prediction Using the UKB Dataset

This article introduces an open-source fine-tuning framework for Coronary Artery Disease (CAD) prediction based on the UK Biobank (UKB) dataset. It supports three strategies: QLoRA, LoRA, and full fine-tuning, integrates DeepSpeed for accelerated training, and uses weighted loss and oversampling to address class imbalance in medical data—providing a flexible and efficient large model fine-tuning solution for the medical AI field.

The medical AI field is undergoing a transformation driven by large language models (LLMs), but medical diagnosis demands higher model accuracy, interpretability, and reliability. Coronary Artery Disease (CAD), one of the leading causes of death globally, has significant value in early prediction. However, medical data faces challenges such as sample imbalance and privacy sensitivity—how to efficiently fine-tune large models to adapt to professional scenarios has become a key focus.

ukb-cad-llm-finetuning is an open-source framework designed specifically for medical binary classification tasks. Built on Hugging Face Transformers, PEFT, and DeepSpeed, it provides a complete solution from data preparation to deployment for Coronary Artery Disease (CAD) prediction using the UKB dataset. Its core design balances flexibility and efficiency, supporting three fine-tuning strategies (QLoRA, standard LoRA, full fine-tuning) to adapt to different hardware environments.

• QLoRA: 4-bit quantization + LoRA, low memory usage, uses the paged_adamw_32bit optimizer—suitable for rapid experiments on consumer GPUs;
• LoRA (bf16): bf16 full-precision base model + LoRA, avoids quantization loss—ideal for high-precision clinical scenarios;
• Full Fine-Tuning: No quantization or LoRA, updates all parameters to pursue ultimate performance (requires sufficient computing resources).

• DeepSpeed Integration: Supports ZeRO-2 (partitioning optimizer states/gradients) and ZeRO-3 (further partitioning model parameters) for distributed training, switchable via configuration files;
• Class Imbalance Handling: Uses WeightedTrainer to implement weighted cross-entropy loss, supports positive sample oversampling, and improves the learning ability for minority classes.

• Configuration System: YAML-driven, separating configurations for models, datasets, tasks, DeepSpeed, etc. (e.g., configs/models/ defines model strategies, configs/experiments/ combines configurations);
• Evaluation and Prediction: After training, load checkpoints via the cli.eval script to output metrics.json (accuracy, F1, etc.) and predictions.jsonl (per-entry results), facilitating integration with existing evaluation workflows.

ukb-cad-llm-finetuning provides a practical template for fine-tuning medical large models, demonstrating the feasibility of adapting general-purpose LLMs to professional medical scenarios and lowering the entry barrier for developers. With the popularization of multimodal medical data in the future, this framework is expected to play a greater role in complex scenarios such as image-text joint diagnosis.