UniMedVL: A Unified Foundation Model for Medical Multimodal Understanding and Generation

UniMedVL is a unified medical multimodal foundation model developed and open-sourced by the uni-medical team. It integrates understanding and generation capabilities via the Observation-Knowledge-Analysis (OKA) three-layer framework. With a parameter scale of 14B, the model has achieved SOTA performance in tasks like medical visual question answering and medical image generation. Its core paper has been accepted by ICML 2026, and it is accompanied by the UniMed-5M dataset (over 5.6 million samples).

Traditional medical AI methods suffer from fragmentation issues such as task isolation and model redundancy. UniMedVL aims to integrate multimodal capabilities through a unified architecture. Its core paper has been accepted by ICML 2026, and the supporting UniMed-5M dataset provides a foundation for training.

The OKA framework is inspired by clinical workflows: Observation Layer: Perceives features of medical images; Knowledge Layer: Performs reasoning by linking to medical knowledge bases; Analysis Layer: Generates reports/answers. The architecture enables cross-task collaborative enhancement.

Three-stage training:

1. Foundation training (85K steps): Establishes vision-language alignment;
2. Instruction fine-tuning (120K steps): Enhances cross-modal understanding;
3. Unified training (70K steps): Builds comprehensive capabilities. The progressive strategy ensures stable training.

In medical VQA, it leads on datasets like SLAKE (75.4%) and PathVQA (53.5%); the average gFID for image generation is 96.29; it has strong generalization ability in cross-modal tasks; the 14B parameter version is more efficient than 7B task-specific models.

It supports scenarios such as auxiliary diagnosis, medical education, scientific research analysis, and multimodal interaction, providing a comprehensive solution for medical AI applications.

It provides pre-trained weights (on HuggingFace), inference code, partially open datasets, and a project homepage to promote technology dissemination.

UniMedVL represents a significant advancement in medical multimodal AI. Its limitations include incomplete open-sourcing of training code; future plans include expanding modalities and enhancing EMR integration.