GMAI-VL: How a 7B-Parameter Medical Vision-Language Model Surpasses 34B-Large Models

GMAI-VL is a vision-language model specifically designed for the medical field. With only 7 billion parameters, it achieves an accuracy of 88.48% on the OmniMedVQA benchmark, surpassing models with 5 times more parameters. The project also open-sources a 5.5 million medical multimodal dataset, providing new solutions for the medical AI field.

The medical AI field has long faced core contradictions: general large models lack professional medical knowledge, while specialized medical models often have limited data scale and insufficient generalization ability. The emergence of GMAI-VL provides a remarkable solution to this problem—surpassing competitors with 34 billion parameters on multiple medical visual question-answering benchmarks using only 7 billion parameters.

Dataset Construction: Adopts an "annotation-guided data generation" process to ensure data quality, containing 5.5 million question-answer pairs (from 219 professional data sources, covering 13 imaging modalities and 18 departments). Subsets include GMAI-MM-Caption (1.7 million), GMAI-MM-Percept (1.3 million), etc. Compared with existing datasets, it has obvious advantages in scale, modal diversity, etc.

Model Architecture: Based on the LLaVA architecture, using InternLM2.5-7B as the language backbone, paired with a CLIP visual encoder and MLP projection layer. Adopts a three-stage progressive training strategy: shallow alignment (projection layer only), deep alignment (projection layer + visual encoder), and instruction fine-tuning (full model).

In the OmniMedVQA benchmark test, GMAI-VL (7 billion parameters) achieves an accuracy of 88.48%, surpassing InternVL2 (40 billion parameters, 78.70%) and HuatuoGPT-Vision (34 billion parameters, 73.23%). It also performs excellently on GMAI-MMBench (62.43%), MMMU H&M (51.3%), and VQA-RAD (66.3%), proving the value of high-quality data and scientific training strategies.

1. Data Quality First: Does not blindly pursue scale; ensures each sample has a reliable medical basis through annotation-guided generation.
2. Progressive Capability Development: Three-stage training avoids knowledge conflicts and gradually improves model capabilities.
3. Open-Source Ecosystem Integration: Uses the XTuner training framework, VLMEvalKit evaluation tool, and InternLM2.5 language backbone, focusing on core medical issues.

1. Medical Image Question-Answering: Assists doctors in quickly screening images and answering questions like "What abnormalities does the X-ray show?"
2. Multimodal Medical Dialogue: Supports dialogue interactions with uploaded images, providing image-based answers.
3. Medical Education Assistance: Helps students understand the correspondence between medical image features and pathological manifestations.

Current Limitations:

• Professional field restrictions: Performance on rare diseases and complex cases remains to be verified.
• Language coverage: Mainly supports Chinese and English.
• Clinical validation: Requires strict clinical validation before being used in actual diagnosis and treatment.

Use Recommendations: Positioned as a research and auxiliary tool, it should not be directly used for clinical diagnosis decisions. Model outputs need to be reviewed by professional medical personnel.

Implications:

1. Data quality is more important than model scale.
2. Open-source collaboration accelerates progress in the field.
3. Progressive training strategies are worth promoting.

Future Outlook:

• More derivative research.
• Specialized optimization for specific diseases/imaging modalities.
• Integration with electronic medical records and PACS systems.
• Improvement of multimodal medical AI evaluation standards.