Multimodal-Model-Zoo: A Curated Resource Library of 100+ Multimodal Large Language Models

The Multimodal-Model-Zoo project aims to address the pain point of scattered resources for Multimodal Large Language Models (MLLMs). It carefully compiles over 100 cutting-edge models, providing comprehensive technical references and navigation for researchers and developers. It covers the evolution path of multimodal technologies and reduces research and model selection costs through structured classification.

The evolution of multimodal large language models ranges from simple image-text alignment to complex cross-modal reasoning: early models focused on aligning visual and text features to achieve basic functions like image description and visual question answering; the new generation models have capabilities such as fine-grained visual understanding, multi-turn dialogue reasoning, and cross-modal generation. The resource library covers the complete technical path.

The project uses multi-dimensional classification: by architecture, it is divided into three categories—encoder-decoder, LLM extension, and end-to-end training; by capability, it includes image-text understanding, video analysis, general models, etc. Each model entry contains scale, training data, innovation points, and official links. Structured organization reduces research costs.

The resource library includes milestone models: some improve fine-grained understanding through innovative visual encoders; some break through cross-modal alignment via training strategies; open-source models maintain performance while lowering deployment thresholds, promoting technology popularization.

For general dialogue systems, choose models with long-context multi-turn reasoning capabilities; for vertical fields (medical imaging, industrial quality inspection), select specially optimized versions; the classification label system helps quickly filter candidate models.

The project embodies the spirit of knowledge sharing and will continue to be updated and improved. It serves as a starting point for beginners to learn and an important tool for senior researchers to track cutting-edge developments.