RzenEmbed: A High-Performance Embedding Model for Multimodal RAG Scenarios

RzenEmbed, an open-source embedding model by 360CVGroup, is optimized for multimodal RAG scenarios and won first place in both the overall and VisDoc categories of the MMEB benchmark. The model is available on GitHub (link: https://github.com/360CVGroup/RzenEmbed) and was released on June 16, 2026. This article will cover its background, technical approach, performance evidence, application scenarios, and more.

In Retrieval-Augmented Generation (RAG) systems, embedding models are key to converting multimodal content like text and images into vectors. Their quality directly impacts retrieval accuracy and generation relevance. Traditional models are mostly designed for single modalities and struggle to handle multimodal documents with mixed text and images. As multimodal RAG is widely used in scenarios like document Q&A and knowledge base retrieval, the demand for specially optimized multimodal embedding models is growing. RzenEmbed is developed to address this need.

RzenEmbed uses a multi-tower architecture to handle different modal inputs:

• Text Encoder: Based on Transformer, optimized for document-level long texts, supporting multilingual and contextual relationships.
• Visual Encoder: Adopts ViT architecture to process visual content like document images, extracting text, charts, and layout information, and shares the vector space with the text encoder.
• Cross-modal Fusion: Achieved through contrastive learning, joint representation learning for image-text alignment, and fine-grained interaction mechanisms.

The training strategy uses multiple stages: Pre-training (large-scale image-text pairs) → Document-level training (long context) → RAG-specific fine-tuning → Instruction fine-tuning. Data construction includes synthetic data, real documents, retrieval pair construction, and hard negative example mining.

MMEB (Massive Multimodal Embedding Benchmark) is an authoritative multimodal embedding evaluation benchmark covering multiple tasks, domains, languages, and real-world scenarios. The VisDoc subset focuses on visual document understanding, testing various document types, mixed text-image layouts, structured information, and cross-page associations.

RzenEmbed won first place in both the overall and VisDoc categories of MMEB. Key factors include: domain focus on multimodal RAG, high-quality document-level training data, optimized network architecture, and multi-stage progressive training.

Application scenarios of RzenEmbed include:

• Enterprise Knowledge Base Retrieval: Product manuals, technical document Q&A, training material search, regulatory compliance queries.
• Academic Literature Analysis: Paper retrieval, experimental method reproduction, literature review generation, research trend analysis.
• Multimodal RAG Systems: Document Q&A, content recommendation, knowledge graph construction, intelligent summarization.

Usage and deployment recommendations:

• Model Loading: Supports Transformers library, Sentence-Transformers, ONNX export, and quantized versions.
• Vector Index Construction: Document parsing → Chunking → Embedding generation → Storage in vector databases (e.g., Chroma, Milvus).
• Retrieval Optimization: Hybrid retrieval (keyword + vector), re-ranking (cross-encoder), query expansion, metadata filtering.

RzenEmbed represents the trend of multimodal embedding models moving toward specialization and scenario-specific optimization. Its double championship in MMEB proves the value of scenario-based optimization. For developers of enterprise-level multimodal knowledge bases and document Q&A systems, it is a high-quality open-source and customizable component.

Future directions: Larger-scale models, support for more modalities (video/audio), longer context, domain specialization (legal/medical); technical optimizations include inference acceleration, compression techniques, and dynamic inference.