BGE-SigLIP: An Embedding Model Unifying Multimodal and Cross-Lingual Representations

The BGE-SigLIP project integrates the SigLIP-2 visual encoder and BGE-M3 text encoder to build a unified vector space, enabling cross-lingual image-text retrieval in over 100 languages, providing new solutions for RAG applications and cross-lingual image search. The project is maintained by Aeluin-Technologies and was released on GitHub on May 26, 2026 (link: https://github.com/Aeluin-Technologies/BGE-SigLIP).

In the current AI ecosystem, visual understanding (e.g., SigLIP series) and text understanding (e.g., BGE-M3) belong to different model families and operate in different vector spaces, making direct joint retrieval impossible. The innovation of BGE-SigLIP lies in mapping the SigLIP-2 visual encoder to the 1024-dimensional vector space of BGE-M3 to achieve unified representation.

1. Unified vector space: Images and texts are projected into the same 1024-dimensional space, and cosine similarity can be calculated directly;
2. Native cross-lingual support: Inherits the multilingual capabilities of BGE-M3, supporting over 100 languages;
3. Asymmetric contrastive fine-tuning: Unidirectionally aligns SigLIP-2 to the BGE-M3 space, preserving the depth of text semantics;
4. Technical route: Fine-tunes the SigLIP-2 visual encoder with the BGE-M3 vector space as the target, compatible with the existing BGE-M3 ecosystem.

1. Multimodal RAG: Retrieves text fragments and images simultaneously, providing rich context for LLMs;
2. Cross-lingual image search: E-commerce platforms support multi-language product image queries;
3. Multimodal content recommendation: Recommends relevant content based on image-text similarity;
4. Image annotation and classification: Completes image classification and annotation with zero/few shots.

Compared to traditional models like CLIP, its advantages are:

1. Stronger text representation: Inherits BGE-M3's long text and fine-grained semantic understanding capabilities;
2. Cross-lingual capability: Natively supports over 100 languages;
3. Ecosystem compatibility: Shares the vector space with the BGE series, facilitating integration into existing systems.

1. Evaluate existing systems: If BGE-M3 is already in use, migration cost is low;
2. Data preparation: Collect domain-specific image-text pairs for fine-tuning to improve performance;
3. Indexing strategy: Use multimodal vector databases like Milvus/Pinecone;
4. Query optimization: Leverage BGE-M3's multi-granularity features to support various query forms.

Current limitations: Only focuses on image-text bimodality; Future outlook: Expand to more modalities such as video and audio, and adapt to vertical fields like medical imaging and satellite images.

BGE-SigLIP adds visual understanding capabilities to the BGE ecosystem through model fusion without sacrificing text embedding quality, and its unified vector space design simplifies multimodal retrieval. It is a noteworthy technical solution for building next-generation RAG systems or multimodal applications.