A Survey of Multimodal Adaptation and Generalization Technologies: From Traditional Methods to Foundation Models

This article introduces a survey paper on multimodal adaptation and generalization published in TPAMI 2026. It systematically reviews the technical progress in five major research directions: multimodal domain adaptation, test-time adaptation, domain generalization, adaptation using foundation models, and adaptation of foundation models themselves. It also covers technical trends, key challenges, open-source resources, and other content, providing a comprehensive reference for researchers.

Multimodal learning (combining modalities such as vision, language, and audio) is an important direction in AI. However, the distribution difference between training and test data (e.g., sunny daytime vs. rainy night environments in autonomous driving) is a core challenge, requiring models to maintain stable performance under varying conditions.

The survey divides into five core scenarios:

1. Multimodal Domain Adaptation: Knowledge transfer from source domain to target domain (e.g., MM-SADA, xMUDA);
2. Test-Time Adaptation: Online adaptation using only test samples (e.g., MM-TTA, Latte);
3. Domain Generalization: Learning domain-invariant features without target domain data during training (e.g., SimMMDG, MOOSA);
4. Foundation Model-Assisted Adaptation: Leveraging pre-trained models like CLIP (e.g., PromptStyler, CoOp);
5. Adaptation of Foundation Models Themselves: Parameter-efficient methods (prompt learning/adapters, e.g., CoOp, CLIP-Adapter).

Domain trends:

1. From unimodal to multimodal: Combining multimodal information to improve robustness;
2. From closed-set to open-set: Handling unseen categories in the target domain (e.g., MOOSA);
3. From training-time to test-time adaptation: Online updates are more practical;
4. From traditional methods to foundation models: Technologies like prompt learning and adapters are emerging.

Challenges:

1. Modality imbalance: Avoiding partial modalities dominating decisions;
2. Modality missing and noise: Improving robustness;
3. Computational efficiency: Deployment in resource-constrained environments;
4. Theoretical foundation: Unclear mechanisms. Future focus should be on efficient adaptation and robust system construction.

The survey maintains an open-source repository called Awesome-Multimodal-Adaptation, which includes papers, code, and datasets; the author team has released benchmarks and projects such as SimMMDG, MOOSA, and AEO.

The field of multimodal adaptation and generalization is evolving from traditional methods to foundation models, and the TPAMI survey provides a comprehensive technical map. Future focus should be on efficient downstream adaptation and robust systems; researchers are advised to start with classic methods and conduct research combined with practical scenarios.