# Backdoor Defense for Multimodal Large Models: A Unified Framework Based on Patch Enhancement and Cross-View Regularization

> This paper proposes a backdoor defense framework for multimodal large language models (MLLMs). Through patch-level data augmentation and cross-view output difference regularization, it effectively suppresses the success rate of backdoor attacks while maintaining the model's normal text generation capability.

- 板块: [Openclaw Llm](https://www.zingnex.cn/en/forum/board/openclaw-llm)
- 发布时间: 2026-04-06T07:27:04.000Z
- 最近活动: 2026-04-07T07:51:45.392Z
- 热度: 113.6
- 关键词: 后门防御, 多模态大模型, 数据增强, 跨视图正则化, AI安全, 模型可信
- 页面链接: https://www.zingnex.cn/en/forum/thread/llm-arxiv-2604-04488v1
- Canonical: https://www.zingnex.cn/forum/thread/llm-arxiv-2604-04488v1
- Markdown 来源: floors_fallback

---

## [Introduction] Unified Backdoor Defense Framework for Multimodal Large Models: Patch Enhancement + Cross-View Regularization

This paper addresses the backdoor attack problem in multimodal large language models (MLLMs) and proposes a unified defense framework based on patch-level data augmentation and cross-view output difference regularization. It effectively suppresses the success rate of backdoor attacks while maintaining the model's normal text generation capability. This framework provides a new technical solution for multimodal AI security.

## Background: Security Risks and Defense Challenges of Multimodal Models

Multimodal large language models are vulnerable to backdoor attacks during the supervised fine-tuning phase: attackers inject a small number of poisoned samples containing hidden trigger patterns, causing the model to output harmful content when receiving trigger signals. Attacks are characterized by low poisoning rates, hidden triggers, and normal performance on non-trigger inputs. Defense faces a dilemma: it needs to suppress backdoor behavior without impairing normal generation capabilities. Existing single-level defenses struggle to balance these, and cross-modal complexity exacerbates the difficulty.

## Methodology: Core Design of the Unified Defense Framework

The framework is based on two insights: backdoor responses are异常稳定 to non-semantic perturbations; backdoor behavior is reflected in both feature and output layers. Core mechanisms include: 1. Patch-level data augmentation: randomly perturbing image patches (shuffling order, adding local noise, etc.) to destroy trigger patterns; 2. Cross-view regularization: maximizing the output difference between original and perturbed views to force the model to focus on real semantics; 3. Output entropy constraint: ensuring generation diversity and avoiding conservative outputs caused by over-defense.

## Evidence: Effect Analysis from Experimental Validation

Evaluated on three mainstream MLLMs, two tasks (image caption generation, visual question answering), and six backdoor attacks: 1. Attack suppression: significantly reduces the success rate of each attack and is stably effective against different trigger patterns; 2. Normal performance: after defense, the model's performance on benchmark tasks is comparable to that of the undefended model; 3. Cross-attack generalization: effective against multiple attack types and captures the essential features of backdoors.

## Conclusion: Technical Significance and Application Value

The significance of this framework includes: 1. Real-world deployment guarantee: provides support for secure deployment of models in scenarios with low poisoning rates and hidden triggers; 2. Innovative defense concept: identifies backdoors from the perspective of cross-view consistency, with stronger generality; 3. Scalability: modular design is easy to integrate into existing training processes and can be combined with other defense technologies.

## Limitations and Future Research Directions

Current limitations: only targets backdoors in the supervised fine-tuning phase; the effect of backdoor defense in the pre-training phase remains to be verified; computational overhead comes from dual-view output comparison. Future directions: explore efficient view generation strategies; study adaptive perturbation intensity; extend to multimodal models of other modalities such as video and audio.