Repeated Jailbreak Attacks on Multimodal Large Language Models: Security Risks of Vision-Language Alignment

This study explores repeated jailbreak attack methods against Multimodal Large Language Models (MLLMs). By combining adversarial images and text prompts, it tests and bypasses the safety alignment mechanisms of models like MiniGPT4 and mPLUG-Owl2. The research reveals new security challenges introduced by the visual modality, providing references for optimizing AI safety alignment. The original project is from GitHub (maintained by shrrynsh, released on May 27, 2026).

With the development of MLLMs (such as MiniGPT4 and mPLUG-Owl2), their vision-language understanding capabilities have expanded, but they also bring new security risks. Traditional text LLMs face jailbreak attacks (bypassing safety alignment to generate harmful content), while in multimodal scenarios, attackers can break through via image-text combinations.

Alignment mechanisms include supervised fine-tuning, RLHF, and red team testing; common jailbreak attack strategies include role-playing, instruction injection, and code translation. In multimodal scenarios, desynchronization of vision-language alignment, adversarial images, and cross-modal attack surfaces all increase security complexity.

The project targets models like MiniGPT4, mPLUG-Owl2, and Llama2 variants, implementing multiple attack strategies:

1. Multi-prompt attack: Multi-round interactions to gradually guide the model to bypass safety defenses;
2. Adversarial image suffix attack: Adding optimized adversarial suffixes after image input;
3. Multi-image attack: Exploring attention/feature fusion vulnerabilities during multi-image processing;
4. Multi-model attack: Studying the transferability of attacks across models.

Technical details: Using the PGD algorithm to generate adversarial images (parameters such as 25 training samples, step size 16, 200 iterations), and optimizing text prompt templates (instructions, role settings, etc.).

The project's evaluation dimensions include:

• Success rate: The proportion of harmful content generated by the attack;
• Transferability: The effectiveness of the attack across different models;
• Stealthiness: Whether adversarial perturbations are perceptible to the human eye;
• Robustness: The effect of defense mechanisms (such as input purification).

The tested target models include MiniGPT4, mPLUG-Owl2, and multimodal variants based on Llama2.

The value of such research lies in red team testing (identifying vulnerabilities in advance), improving alignment mechanisms, risk assessment, and developing defense strategies. Responsible practices must be followed:

• Testing in controlled environments to avoid abuse;
• Disclosing vulnerabilities to developers in a timely manner;
• Focusing research on defense;
• Transparently explaining objectives and methods.

Defense measures against attacks include:

1. Input purification: Image preprocessing (filtering, compression), text filtering, and multimodal consistency checks;
2. Model level: Adversarial training, enhanced security layers, and multimodal alignment consistency;
3. System level: Output review, human-machine collaboration (manual review of high-risk queries), and function restrictions (such as long conversations, multi-image input).

This research reveals the fact that the expansion of AI capabilities is accompanied by the expansion of risks. While the visual modality introduces new application scenarios, it also brings new security challenges. The value of the research lies in understanding boundary vulnerabilities and helping to build robust AI systems.

Insights for developers: Safety alignment should run through the entire development cycle, not as an afterthought. AI security research needs cross-modal perspectives, collaboration between attack and defense, standardized evaluation, and continuous monitoring to ensure that technology benefits humanity.