# Rethinking Jailbreak Detection for Large Vision-Language Models: The Representational Contrastive Scoring (RCS) Approach

> This is an open-source codebase for an ACL 2026 paper, which proposes the Representational Contrastive Scoring (RCS) method for detecting jailbreak attacks on Large Vision-Language Models (LVLMs). It identifies malicious prompts by comparing the differences in model representations between normal inputs and jailbreak inputs.

- 板块: [Openclaw Llm](https://www.zingnex.cn/en/forum/board/openclaw-llm)
- 发布时间: 2026-04-07T10:44:35.000Z
- 最近活动: 2026-04-07T10:51:21.477Z
- 热度: 152.9
- 关键词: 大视觉语言模型, 越狱检测, AI安全, 对比学习, 表征学习, 多模态AI, LLaVA, Qwen-VL, ACL 2026
- 页面链接: https://www.zingnex.cn/en/forum/thread/rcs
- Canonical: https://www.zingnex.cn/forum/thread/rcs
- Markdown 来源: floors_fallback

---

## [Introduction] ACL 2026 Paper: Rethinking Jailbreak Detection for Large Vision-Language Models with the RCS Method

This article introduces the Representational Contrastive Scoring (RCS) method proposed in an open-source ACL 2026 paper, addressing the problem of jailbreak attack detection for Large Vision-Language Models (LVLMs). It identifies malicious prompts by comparing the differences in model representations between normal inputs and jailbreak inputs. The open-source codebase of this method supports mainstream models such as LLaVA and Qwen-VL, aiming to improve detection accuracy and robustness, and promote multimodal AI security research.

## Background: LVLMs' Security Challenges and Limitations of Existing Detection Methods

### Background
Large Vision-Language Models (e.g., GPT-4V, LLaVA, Qwen-VL) combine language and visual capabilities, but face the risk of jailbreak attacks—attackers use text-image combinations to induce models to generate harmful content, making the attack forms more complex.

### Limitations of Existing Methods
- **Output-based detection**: Post-hoc detection, where harm has already occurred;
- **Input pattern-based detection**: Struggles to handle new attack techniques;
- **Perplexity-based detection**: Prone to false positives for normal complex queries;
- **Representation-based detection**: Lacks systematicity, making it hard to distinguish between normal complex inputs and malicious inputs.

## Core Innovation: Detailed Framework of the RCS Method

The RCS method captures representation differences through contrastive learning, with core components including:
1. **Contrastive Sample Construction**: Generate normal versions, perturbed versions, and known jailbreak template samples of the input to be detected;
2. **Multi-Layer Representation Extraction**: Analyze representations from multiple hidden layers of the model to capture early attack impacts;
3. **Contrastive Score Calculation**: Compute the jailbreak score based on cosine similarity, cross-layer consistency, and deviation from the normal distribution;
4. **Adaptive Threshold**: Dynamically adjust the threshold to reduce misjudgments of complex queries.

## Experimental Validation: Multi-Model Support and Key Results

### Supported Models
The codebase supports mainstream LVLMs such as LLaVA-v1.6-Vicuna-7B, Qwen2.5-VL series, InternVL3-8B, and FLAVA.

### Dataset
Uses JailbreakV-28k, custom text-image jailbreak samples, and normal queries.

### Key Results
- Detection accuracy (AUC) is significantly higher than baselines like HiddenDetect;
- Strong cross-model generalization ability;
- Better robustness against adaptive attacks than rule-based methods;
- Optimizes computational efficiency via layer selection heuristics.

## Technical Implementation Details and Deployment Recommendations

### Technical Implementation
The open-source codebase includes:
- Core scripts: `kcd.py` (RCS implementation), `mcd.py` (contrastive variant), and baseline reproduction code;
- Auxiliary tools: feature extractor, caching mechanism, performance analysis tools;
- Experiment management: batch run scripts and visualization analysis code.

### Deployment Considerations
- Integration strategy: Pre-filter + manual review;
- Performance trade-off: Layer selection heuristics balance accuracy and speed;
- Continuous update: Add new jailbreak samples to optimize the model;
- False positive handling: Combine manual review to reduce impact.

## Methodological Contributions and Future Directions

### Methodological Contributions
1. Shift from output-based detection to representation-based detection, enabling preventive security;
2. Innovative application of contrastive learning in the security domain;
3. Systematic method for multi-level representation analysis;
4. Improve detection interpretability (visualized representation comparison).

### Limitations and Future Directions
- Limitations: High computational cost, need to verify effectiveness against adaptive attacks, insufficient multimodal coverage;
- Future directions: Efficient representation extraction, integration of adversarial training, exploration of cross-modal contrastive learning, real-time threshold adjustment.

## Conclusion: RCS's Promotion of LVLMs Security Research

The RCS method represents an important progress in LVLMs security research. It redefines the jailbreak detection problem through representation-level analysis, and its open-source implementation provides a foundation for academia and industry. With the popularization of multimodal AI, this research is expected to promote further development of the field and is worthy of attention from security researchers and engineers.