Deep-VRM: Deep Residual Injection for Full-Spectrum Forensic Signal Perception in Multimodal Large Language Models

This article introduces Deep-VRM, a paper accepted by ICML 2026. The technology enhances the forensic signal perception capability of multimodal large language models (MLLMs) through a deep residual injection mechanism, implements two-stage training based on Qwen2.5-VL, and provides new ideas for AI-generated content detection and multimedia forensics.

Original Author/Maintainer: KQL11 Source Platform: GitHub Original Title: Deep-VRM: Deep Residual Injection for Full-Spectrum Forensic Signal Perception in Multimodal Large Language Models Original Link: https://github.com/KQL1/Deep-VRM Source Publication Time/Update Time: 2026-05-25

With the rapid development of generative AI technology, multimodal large language models (MLLMs) perform excellently in tasks like image understanding, but the demand to distinguish between real and AI-generated content is increasingly urgent. The proliferation of deepfake technology has made multimedia forensics a focus.

Traditional forensic methods are designed for specific tampering techniques and struggle to cope with rapidly iterating generative models; existing MLLMs excel at high-level semantic understanding but lack sensitivity to subtle forensic clues hidden in images (such as compression traces, noise patterns, generation artifacts, etc.).

Deep-VRM enables MLLMs to have full-spectrum forensic signal perception capability through a deep residual injection mechanism:

• Full-spectrum perception: Captures multi-band clues such as low-frequency (overall structural anomalies), medium-frequency (unnatural texture boundaries), and high-frequency (abnormal noise distribution)

Two-stage training strategy based on Qwen2.5-VL:

1. Base model training: Uses standard visual instruction fine-tuning data to establish visual-language alignment capability
2. Residual injection training: Introduces the DeepVRM module, injects low-level visual features via residual connections, including residual feature extraction, multi-scale fusion, and adaptive injection (gating mechanism controls intensity)

Inferred from the code repository structure: Adopts a modular architecture, supporting efficient training with the ms-swift framework.

Evaluation will cover the following tasks:

• Generated image detection: Distinguish between real photos and AI-generated images
• Tampering detection: Locate tampered areas like splicing and copy-paste
• Deepfake detection: Identify traces of face-swapped videos/voice forgery
• Multimodal consistency verification: Detect consistency between images and text descriptions

The full-spectrum perception feature of Deep-VRM gives it potential advantages in fine-grained analysis scenarios.

The project provides complete training and inference scripts:

• run_Stage1.sh: First-stage training script
• run_Stage2.sh: Second-stage residual injection training script
• Models/DeepVRM/: Core model implementation
• ms-swift/: Swift training framework integration

Supports parameter-efficient fine-tuning methods (e.g., LoRA, QLoRA), and the modular design facilitates reproduction and extension.

Limitations:

1. Training data and model weights have not been made public yet
2. Cross-domain generalization ability (unseen generation/tampering techniques) needs verification
3. Computational overhead caused by residual injection needs optimization

Future Directions:

• Explore lightweight residual injection architectures
• Extend to video forensics scenarios
• Develop interpretability tools
• Establish a unified benchmark testing platform

Deep-VRM combines fine-grained forensic signal perception with powerful semantic understanding, opening up new directions for AI-generated content detection and multimedia forensics.

It provides technical references for AI security, content moderation, and digital forensics fields. The open-source code contributes a reproducible foundation to the community, and we look forward to the complete version driving the development of the field.