Exploration and Limitations of Multimodal Models in Identity Document Presentation Attack Detection

This article explores the application of large multimodal models such as Paligemma, Llava, and Qwen in identity document presentation attack detection (PAD), finding that these general-purpose models perform poorly in this security task, analyzing the reasons, and pointing out future improvement directions.

With the popularization of digital identities, identity document verification has become a core link, but it faces the threat of presentation attacks (forged photos/screen displays/printed copies to deceive systems). Traditional PAD relies on visual feature analysis and gradually shows limitations in the face of sophisticated forgeries.

The study explores applying multimodal models such as Paligemma (lightweight document understanding), Llava (visual question answering), and Qwen-VL (excellent in Chinese scenarios) to PAD. The input includes identity document images and text metadata, expecting to fuse visual features and semantic clues to improve detection effectiveness.

The experiment found that these models, which perform well in general visual-language tasks, are ineffective in distinguishing real documents from presentation attacks. Although they can recognize text and layout, they cannot effectively judge authenticity.

1. Pre-training data lacks specialized identity document presentation attack samples; 2. Models focus on semantic content ("what it is"), while PAD needs to judge physical authenticity ("whether it is real"); 3. Models do not understand the unique security elements of documents (holograms, microtext, etc.).

General capability ≠ specialized capability; targeted fine-tuning and domain data are needed; security tasks require considering special evaluation criteria such as adversarial robustness and false positive costs.

1. Build a specialized multimodal dataset for identity document PAD; 2. Design pre-training objectives for authenticity judgment; 3. Fuse multimodal features with traditional PAD technologies; 4. Adversarial training to improve model robustness.