From Vision to Text: A Compact Multimodal Approach for ID Card Presentation Attack Detection

This study addresses challenges such as cross-domain generalization and data scarcity in ID card presentation attack detection (PAD) by proposing a compact multimodal model that combines vision and text, achieving robust detection through generative and discriminative modules. The study finds that the model exhibits strong cross-domain generalization after supervised fine-tuning but performs poorly in zero-shot settings, emphasizing the critical role of real data in ensuring model reliability and providing a new direction for authentication security.

Challenges in ID Card Presentation Attack Detection

With the popularity of digital identity verification, ID cards have become important credentials, but presentation attacks (e.g., printed photos, screen displays, 3D masks) threaten security. PAD technology needs to identify forgeries but faces three major challenges:

1. Cross-domain generalization problem: Large differences exist between model training and deployment environments; privacy restrictions lead to a lack of real data, resulting in decreased cross-domain performance;
2. Data scarcity: Privacy regulations (such as GDPR) limit the collection of large-scale real data, relying on synthetic/small-scale data;
3. Diversity of attack methods: From simple printing to complex 3D masks, attack features are diverse, requiring models to generalize and identify unknown types.

Core Idea of the Multimodal Approach

ID cards contain visual (image quality, texture) and textual (name, ID number) information; fusing the two can complement each other:

• Complementary information: Vision captures physical characteristics, while text verifies content rationality;
• Attack robustness: Attacks are difficult to replicate reasonable text (e.g., ID card check digits);
• Cross-domain stability: Text is not affected by cameras/lighting, improving cross-domain generalization.

Model Architecture

A generative module and a discriminative module are designed:

Generative Module

• Feature encoder: Encodes images into compact visual features;
• Text detection and recognition: Locates and recognizes text regions;
• Feature enhancement: Enhances attack-sensitive features.

Discriminative Module

• Cross-modal fusion: Deeply fuses visual and textual features;
• Consistency verification: Verifies the consistency between visual and textual content;
• Attack classification: Determines whether it is a presentation attack.

Compact Design

The number of parameters is much smaller than traditional large models, suitable for real-time operation on edge devices.

Key Experimental Findings

1. Strong generalization after supervised fine-tuning: The multimodal model shows strong cross-domain generalization after supervised fine-tuning, proving the value of fusion and the effectiveness of the compact design;
2. Failure in zero-shot settings: Poor performance in zero-shot settings requires domain-specific supervision signals, and general pre-training is insufficient;
3. Importance of real data: Subtle differences in real data (paper texture, printing quality) are crucial for robust detection;
4. Limitations of synthetic data: Synthetic data cannot reflect real challenges, and evaluation based on it may overestimate actual performance.

Technical Significance and Industry Impact

1. New direction for multimodal security: Demonstrates the value of vision-text fusion in document verification, which can be extended to passports, driver's licenses, and other scenarios;
2. Call for data quality: Emphasizes the gap between synthetic and real data, calling for the construction of real and diverse datasets;
3. Practical deployment guidance: Zero-shot deployment is not feasible, requiring domain fine-tuning; model capacity should match data volume; cross-domain performance needs to be verified with real data.

Limitations and Future Directions

Limitations

• Data constraints: Privacy regulations lead to insufficient data;
• Attack coverage: Mainly focuses on known attacks, and robustness to unknown attacks needs to be verified;
• Fusion strategy: Current fusion is relatively simple;
• Real-time performance: Optimization is needed for scenarios with extremely high throughput.

Future Directions

• Explore federated learning and differential privacy technologies to utilize more data;
• Improve robustness to new unknown attacks;
• Optimize cross-modal attention mechanisms;
• Further enhance real-time performance.

Research Summary

This study proposes a compact multimodal approach combining vision and text for ID card presentation attack detection. Through generative and discriminative modules, the model exhibits strong cross-domain generalization after supervised fine-tuning but performs poorly in zero-shot settings. The study emphasizes the critical role of real data in model reliability, calls for re-evaluating synthetic data benchmarks, and provides guidance for building more robust authentication systems.