Continuous Multimodal Facial Authentication System: Detecting Deepfakes Using 'Biometric Inconsistency'

This project proposes an innovative continuous multimodal facial authentication framework. Using dual-path 3D-CNN and Model-Agnostic Meta-Learning (MAML) technologies, it detects the temporal asynchrony of biometric features between the eye and lip regions, effectively identifying deepfake videos. The core idea shifts from traditional pixel artifact detection to 'biometric inconsistency' recognition, offering advantages such as tool independence and high data efficiency.

With the development of generative AI, the quality of deepfake videos has improved, and traditional detection methods based on pixel-level artifacts are prone to failure due to video compression or resolution adjustments. This project shifts its approach: instead of looking for pixel traces, it detects 'biometric inconsistency' between different facial regions (e.g., eyes and lips)—deepfakes struggle to simulate the physiological coordination between real human facial regions.

The system uses a dual-path fusion architecture to independently process eye and lip movement dynamics:

1. Optical Flow Feature Extraction: The Farneback algorithm is used to extract dense optical flow features, highlighting motion information and suppressing irrelevant interference;
2. Dual-Path Processing: The eye path focuses on eye movement, blink frequency, etc., while the lip path focuses on lip opening/closing changes. Each path uses an independent 3D-CNN to learn temporal features;
3. Synthetic Training: Artificially apply time shifts to the eye/lip paths of real videos to generate 'pseudo-fake' samples, allowing the model to learn the essence of inconsistency and achieve tool independence and data efficiency.

• MAML Application: Through Model-Agnostic Meta-Learning, the model can quickly adapt to new users' facial dynamics from a small number of registered videos, reducing deployment costs;
• Real-Time System: The backend uses FastAPI + PyTorch to implement a WebSocket server (supports 30FPS), a LIFO queue (to process the latest frames), and parallel AI worker threads; the frontend uses React + Vite to provide a real-time dashboard, HUD interface, and attack simulation functions.

Evaluation results on the GRID and MOBIO datasets show:

Method	Dataset	Deepfake Tool	Detection Area	Accuracy	Computational Cost
This System	GRID	Synthetic Inconsistency	Joint	100%	Medium (~0.6M parameters)
This System	MOBIO	Synthetic Inconsistency	Joint	96.63%	Medium (~0.6M parameters)
Compared to methods like XceptionNet (~96% accuracy, 23M parameters), this system has a smaller parameter count (0.6M) yet achieves better or comparable performance, demonstrating the efficiency of the architecture.

Application Scenarios: Remote identity authentication (bank account opening, government service processing), video conference security, social media moderation, interview proctoring, etc.; Challenges: Environmental factors such as lighting, angle, and occlusion affect performance; real-time optical flow calculation and WebSocket transmission require certain hardware support.

This project represents an important advancement in the field of deepfake detection. By combining the biometric inconsistency approach with dual-path 3D-CNN, synthetic training, and MAML technologies, it achieves efficient and lightweight detection capabilities. In the future, it can be integrated with hardware-level security mechanisms (such as Trusted Execution Environments) to continuously evolve and respond to the development of forgery technologies.