Lightweight Multimodal Deception Detection Model: Towards an Efficient, Interpretable Unified Architecture

This article proposes a lightweight multimodal deception detection model. Through a unified architecture, it achieves deep fusion of text, speech, and visual signals. While ensuring detection accuracy, it significantly reduces computational overhead and improves the model's interpretability and adaptability. It addresses issues such as large size and difficult deployment of existing multimodal models, making it suitable for edge devices and real-time scenarios.

Traditional deception detection relies on a single modality, which is vulnerable to adversarial attacks and struggles to capture multi-dimensional deception features. Existing multimodal LLMs are bulky and have high computational overhead, limiting their application in edge devices and real-time scenarios. Therefore, developing a lightweight unified multimodal deception detection model has become an urgent need.

Core Design Principles: Lightweight (model compression, knowledge distillation, etc.), unified multimodal fusion (end-to-end architecture), enhanced interpretability (attention visualization), dynamic adaptability (adaptive learning module). Technical Architecture: Multimodal feature extraction layer (text/speech/visual encoders), cross-modal bidirectional cross-attention fusion, lightweight strategies (knowledge distillation, dynamic inference path, quantization and pruning).

Datasets: Public datasets covering multiple domains (e.g., court testimony, interviews) and multiple deception types. Results: F1 score increased by 12-18% compared to single-modal baselines; inference speed improved by 5x, memory usage reduced by over 70%; can locate key evidence (e.g., text words, speech pauses, facial micro-expressions); good cross-domain generalization ability, requiring only a small amount of domain adaptation to transfer to new scenarios.

Security and Justice: Real-time early warning on portable devices, with interpretability meeting regulatory requirements; Finance and Business: Integrated into mobile applications to provide low-cost risk control tools; Human-Computer Interaction: Runs on embedded platforms to enhance the interaction security of virtual assistants.

Limitations: Fairness under cultural differences needs verification, insufficient defense against adversarial attacks, privacy protection issues to be resolved; Future: Self-supervised pre-training to improve generalization, federated learning to protect privacy, causal reasoning to enhance out-of-distribution stability.

The research successfully balances accuracy, efficiency, and interpretability. Insights: Multimodal fusion needs to focus on effective information interaction; lightweight and interpretability should be first-level design goals; AI systems need to integrate technical performance, deployment costs, and ethical constraints.