Inter-Stance: Release of a Multimodal Dyadic Corpus for Dialogue Stance Analysis

Binghamton University, State University of New York, and other institutions have released the Inter-Stance dataset. This is a multimodal dyadic interaction corpus for dialogue stance analysis, containing 45 pairs of participants (90 people). It synchronously collects 2D/3D facial videos, thermal imaging, speech, and multiple physiological signals, providing an unprecedented research resource for computational social science and affective computing, and addressing the gap in dyadic multimodal interaction data in existing datasets.

Human social interaction is a complex multimodal process, where language, facial expressions, physiological responses, etc., convey attitudes and emotions, and there exists a 'dyadic effect'. However, existing datasets have limitations: either single-person and single-modal, or dyadic interactions lack rich sensors. There is no public dataset that simultaneously provides dyadic multimodal records (including 2D/3D videos, thermal imaging, physiological signals) and self-report data, which restricts the development of computational modeling for interpersonal interactions.

Core scale of the dataset: 45 pairs of participants (including acquaintances and strangers), 270 multimodal sequences, over 1400 minutes of interaction data, 20TB total open sharing. Collection configuration covers:

• Visual: 2D HD video, 3D facial geometry, thermal imaging video and temperature data
• Speech and language: High-fidelity audio, synchronized transcribed text
• Physiological signals: Autonomic nervous system indicators such as PPG, EDA, HR, BP, RR The synchronous collection capability allows exploration of the correlation between external behavior and internal physiological states.

An IRB-approved experimental scheme was adopted, using dialogue tasks (on socially divisive topics) to induce natural stance behaviors. Annotations include three core stance categories: agreement, disagreement, and neutrality, as well as annotations of social signals and emotional synchrony, providing materials for studying interpersonal emotional influences.

Inter-Stance has obvious advantages over existing datasets:

Dataset	Number of Dyads	Participants	Modality	Key Limitations
IEMOCAP	5 pairs	10 actors	Audio, video, motion capture	Lacks 3D facial geometry and thermal imaging
RECOLA	23 pairs	46 students	Audio, video, ECG, EDA	Remote video setup; lacks 3D and thermal imaging
HMI-Mimicry	54 pairs	12 confederates +48 participants	Audio, video	Lacks 3D, thermal imaging and physiological data
BP4D+	-	Single person	2D/3D video, thermal imaging, physiological signals	Only single-person collection; no dyadic interaction
Inter-Stance	45 pairs	90 people	2D/3D video, thermal imaging, speech, text, PPG, EDA, HR, BP, RR	Complete dyadic multimodal collection
It is the first to combine dyadic interaction with rich multimodal collection (3D, thermal imaging, physiological signals).

The dataset brings value to multiple domains:

1. Multimodal stance detection: Explore the collaborative expression of stance through visual/physiological/language signals, aiding the development of socially aware AI;
2. Interpersonal emotional synchrony modeling: Analyze emotional contagion, behavioral mimicry, and physiological synchrony;
3. Social signal processing: Identify subtle social signals in agreement/disagreement/neutral states;
4. Mental health research: Study behavioral characteristics of patients with mental disorders through interactions between acquaintances and strangers.

It brings new challenges: integrating 20TB of multimodal heterogeneous data, modeling dyadic dynamic influences, extracting high-dimensional temporal stance features, which promotes the development of technologies such as multimodal machine learning. Methodologically, the IRB-approved scheme successfully induced natural stance behaviors, providing a reference paradigm for data collection in affective computing.

The release of Inter-Stance marks a new stage in multimodal social interaction research. It provides key data resources for fields such as computational social science, affective computing, and human-computer interaction, lays the foundation for building socially aware AI systems, and is an important resource for researchers in computational modeling of social behaviors.