JASMINE: An Autism Spectrum Disorder Screening System Based on Pose Estimation

JASMINE is an open-source Autism Spectrum Disorder (ASD) screening assistance system based on 2D pose estimation technology, designed to help medical professionals efficiently and objectively detect ASD indicators in children. Its core features include a privacy-preserving design (only processes skeletal key point coordinates, no raw video storage), multi-dimensional feature engineering and integrated models (combining machine learning and deep learning), providing a reliable auxiliary tool for early ASD screening.

JASMINE takes privacy protection as its core design concept: it only processes the 25 joint coordinates extracted from videos and does not store raw video frames or images; it supports completing all processing on local devices to avoid sensitive data transmission risks; it complies with medical data privacy regulations and is suitable for deployment in medical institutions, schools, or home environments, with no need to worry about identity leakage issues.

The technical architecture covers a complete pipeline: Video Input → OpenPose/MediaPipe Pose Estimation → 25 Joint Extraction (normalized coordinates) → Multi-dimensional Feature Engineering (kinematics: joint angles, speed, distance, symmetry; statistics: mean, standard deviation, temporal dynamics, frequency domain analysis) → Integrated Models (Random Forest, SVM, LSTM, Transformer) → Prediction Results. The models improve reliability through voting or weighted averaging.

The system is trained and validated based on the MMASD benchmark dataset, which includes: CSV format key point files (25-26 joints ×3 coordinates per frame), normalized coordinates [0,1], action types and ASD status labels, OpenPose JSON outputs, and subject metadata. The standardized format facilitates reproduction and expansion.

The application interface is built using Streamlit, including a homepage (system overview), model comparison (performance metrics/confusion matrix), inference testing (upload data to get prediction results), and pose visualization (frame-by-frame skeleton viewing); the tech stack is Python, relying on OpenCV, MediaPipe, PyTorch, Scikit-learn, etc., with a modular code structure (config, data, features, models, visualization).

Currently, JASMINE is a research demonstration project, not a clinical diagnostic tool, and its results cannot be directly used for decision-making. Future directions: integrate 3D pose estimation, expand non-MMASD datasets, support real-time video streams, develop MediaPipe real-time pipelines, add permission systems, and generate automated reports.

JASMINE demonstrates the application potential of AI in the medical field. Through privacy-first design, multi-dimensional features, and integrated models, it provides objective and efficient assistance for ASD screening. Its open-source nature and interpretability set an example for AI medical applications, while emphasizing the importance of patient privacy and medical safety, making it an excellent starting point for researchers entering the AI medical field.