Passive Assessment of Depression Severity from AI Mental Health Conversations: A Large Model Fine-Tuning Approach

This article introduces a passive assessment method for depression severity based on large language models (LLMs), aiming to address issues such as low completion rates and selection bias in traditional self-report scales (e.g., PHQ-9). By analyzing daily conversations between users and AI mental health applications, this method enables continuous symptom monitoring without additional questionnaires. The study uses real user data, combined with a pseudo-label data augmentation strategy, to fine-tune the open-source model Qwen3.5-27B, achieving good predictive performance and opening up new directions for mental health monitoring.

Depression is the leading cause of disability globally, and early identification is crucial. However, traditional scales like PHQ-9 have limitations: low completion rates, selection bias, low temporal resolution, and additional burden. An ideal monitoring approach should be passive—automatically collecting data while users use services normally, enabling continuous monitoring, reducing burden, and minimizing bias.

1. Data foundation: Conversation data from 3111 users + PHQ-9 labels; 2. Data augmentation: Using Claude Opus to generate pseudo-labels, expanding to 6283 users; 3. Model architecture: Qwen3.5-27B base model + regression head to predict PHQ-9 total score; 4. Training strategy: Supervised learning fine-tuning to optimize prediction accuracy.

On the test set of 842 users, the model performance is as follows: MAE=2.6 points (small average deviation), RMSE=4.0 points, Pearson correlation coefficient r=0.80 (strong correlation), AUC=0.91 at the PHQ-9≥10 threshold (good classification ability). The AUC exceeds 0.87 across all severity thresholds, covering the entire clinical spectrum.

Contributions: The first large-scale real-user validation study, pseudo-label data augmentation strategy, and open-source model achieving commercial model performance. Application prospects: Continuous symptom monitoring, early warning, personalized intervention, and auxiliary research data collection.

Ethical issues: Informed consent, data privacy, clinical boundaries (cannot replace professional diagnosis), and intervention responsibility. Limitations: Sample representativeness (users of specific applications), dependence on conversation quality, lack of longitudinal validation, and unclear causal relationships. Future directions: Validation in diverse populations, interpretability research, expansion to other mental conditions, and long-term longitudinal studies.

Insights: Reassessing the value of conversation data, shifting product design to unobtrusive monitoring, and integrating auxiliary tools into clinical practice. Conclusion: This method represents progress in mental health monitoring technology; it is necessary to balance technological innovation and ethical responsibility, providing valuable experience for the industry.