Performance results from app-based mental health assessment using machine learning on voice biomarker data

Introduction

This work reports on the results from a 12-week study testing an app-based voice biomarker mental health assessment. The study population were young adults aged 16-24, with weekly assessments conducted at the participants' convenience. Participants were recruited using social media, schools, mental health charities and universities. The young people were able to self-identify as mentally well or struggling with their mental health.

Methods

The study consisted of 12 weekly assessments, with an average assessment lasting 10 minutes. During an assessment the participants were prompted with the question "How has your day been?" to gather spontaneous voice data for 40 seconds. Participants then completed the Patient Healthcare Questionnaire-9 (PHQ-9) to evaluate major depressive disorder (MDD), the Generalized Anxiety Disorder-7 (GAD-7) for anxiety assessment, and the DSM-5 Cross-Cutting Symptom Measure, which evaluates a wide range of mental health symptoms. These mental health scale results were correlated with results from vocal biomarker focused machine learning detection models for depression and anxiety. Participants also provided demographic and mental health information as part of the consent process. The information derived from this was used to assess the model performance in the various demographic and mental health subgroups. Results The demographic factors were found to have little effect on model performance. However, significant differences in the detection model's performance were observed as the severity of depression and anxiety increased. The severity thresholds of the PHQ-9 and GAD-7 scales were used for this analysis. Depression detection model performance decreased when severity increased. The ‘None’ depressed category performed better than the ‘Moderate’ depression category. We also found comorbid anxiety impacted the accuracy of the depression detection model. Depression model accuracy was higher when there was ‘minimal’ anxiety compared to performance in the presence of ‘moderate’ anxiety. For the anxiety detection model, performance decreased as anxiety severity increased. There were significant differences in model performance between participants with ‘Minimal’ anxiety and those with 'Moderate’ anxiety. Similarly, the results showed that the anxiety model accuracy was impacted by comorbid depression, with increases in depression severity decreasing anxiety detection model performance. For the ‘None’ depressed category accuracy was higher when compared to ‘moderate’ depression.

Conclusion

The models used to detect anxiety and depression were impacted by mental health factors, but less so by demographic factors. These results show model performance for both disorders is negatively impacted by the severity of the target disorder and other comorbid disorders.