Abstract
Background:
In recent years, advances in technology have led to an influx of mental health apps, in particular the development of mental health and well-being chatbots, which have already shown promise in terms of their efficacy, availability, and accessibility. The ChatPal chatbot was developed to promote positive mental well-being among citizens living in rural areas. ChatPal is a multilingual chatbot, available in English, Scottish Gaelic, Swedish, and Finnish, containing psychoeducational content and exercises such as mindfulness and breathing, mood logging, gratitude, and thought diaries.
Objective:
The primary objective of this study is to evaluate a multilingual mental health and well-being chatbot (ChatPal) to establish if it has an effect on mental well-being. Secondary objectives include investigating the characteristics of individuals that showed improvements in well-being along with those with worsening well-being and applying thematic analysis to user feedback.
Methods:
A pre-post intervention study was conducted where participants were recruited to use the intervention (ChatPal) for a 12-week period. Recruitment took place across 5 regions: Northern Ireland, Scotland, the Republic of Ireland, Sweden, and Finland. Outcome measures included the Short Warwick-Edinburgh Mental Well-Being Scale, the World Health Organization-Five Well-Being Index, and the Satisfaction with Life Scale, which were evaluated at baseline, midpoint, and end point. Written feedback was collected from participants and subjected to qualitative analysis to identify themes.
Results:
A total of 348 people were recruited to the study (n=254, 73% female; n=94, 27% male) aged between 18 and 73 (mean 30) years. The well-being scores of participants improved from baseline to midpoint and from baseline to end point; however, improvement in scores was not statistically significant on the Short Warwick-Edinburgh Mental Well-Being Scale (P=.42), the World Health Organization-Five Well-Being Index (P=.52), or the Satisfaction With Life Scale (P=.81). Individuals that had improved well-being scores (n=16) interacted more with the chatbot and were significantly younger compared to those whose well-being declined over the study (P=.03). Three themes were identified from user feedback, including “positive experiences,” “mixed or neutral experiences,” and “negative experiences.” Positive experiences included enjoying exercises provided by the chatbot, while most of the mixed, neutral, or negative experiences mentioned liking the chatbot overall, but there were some barriers, such as technical or performance errors, that needed to be overcome.
Conclusions:
Marginal improvements in mental well-being were seen in those who used ChatPal, albeit nonsignificant. We propose that the chatbot could be used along with other service offerings to complement different digital or face-to-face services, although further research should be carried out to confirm the effectiveness of this approach. Nonetheless, this paper highlights the need for blended service offerings in mental health care.
In recent years, advances in technology have led to an influx of mental health apps, in particular the development of mental health and well-being chatbots, which have already shown promise in terms of their efficacy, availability, and accessibility. The ChatPal chatbot was developed to promote positive mental well-being among citizens living in rural areas. ChatPal is a multilingual chatbot, available in English, Scottish Gaelic, Swedish, and Finnish, containing psychoeducational content and exercises such as mindfulness and breathing, mood logging, gratitude, and thought diaries.
Objective:
The primary objective of this study is to evaluate a multilingual mental health and well-being chatbot (ChatPal) to establish if it has an effect on mental well-being. Secondary objectives include investigating the characteristics of individuals that showed improvements in well-being along with those with worsening well-being and applying thematic analysis to user feedback.
Methods:
A pre-post intervention study was conducted where participants were recruited to use the intervention (ChatPal) for a 12-week period. Recruitment took place across 5 regions: Northern Ireland, Scotland, the Republic of Ireland, Sweden, and Finland. Outcome measures included the Short Warwick-Edinburgh Mental Well-Being Scale, the World Health Organization-Five Well-Being Index, and the Satisfaction with Life Scale, which were evaluated at baseline, midpoint, and end point. Written feedback was collected from participants and subjected to qualitative analysis to identify themes.
Results:
A total of 348 people were recruited to the study (n=254, 73% female; n=94, 27% male) aged between 18 and 73 (mean 30) years. The well-being scores of participants improved from baseline to midpoint and from baseline to end point; however, improvement in scores was not statistically significant on the Short Warwick-Edinburgh Mental Well-Being Scale (P=.42), the World Health Organization-Five Well-Being Index (P=.52), or the Satisfaction With Life Scale (P=.81). Individuals that had improved well-being scores (n=16) interacted more with the chatbot and were significantly younger compared to those whose well-being declined over the study (P=.03). Three themes were identified from user feedback, including “positive experiences,” “mixed or neutral experiences,” and “negative experiences.” Positive experiences included enjoying exercises provided by the chatbot, while most of the mixed, neutral, or negative experiences mentioned liking the chatbot overall, but there were some barriers, such as technical or performance errors, that needed to be overcome.
Conclusions:
Marginal improvements in mental well-being were seen in those who used ChatPal, albeit nonsignificant. We propose that the chatbot could be used along with other service offerings to complement different digital or face-to-face services, although further research should be carried out to confirm the effectiveness of this approach. Nonetheless, this paper highlights the need for blended service offerings in mental health care.
| Original language | English |
|---|---|
| Article number | e43051 |
| Pages (from-to) | 1-14 |
| Number of pages | 15 |
| Journal | Journal of Medical Internet Research |
| Volume | 25 |
| Early online date | 6 Jul 2023 |
| DOIs | |
| Publication status | Published online - 6 Jul 2023 |
Bibliographical note
Funding Information:The ChatPal consortium acknowledges the support provided by the Interreg VB Northern Periphery Arctic Programme under the grant for Conversational Interfaces Supporting Mental Health and Well-being of People in Sparsely Populated Areas (ChatPal) project number 345. The authors would like to thank all the clients, participants, project members, supporters, and researchers at Ulster University, the University of Eastern Finland, Norrbotten Association of Local Municipalities, Region Norrbotten, Luleå University of Technology, NHS Western Isles, Action Mental Health, Munster Technological University, and Health Innovation Hub Ireland for participating in this research.
Funding Information:
The ChatPal project [20], funded by the Northern Periphery and Arctic (NPA) Programme, involves the design, development, and trialing of a mental health chatbot to support the well-being of individuals in rural NPA areas [21]. Mental health care professionals were surveyed to establish their views toward health-related chatbots [22], and workshops were held across NPA regions to gather user needs [5]. The ChatPal chatbot was subsequently developed based on use cases that professionals endorse and to meet the needs of end users. The chatbot is a mental health promotion tool available in 4 languages, including English, Scottish Gaelic, Swedish, and Finnish. The chatbot is not designed to diagnose or treat those with severe mental illness but instead helps promote good mental health and well-being. This paper focuses on the evaluation of a pre-post intervention study for the ChatPal chatbot.
Publisher Copyright:
©Courtney Potts, Frida Lindström, Raymond Bond, Maurice Mulvenna, Frederick Booth, Edel Ennis, Karolina Parding, Catrine Kostenius, Thomas Broderick, Kyle Boyd, Anna-Kaisa Vartiainen, Heidi Nieminen, Con Burns, Andrea Bickerdike, Lauri Kuosmanen, Indika Dhanapala, Alex Vakaloudis, Brian Cahill, Marion MacInnes, Martin Malcolm, Siobhan O'Neill.
Keywords
- conversational user interfaces;
- digital interventions
- Warwick-Edinburgh Mental Well-Being Scale
- Satisfaction With Life Scale;
- World Health Organization-Five Well-Being Index Scale
- mental health
- apps
- health care
- mixed methods
- conversation agent
- mental well-being
- digital health intervention
- conversational user interfaces
- Satisfaction With Life Scale
