Insights and lessons learned from trialling a mental health chatbot in the wild

Courtney Potts; RR Bond; Maurice Mulvenna; Edel Ennis; Andrea Bickerdike; Edward Coughlan; Thomas Broderick; Con Burns; Michael McTear; Lauri Kuosmanen; Heidi Nieminen; Kyle Boyd; Brian Cahill; Alex Vakaloudis; Indika Dhanapala; Anna-Kaisa Vartiainen; Catrine Kostenius; Martin Malcolm

doi:10.1109/ISCC53001.2021.9631395

Insights and lessons learned from trialling a mental health chatbot in the wild

Courtney Potts, RR Bond, Maurice Mulvenna, Edel Ennis, Andrea Bickerdike, Edward Coughlan, Thomas Broderick, Con Burns, Michael McTear, Lauri Kuosmanen, Heidi Nieminen, Kyle Boyd, Brian Cahill, Alex Vakaloudis, Indika Dhanapala, Anna-Kaisa Vartiainen, Catrine Kostenius, Martin Malcolm

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

3 Citations (Scopus)

33 Downloads (Pure)

Abstract

This study reports on the development and 'in the wild' trialling of a chatbot (ChatPal) which promotes good mental wellbeing. A stakeholder-centered approach for design was adopted where end users, mental health professionals and service users were involved in the design which was centered around positive psychology. In the wild usage of the chatbot was investigated from Jul-20-Mar-21. Exploratory analyses of usage metrics were carried out using the event log data. User tenure, unique usage days, total chatbot interactions and average daily interactions were used in K-means clustering to identify user archetypes. The chatbot was used by a variety of age groups (18-65+) and genders, mainly those living in Ireland. K-means clustering identified three clusters: sporadic users (n=4), frequent transient users (n=38) and abandoning users (n=169) each with distinct usage characteristics. This study highlights the importance of event log data analysis for making improvements to the mental health chatbot.

Original language	English
Title of host publication	26th IEEE Symposium on Computers and Communications, ISCC 2021
Publisher	IEEE Xplore
Pages	1-6
Number of pages	6
ISBN (Electronic)	978-1-6654-2744-9
ISBN (Print)	978-1-6654-2745-6
DOIs	https://doi.org/10.1109/ISCC53001.2021.9631395
Publication status	Published (in print/issue) - 15 Dec 2021
Event	ICTS4eHealth - Online conference, Athens, Greece Duration: 5 Sept 2021 → 8 Sept 2021 https://icts4ehealth.icar.cnr.it/

Publication series

Name	Proceedings - IEEE Symposium on Computers and Communications
Volume	2021-September
ISSN (Print)	1530-1346

Conference

Conference	ICTS4eHealth
Country/Territory	Greece
City	Athens
Period	5/09/21 → 8/09/21
Internet address	https://icts4ehealth.icar.cnr.it/

Bibliographical note

Funding Information:
The ChatPal consortium acknowledges the support provided by the Interreg VB Northern Periphery & Arctic Programme under the grant for Conversational Interfaces Supporting Mental Health and Wellbeing of People in Sparsely Populated Areas (ChatPal) project number 345.

Publisher Copyright:
© 2021 IEEE.

Keywords

Conversational user interfaces
event log
eHealth
mental wellbeing
co-design
COVID-19

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1109/ISCC53001.2021.9631395

Accepted author manuscriptAuthor Accepted Manuscript, 520 KBLicence: Unspecified

Cite this

Potts, C., Bond, RR., Mulvenna, M., Ennis, E., Bickerdike, A., Coughlan, E., Broderick, T., Burns, C., McTear, M., Kuosmanen, L., Nieminen, H., Boyd, K., Cahill, B., Vakaloudis, A., Dhanapala, I., Vartiainen, A.-K., Kostenius, C., & Malcolm, M. (2021). Insights and lessons learned from trialling a mental health chatbot in the wild. In 26th IEEE Symposium on Computers and Communications, ISCC 2021 (pp. 1-6). (Proceedings - IEEE Symposium on Computers and Communications; Vol. 2021-September). IEEE Xplore. https://doi.org/10.1109/ISCC53001.2021.9631395

@inproceedings{7437d307aaaf49febb6fc34b7a0b09c5,

title = "Insights and lessons learned from trialling a mental health chatbot in the wild",

abstract = "This study reports on the development and 'in the wild' trialling of a chatbot (ChatPal) which promotes good mental wellbeing. A stakeholder-centered approach for design was adopted where end users, mental health professionals and service users were involved in the design which was centered around positive psychology. In the wild usage of the chatbot was investigated from Jul-20-Mar-21. Exploratory analyses of usage metrics were carried out using the event log data. User tenure, unique usage days, total chatbot interactions and average daily interactions were used in K-means clustering to identify user archetypes. The chatbot was used by a variety of age groups (18-65+) and genders, mainly those living in Ireland. K-means clustering identified three clusters: sporadic users (n=4), frequent transient users (n=38) and abandoning users (n=169) each with distinct usage characteristics. This study highlights the importance of event log data analysis for making improvements to the mental health chatbot.",

keywords = "Conversational user interfaces, event log, eHealth, mental wellbeing, co-design, COVID-19",

author = "Courtney Potts and RR Bond and Maurice Mulvenna and Edel Ennis and Andrea Bickerdike and Edward Coughlan and Thomas Broderick and Con Burns and Michael McTear and Lauri Kuosmanen and Heidi Nieminen and Kyle Boyd and Brian Cahill and Alex Vakaloudis and Indika Dhanapala and Anna-Kaisa Vartiainen and Catrine Kostenius and Martin Malcolm",

note = "Funding Information: The ChatPal consortium acknowledges the support provided by the Interreg VB Northern Periphery & Arctic Programme under the grant for Conversational Interfaces Supporting Mental Health and Wellbeing of People in Sparsely Populated Areas (ChatPal) project number 345. Publisher Copyright: {\textcopyright} 2021 IEEE.; ICTS4eHealth ; Conference date: 05-09-2021 Through 08-09-2021",

year = "2021",

month = dec,

day = "15",

doi = "10.1109/ISCC53001.2021.9631395",

language = "English",

isbn = "978-1-6654-2745-6",

series = "Proceedings - IEEE Symposium on Computers and Communications",

publisher = "IEEE Xplore",

pages = "1--6",

booktitle = "26th IEEE Symposium on Computers and Communications, ISCC 2021",

address = "United States",

url = "https://icts4ehealth.icar.cnr.it/",

}

Potts, C , Bond, RR , Mulvenna, M , Ennis, E, Bickerdike, A, Coughlan, E, Broderick, T, Burns, C, McTear, M, Kuosmanen, L, Nieminen, H, Boyd, K, Cahill, B, Vakaloudis, A, Dhanapala, I, Vartiainen, A-K, Kostenius, C & Malcolm, M 2021, Insights and lessons learned from trialling a mental health chatbot in the wild. in 26th IEEE Symposium on Computers and Communications, ISCC 2021. Proceedings - IEEE Symposium on Computers and Communications, vol. 2021-September, IEEE Xplore, pp. 1-6, ICTS4eHealth, Athens, Greece, 5/09/21. https://doi.org/10.1109/ISCC53001.2021.9631395

Insights and lessons learned from trialling a mental health chatbot in the wild. / Potts, Courtney ; Bond, RR ; Mulvenna, Maurice et al.
26th IEEE Symposium on Computers and Communications, ISCC 2021. IEEE Xplore, 2021. p. 1-6 (Proceedings - IEEE Symposium on Computers and Communications; Vol. 2021-September).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Insights and lessons learned from trialling a mental health chatbot in the wild

AU - Potts, Courtney

AU - Bond, RR

AU - Mulvenna, Maurice

AU - Ennis, Edel

AU - Bickerdike, Andrea

AU - Coughlan, Edward

AU - Broderick, Thomas

AU - Burns, Con

AU - McTear, Michael

AU - Kuosmanen, Lauri

AU - Nieminen, Heidi

AU - Boyd, Kyle

AU - Cahill, Brian

AU - Vakaloudis, Alex

AU - Dhanapala, Indika

AU - Vartiainen, Anna-Kaisa

AU - Kostenius, Catrine

AU - Malcolm, Martin

N1 - Funding Information: The ChatPal consortium acknowledges the support provided by the Interreg VB Northern Periphery & Arctic Programme under the grant for Conversational Interfaces Supporting Mental Health and Wellbeing of People in Sparsely Populated Areas (ChatPal) project number 345. Publisher Copyright: © 2021 IEEE.

PY - 2021/12/15

Y1 - 2021/12/15

N2 - This study reports on the development and 'in the wild' trialling of a chatbot (ChatPal) which promotes good mental wellbeing. A stakeholder-centered approach for design was adopted where end users, mental health professionals and service users were involved in the design which was centered around positive psychology. In the wild usage of the chatbot was investigated from Jul-20-Mar-21. Exploratory analyses of usage metrics were carried out using the event log data. User tenure, unique usage days, total chatbot interactions and average daily interactions were used in K-means clustering to identify user archetypes. The chatbot was used by a variety of age groups (18-65+) and genders, mainly those living in Ireland. K-means clustering identified three clusters: sporadic users (n=4), frequent transient users (n=38) and abandoning users (n=169) each with distinct usage characteristics. This study highlights the importance of event log data analysis for making improvements to the mental health chatbot.

AB - This study reports on the development and 'in the wild' trialling of a chatbot (ChatPal) which promotes good mental wellbeing. A stakeholder-centered approach for design was adopted where end users, mental health professionals and service users were involved in the design which was centered around positive psychology. In the wild usage of the chatbot was investigated from Jul-20-Mar-21. Exploratory analyses of usage metrics were carried out using the event log data. User tenure, unique usage days, total chatbot interactions and average daily interactions were used in K-means clustering to identify user archetypes. The chatbot was used by a variety of age groups (18-65+) and genders, mainly those living in Ireland. K-means clustering identified three clusters: sporadic users (n=4), frequent transient users (n=38) and abandoning users (n=169) each with distinct usage characteristics. This study highlights the importance of event log data analysis for making improvements to the mental health chatbot.

KW - Conversational user interfaces

KW - event log

KW - eHealth

KW - mental wellbeing

KW - co-design

KW - COVID-19

UR - https://icts4ehealth.icar.cnr.it/

UR - http://www.scopus.com/inward/record.url?scp=85123221410&partnerID=8YFLogxK

U2 - 10.1109/ISCC53001.2021.9631395

DO - 10.1109/ISCC53001.2021.9631395

M3 - Conference contribution

SN - 978-1-6654-2745-6

T3 - Proceedings - IEEE Symposium on Computers and Communications

SP - 1

EP - 6

BT - 26th IEEE Symposium on Computers and Communications, ISCC 2021

PB - IEEE Xplore

T2 - ICTS4eHealth

Y2 - 5 September 2021 through 8 September 2021

ER -

Insights and lessons learned from trialling a mental health chatbot in the wild

Abstract

Publication series

Conference

Bibliographical note

Keywords

UN SDGs

Access to Document

Other files and links

Fingerprint

Cite this