A comparative study of chatbot response generation: traditional approaches versus large language models

Michael McTear; Sheen Varghese Marokkie; Y Bi

doi:10.1007/978-3-031-40286-9_7

A comparative study of chatbot response generation: traditional approaches versus large language models

Michael McTear, Sheen Varghese Marokkie, Y Bi

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

1 Citation (Scopus)

Abstract

Chatbot responses can be generated using traditional rule-based conversation design or through the use of large language models (LLMs). In this paper we compare the quality of responses provided by LLM-based chatbots with those provided by traditional conversation design. The results suggest that in some cases the use of LLMs could improve the quality of chatbot responses. The paper concludes by suggesting that a combination of approaches is the best way forward and suggests some directions for future work.

Original language	English
Title of host publication	Knowledge Science, Engineering and Management - 16th International Conference, KSEM 2023
Editors	Zhi Jin, Yuncheng Jiang, Wenjun Ma, Robert Andrei Buchmann, Ana-Maria Ghiran, Yaxin Bi
Place of Publication	Germany
Pages	70-79
Number of pages	10
Volume	LNCS, volume 14118
Edition	1
ISBN (Electronic)	978-3-031-40285-2
DOIs	https://doi.org/10.1007/978-3-031-40286-9_7
Publication status	Published online - 9 Aug 2023
Event	16th International Conference, KSEM 2023, - China, Guangzhou Duration: 16 Aug 2023 → 19 Oct 2023 https://www.ksem2023.conferences.academy

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	14118 LNAI
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	16th International Conference, KSEM 2023,
Abbreviated title	KSEM 2023
City	Guangzhou
Period	16/08/23 → 19/10/23
Internet address	https://www.ksem2023.conferences.academy

Bibliographical note

Funding Information:
Michael McTear received support from the e-VITA project (https://www.e-vita.coach/) (accessed on 23 April 2023). Sheen Varghese Marokkie and Yaxin Bi received support from the School of Computing, Ulster University (https://www.ulster.ac.uk/faculties/computing-engineering-and-the-built-environme nt/computing).

Funding Information:
1The e-VITA project has received funding from the European Union H2020 Pro-gramme under grant agreement no. 101016453. The Japanese consortium received funding from the Japanese Ministry of Internal Affairs and Communication (MIC), Grant no. JPJ000595.

Publisher Copyright:
© 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.

Keywords

ChatGTP
NLP
ChatGPT
Conversational AI
Conversation design
Rasa
Large Language Models
Bard

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1007/978-3-031-40286-9_7

https://link.springer.com/chapter/10.1007/978-3-031-40286-9_7Licence: Unspecified

Cite this

McTear, M., Varghese Marokkie, S., & Bi, Y. (2023). A comparative study of chatbot response generation: traditional approaches versus large language models. In Z. Jin, Y. Jiang, W. Ma, R. A. Buchmann, A.-M. Ghiran, & Y. Bi (Eds.), Knowledge Science, Engineering and Management - 16th International Conference, KSEM 2023 (1 ed., Vol. LNCS, volume 14118, pp. 70-79). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 14118 LNAI).. Advance online publication. https://doi.org/10.1007/978-3-031-40286-9_7

McTear, Michael ; Varghese Marokkie, Sheen ; Bi, Y. / A comparative study of chatbot response generation: traditional approaches versus large language models. Knowledge Science, Engineering and Management - 16th International Conference, KSEM 2023. editor / Zhi Jin ; Yuncheng Jiang ; Wenjun Ma ; Robert Andrei Buchmann ; Ana-Maria Ghiran ; Yaxin Bi. Vol. LNCS, volume 14118 1. ed. Germany, 2023. pp. 70-79 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{4349a10819224b5dabee85510a8b66b2,

title = "A comparative study of chatbot response generation: traditional approaches versus large language models",

abstract = "Chatbot responses can be generated using traditional rule-based conversation design or through the use of large language models (LLMs). In this paper we compare the quality of responses provided by LLM-based chatbots with those provided by traditional conversation design. The results suggest that in some cases the use of LLMs could improve the quality of chatbot responses. The paper concludes by suggesting that a combination of approaches is the best way forward and suggests some directions for future work.",

keywords = "ChatGTP, NLP, ChatGPT, Conversational AI, Conversation design, Rasa, Large Language Models, Bard",

author = "Michael McTear and {Varghese Marokkie}, Sheen and Y Bi",

note = "Funding Information: Michael McTear received support from the e-VITA project (https://www.e-vita.coach/) (accessed on 23 April 2023). Sheen Varghese Marokkie and Yaxin Bi received support from the School of Computing, Ulster University (https://www.ulster.ac.uk/faculties/computing-engineering-and-the-built-environme nt/computing). Funding Information: 1The e-VITA project has received funding from the European Union H2020 Pro-gramme under grant agreement no. 101016453. The Japanese consortium received funding from the Japanese Ministry of Internal Affairs and Communication (MIC), Grant no. JPJ000595. Publisher Copyright: {\textcopyright} 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.; 16th International Conference, KSEM 2023, , KSEM 2023 ; Conference date: 16-08-2023 Through 19-10-2023",

year = "2023",

month = aug,

day = "9",

doi = "10.1007/978-3-031-40286-9_7",

language = "English",

isbn = "9783031402852",

volume = "LNCS, volume 14118",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

pages = "70--79",

editor = "Zhi Jin and Yuncheng Jiang and Wenjun Ma and Buchmann, {Robert Andrei} and Ana-Maria Ghiran and Yaxin Bi",

booktitle = "Knowledge Science, Engineering and Management - 16th International Conference, KSEM 2023",

edition = "1",

url = "https://www.ksem2023.conferences.academy",

}

McTear, M, Varghese Marokkie, S & Bi, Y 2023, A comparative study of chatbot response generation: traditional approaches versus large language models. in Z Jin, Y Jiang, W Ma, RA Buchmann, A-M Ghiran & Y Bi (eds), Knowledge Science, Engineering and Management - 16th International Conference, KSEM 2023. 1 edn, vol. LNCS, volume 14118, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 14118 LNAI, Germany, pp. 70-79, 16th International Conference, KSEM 2023, , Guangzhou, 16/08/23. https://doi.org/10.1007/978-3-031-40286-9_7

A comparative study of chatbot response generation: traditional approaches versus large language models. / McTear, Michael; Varghese Marokkie, Sheen; Bi, Y.
Knowledge Science, Engineering and Management - 16th International Conference, KSEM 2023. ed. / Zhi Jin; Yuncheng Jiang; Wenjun Ma; Robert Andrei Buchmann; Ana-Maria Ghiran; Yaxin Bi. Vol. LNCS, volume 14118 1. ed. Germany, 2023. p. 70-79 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 14118 LNAI).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - A comparative study of chatbot response generation: traditional approaches versus large language models

AU - McTear, Michael

AU - Varghese Marokkie, Sheen

AU - Bi, Y

N1 - Funding Information: Michael McTear received support from the e-VITA project (https://www.e-vita.coach/) (accessed on 23 April 2023). Sheen Varghese Marokkie and Yaxin Bi received support from the School of Computing, Ulster University (https://www.ulster.ac.uk/faculties/computing-engineering-and-the-built-environme nt/computing). Funding Information: 1The e-VITA project has received funding from the European Union H2020 Pro-gramme under grant agreement no. 101016453. The Japanese consortium received funding from the Japanese Ministry of Internal Affairs and Communication (MIC), Grant no. JPJ000595. Publisher Copyright: © 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.

PY - 2023/8/9

Y1 - 2023/8/9

N2 - Chatbot responses can be generated using traditional rule-based conversation design or through the use of large language models (LLMs). In this paper we compare the quality of responses provided by LLM-based chatbots with those provided by traditional conversation design. The results suggest that in some cases the use of LLMs could improve the quality of chatbot responses. The paper concludes by suggesting that a combination of approaches is the best way forward and suggests some directions for future work.

AB - Chatbot responses can be generated using traditional rule-based conversation design or through the use of large language models (LLMs). In this paper we compare the quality of responses provided by LLM-based chatbots with those provided by traditional conversation design. The results suggest that in some cases the use of LLMs could improve the quality of chatbot responses. The paper concludes by suggesting that a combination of approaches is the best way forward and suggests some directions for future work.

KW - ChatGTP

KW - NLP

KW - ChatGPT

KW - Conversational AI

KW - Conversation design

KW - Rasa

KW - Large Language Models

KW - Bard

UR - http://www.scopus.com/inward/record.url?scp=85172998868&partnerID=8YFLogxK

U2 - 10.1007/978-3-031-40286-9_7

DO - 10.1007/978-3-031-40286-9_7

M3 - Conference contribution

SN - 9783031402852

VL - LNCS, volume 14118

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 70

EP - 79

BT - Knowledge Science, Engineering and Management - 16th International Conference, KSEM 2023

A2 - Jin, Zhi

A2 - Jiang, Yuncheng

A2 - Ma, Wenjun

A2 - Buchmann, Robert Andrei

A2 - Ghiran, Ana-Maria

A2 - Bi, Yaxin

CY - Germany

T2 - 16th International Conference, KSEM 2023,

Y2 - 16 August 2023 through 19 October 2023

ER -

McTear M, Varghese Marokkie S, Bi Y. A comparative study of chatbot response generation: traditional approaches versus large language models. In Jin Z, Jiang Y, Ma W, Buchmann RA, Ghiran AM, Bi Y, editors, Knowledge Science, Engineering and Management - 16th International Conference, KSEM 2023. 1 ed. Vol. LNCS, volume 14118. Germany. 2023. p. 70-79. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). Epub 2023 Aug 9. doi: 10.1007/978-3-031-40286-9_7

A comparative study of chatbot response generation: traditional approaches versus large language models

Abstract

Publication series

Conference

Bibliographical note

Keywords

UN SDGs

Access to Document

Other files and links

Fingerprint

Cite this