TEmoX: Classification of Textual Emotion using Ensemble of Transformers

Avishek Das, Mohammed Moshiul Hoque, Omar Sharif, M. Ali Akber Dewan, Nazmul Siddique

Research output: Contribution to journalArticlepeer-review

1 Citation (Scopus)
20 Downloads (Pure)

Abstract

Textual emotion classification (TxtEC) refers to the classification of emotion expressed by individuals in textual form. The widespread use of the Internet and numerous Web 2.0 applications has emerged in an expeditious growth of textual interactions. However, determining emotion from texts is challenging due to their unorganized, unstructured, and disordered forms. While research in textual emotion classification has made considerable breakthroughs for high-resource languages, it is yet challenging for low-resource languages like Bengali. This work presents a transformer-based ensemble approach (called TEmoX) to categorize Bengali textual data into six integral emotions: joy, anger, disgust, fear, sadness, and surprise. This research investigates 38 classifier models developed using four machine learning LR, RF, MNB, SVM, three deep-learning CNN, BiLSTM, CNN+BiLSTM, five transformer-based m-BERT, XLM-R, Bangla-BERT-1, Bangla-BERT-2, and Indic-DistilBERT techniques with two ensemble strategies and three embedding techniques. The developed models are trained, tuned, and tested on the three versions of the Bengali emotion text corpus BEmoC-v1, BEmoC-v2, BEmoC-v3. The experimental outcomes reveal that the weighted ensemble of four transformer models En-22: Bangla-BERT-2, XLM-R, Indic-DistilBERT, Bangla-BERT-1 outperforms the baseline models and existing methods by providing the maximum weighted F1 -score (80.24%) on BEmoC-v3. The dataset, models, and fractions of codes are available at https://github.com/avishek-018/TEmoX .
Original languageEnglish
Pages (from-to)109803 - 109818
Number of pages16
JournalIEEE Access
Volume11
Early online date26 Sept 2023
DOIs
Publication statusPublished (in print/issue) - 11 Oct 2023

Bibliographical note

Funding Information:
This work was supported in part by the Natural Sciences and Engineering Research Council (NSERC), Canada; and in part by the Directorate of Research and Extension, Chittagong University of Engineering and Technology (CUET).

Publisher Copyright:
© 2013 IEEE.

Keywords

  • General Engineering
  • General Materials Science
  • General Computer Science
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'TEmoX: Classification of Textual Emotion using Ensemble of Transformers'. Together they form a unique fingerprint.

Cite this