Affective state detection via facial expression analysis within a human-computer interaction context

Research output: Contribution to journalArticle

5 Citations (Scopus)

Abstract

The advancement in technology indicates that there is an opportunity to enhance human–computer interaction by way of affective state recognition. Affective state recognition is typically based on passive stimuli such as watching video clips, which does not reflect genuine interaction. This paper presents a study on affective state recognition using active stimuli, i.e. facial expressions of users when they attempt computerised tasks, particularly across typical usage of computer systems. A data collection experiment is presented for acquiring data from normal users whilst they interact with software, attempting to complete a set of predefined tasks. In addition, a hierarchical machine learning approach is presented for facial expression-based affective state recognition, which employs an Euclidean distance-based feature representation, conjointly with a customised encoding for users’ self-reported affective states. Consequently, the aim is to find the potential relationship between the facial expressions, as de ned by Paul Ekman, and the self-reported emotional states specified by users using Russells Circumplex model, in relation to the actual feelings and affective states. The main findings of this study suggest that facial expressions cannot precisely reveal the actual feelings of users whilst interacting with common computerised tasks. Moreover, during active interaction tasks more variation occurs within the facial expressions of participants than occurs within passive interaction.
LanguageEnglish
Pages1-10
JournalJournal Ambient Intelligence andHumanized Computing
Volume8
Early online date4 Dec 2017
DOIs
Publication statusE-pub ahead of print - 4 Dec 2017

Fingerprint

Human computer interaction
Learning systems
Computer systems
Experiments

Keywords

  • Emotions
  • Human Computer Interaction
  • Facial Expression

Cite this

@article{c86711390ed74e91ba4a4b402f9f4ea2,
title = "Affective state detection via facial expression analysis within a human-computer interaction context",
abstract = "The advancement in technology indicates that there is an opportunity to enhance human–computer interaction by way of affective state recognition. Affective state recognition is typically based on passive stimuli such as watching video clips, which does not reflect genuine interaction. This paper presents a study on affective state recognition using active stimuli, i.e. facial expressions of users when they attempt computerised tasks, particularly across typical usage of computer systems. A data collection experiment is presented for acquiring data from normal users whilst they interact with software, attempting to complete a set of predefined tasks. In addition, a hierarchical machine learning approach is presented for facial expression-based affective state recognition, which employs an Euclidean distance-based feature representation, conjointly with a customised encoding for users’ self-reported affective states. Consequently, the aim is to find the potential relationship between the facial expressions, as de ned by Paul Ekman, and the self-reported emotional states specified by users using Russells Circumplex model, in relation to the actual feelings and affective states. The main findings of this study suggest that facial expressions cannot precisely reveal the actual feelings of users whilst interacting with common computerised tasks. Moreover, during active interaction tasks more variation occurs within the facial expressions of participants than occurs within passive interaction.",
keywords = "Emotions, Human Computer Interaction, Facial Expression",
author = "Anas Samara and Leo Galway and Raymond Bond and Hui Wang",
note = "This is a post-peer-review, pre-copyedit version of an article published in Journal of Ambient Intelligence and Humanized Computing. The final authenticated version is available online at: http://dx.doi.org/10.1007/s12652-017-0636-8.",
year = "2017",
month = "12",
day = "4",
doi = "10.1007/s12652-017-0636-8",
language = "English",
volume = "8",
pages = "1--10",
journal = "Journal of Ambient Intelligence and Humanized Computing",
issn = "1868-5137",

}

TY - JOUR

T1 - Affective state detection via facial expression analysis within a human-computer interaction context

AU - Samara, Anas

AU - Galway, Leo

AU - Bond, Raymond

AU - Wang, Hui

N1 - This is a post-peer-review, pre-copyedit version of an article published in Journal of Ambient Intelligence and Humanized Computing. The final authenticated version is available online at: http://dx.doi.org/10.1007/s12652-017-0636-8.

PY - 2017/12/4

Y1 - 2017/12/4

N2 - The advancement in technology indicates that there is an opportunity to enhance human–computer interaction by way of affective state recognition. Affective state recognition is typically based on passive stimuli such as watching video clips, which does not reflect genuine interaction. This paper presents a study on affective state recognition using active stimuli, i.e. facial expressions of users when they attempt computerised tasks, particularly across typical usage of computer systems. A data collection experiment is presented for acquiring data from normal users whilst they interact with software, attempting to complete a set of predefined tasks. In addition, a hierarchical machine learning approach is presented for facial expression-based affective state recognition, which employs an Euclidean distance-based feature representation, conjointly with a customised encoding for users’ self-reported affective states. Consequently, the aim is to find the potential relationship between the facial expressions, as de ned by Paul Ekman, and the self-reported emotional states specified by users using Russells Circumplex model, in relation to the actual feelings and affective states. The main findings of this study suggest that facial expressions cannot precisely reveal the actual feelings of users whilst interacting with common computerised tasks. Moreover, during active interaction tasks more variation occurs within the facial expressions of participants than occurs within passive interaction.

AB - The advancement in technology indicates that there is an opportunity to enhance human–computer interaction by way of affective state recognition. Affective state recognition is typically based on passive stimuli such as watching video clips, which does not reflect genuine interaction. This paper presents a study on affective state recognition using active stimuli, i.e. facial expressions of users when they attempt computerised tasks, particularly across typical usage of computer systems. A data collection experiment is presented for acquiring data from normal users whilst they interact with software, attempting to complete a set of predefined tasks. In addition, a hierarchical machine learning approach is presented for facial expression-based affective state recognition, which employs an Euclidean distance-based feature representation, conjointly with a customised encoding for users’ self-reported affective states. Consequently, the aim is to find the potential relationship between the facial expressions, as de ned by Paul Ekman, and the self-reported emotional states specified by users using Russells Circumplex model, in relation to the actual feelings and affective states. The main findings of this study suggest that facial expressions cannot precisely reveal the actual feelings of users whilst interacting with common computerised tasks. Moreover, during active interaction tasks more variation occurs within the facial expressions of participants than occurs within passive interaction.

KW - Emotions

KW - Human Computer Interaction

KW - Facial Expression

U2 - 10.1007/s12652-017-0636-8

DO - 10.1007/s12652-017-0636-8

M3 - Article

VL - 8

SP - 1

EP - 10

JO - Journal of Ambient Intelligence and Humanized Computing

T2 - Journal of Ambient Intelligence and Humanized Computing

JF - Journal of Ambient Intelligence and Humanized Computing

SN - 1868-5137

ER -