Affective state detection via facial expression analysis within a human-computer interaction context

Research output: Contribution to journalArticle

13 Citations (Scopus)

Abstract

The advancement in technology indicates that there is an opportunity to enhance human–computer interaction by way of affective state recognition. Affective state recognition is typically based on passive stimuli such as watching video clips, which does not reflect genuine interaction. This paper presents a study on affective state recognition using active stimuli, i.e. facial expressions of users when they attempt computerised tasks, particularly across typical usage of computer systems. A data collection experiment is presented for acquiring data from normal users whilst they interact with software, attempting to complete a set of predefined tasks. In addition, a hierarchical machine learning approach is presented for facial expression-based affective state recognition, which employs an Euclidean distance-based feature representation, conjointly with a customised encoding for users’ self-reported affective states. Consequently, the aim is to find the potential relationship between the facial expressions, as de ned by Paul Ekman, and the self-reported emotional states specified by users using Russells Circumplex model, in relation to the actual feelings and affective states. The main findings of this study suggest that facial expressions cannot precisely reveal the actual feelings of users whilst interacting with common computerised tasks. Moreover, during active interaction tasks more variation occurs within the facial expressions of participants than occurs within passive interaction.
Original languageEnglish
Pages (from-to)1-10
JournalJournal Ambient Intelligence andHumanized Computing
Volume8
Early online date4 Dec 2017
DOIs
Publication statusE-pub ahead of print - 4 Dec 2017

Keywords

  • Emotions
  • Human Computer Interaction
  • Facial Expression

Fingerprint Dive into the research topics of 'Affective state detection via facial expression analysis within a human-computer interaction context'. Together they form a unique fingerprint.

Cite this