Towards mobile cognitive fatigue assessment as indicated by physical, social, environmental and emotional factors

Edward Price, George Moore, Leo Galway, Mark Linden

Research output: Contribution to journalArticlepeer-review

3 Citations (Scopus)
67 Downloads (Pure)

Abstract

This research sought to establish which in-situ measures of cognitive fatigue, physical activity, social interaction, location, emotional state and facial landmarks, made using a smartphone application, could be used to indicate episodes of cognitive fatigue. This assessment was realised using cognitive tests (assessing memory, attention, reaction time, information processing speed and executive function), self-assessment, contextual factors and facial feature analysis. This study also investigated the use of an ensemble algorithm for the classification of cognitive fatigue utilising facial features and a Rotation Forest approach. Self-assessment of cognitive fatigue was shown to directly correlate with reaction time through a Psychomotor Vigilance Task (r=.643, p=.001), and self-reported increases in the level of social activity (r=.377, p=.001). Facial feature analysis revealed dominant emotions of sadness and anger when participants were cognitively fatigued. It also revealed underlying facial cues that indicated higher levels of cognitive fatigue including expressions of negative valence, and Facial Action Coding System units of increased brow furrow, eyelid tightening and lip suck. In addition, a Principle Component Analysis based Rotation Forest ensemble with a ternary output demonstrated a cognitive fatigue classification accuracy of 82.17%. The findings presented indicate that the inclusion of data relating to surrounding cognitive, social, physical and emotional factors can improve the accuracy of mobile in-situ cognitive fatigue assessment using our previously validated smartphone-based cognitive fatigue assessment approach. The findings further suggest gross-level fatigue status may be potentially classified to a reasonable degree of accuracy using facial features, which may give rise to personalised in-situ fatigue detection.
Original languageEnglish
Pages (from-to)116465-116479
Number of pages15
JournalIEEE Access
Volume7
Issue number1
DOIs
Publication statusPublished (in print/issue) - 15 Aug 2019

Keywords

  • Affective Computing
  • Cognitive Fatigue
  • Cognitive Tests
  • Context
  • Facial Analysis
  • Human Computer Interaction
  • Machine Learning
  • Mobile Applications
  • Neuropsychology
  • Smart Healthcare

Fingerprint

Dive into the research topics of 'Towards mobile cognitive fatigue assessment as indicated by physical, social, environmental and emotional factors'. Together they form a unique fingerprint.

Cite this