Towards mobile cognitive fatigue assessment as indicated by physical, social, environmental and emotional factors

Edward Price, George Moore, Leo Galway, Mark Linden

Research output: Contribution to journalArticle

12 Downloads (Pure)

Abstract

This research sought to establish which in-situ measures of cognitive fatigue, physical activity, social interaction, location, emotional state and facial landmarks, made using a smartphone application, could be used to indicate episodes of cognitive fatigue. This assessment was realised using cognitive tests (assessing memory, attention, reaction time, information processing speed and executive function), self-assessment, contextual factors and facial feature analysis. This study also investigated the use of an ensemble algorithm for the classification of cognitive fatigue utilising facial features and a Rotation Forest approach. Self-assessment of cognitive fatigue was shown to directly correlate with reaction time through a Psychomotor Vigilance Task (r=.643, p=.001), and self-reported increases in the level of social activity (r=.377, p=.001). Facial feature analysis revealed dominant emotions of sadness and anger when participants were cognitively fatigued. It also revealed underlying facial cues that indicated higher levels of cognitive fatigue including expressions of negative valence, and Facial Action Coding System units of increased brow furrow, eyelid tightening and lip suck. In addition, a Principle Component Analysis based Rotation Forest ensemble with a ternary output demonstrated a cognitive fatigue classification accuracy of 82.17%. The findings presented indicate that the inclusion of data relating to surrounding cognitive, social, physical and emotional factors can improve the accuracy of mobile in-situ cognitive fatigue assessment using our previously validated smartphone-based cognitive fatigue assessment approach. The findings further suggest gross-level fatigue status may be potentially classified to a reasonable degree of accuracy using facial features, which may give rise to personalised in-situ fatigue detection.
Original languageEnglish
Pages (from-to)116465-116479
Number of pages15
JournalIEEE Access
Volume7
Issue number1
DOIs
Publication statusPublished - 15 Aug 2019

Keywords

  • Affective Computing
  • Cognitive Fatigue
  • Cognitive Tests
  • Context
  • Facial Analysis
  • Human Computer Interaction
  • Machine Learning
  • Mobile Applications
  • Neuropsychology
  • Smart Healthcare

Fingerprint Dive into the research topics of 'Towards mobile cognitive fatigue assessment as indicated by physical, social, environmental and emotional factors'. Together they form a unique fingerprint.

  • Profiles

    Research Output

    • 3 Conference contribution
    • 1 Article

    Validation of a smartphone based approach to in-situ cognitive fatigue assessment

    Price, E., Moore, G., Galway, L. & Linden, M., 17 Aug 2017, In : JMIR mHealth and uHealth. 5, 8, p. 1-13

    Research output: Contribution to journalArticle

    Open Access
    File
  • 1 Downloads (Pure)

    From Paper to Play - Design and Validation of a Smartphone Based Cognitive Fatigue Assessment Application

    Price, E., Moore, G., Galway, L. & Linden, M., 2 Nov 2016, Lecture Notes in Computer Science. Springer, Vol. 10069. p. 321-333 12 p.

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Open Access
    File
  • 43 Downloads (Pure)

    User Centred Design of a Smartphone-based Cognitive Fatigue Assessment Application

    Price, E., Moore, G., Galway, L. & Linden, M., 29 Nov 2016, Unknown Host Publication. Association for Computing Machinery, 8 p.

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Open Access
  • 3 Citations (Scopus)

    Cite this