Temporal visual attention metrics reveal distinct behaviour within demographic groups and performance levels of healthcare professionals when performing clinical tasks and procedures

  • Jonathan Currie

Student thesis: Doctoral Thesis

Abstract

Introduction: Research has been undertaken to examine the use of eye tracking and other sensors for assessing the performance of healthcare professionals carrying out clinical tasks and procedures. The work is underpinned by the hypothesis that computational eye tracking metrics can be a surrogate to
measuring competency.

Methods: The thesis involves two distinct studies that use eye tracking technology and data science techniques, producing three datasets (two belonging to the first study). The first study investigated how eye tracking, using a Tobii X60 (stationary) eye tracker, could predict the competency of nurses whilst reading vital signs on a bedside monitor. Data science methods were applied to the collected data with first regression modelling and then classification
modelling (e.g., decision trees). The second study used wearable eye tracking glasses by SMI and other wearable computing sensors to measure the performance of interventional cardiologists when carrying out angiographic
procedures in a catheterisation laboratory using a state-of-the-art simulator (Mentice VIST). As this was a smaller sample, the data science approach was less ambitious and was focused on finding significant differences between groups and strong correlations if they existed.

Results: In the first study, a total of 59 participants were recruited and concluded
that eye-tracking could be used to distinguish between subjects, by performance level and characteristics. This was first, with an initial dataset of n=48, linear regression model with 62 eye tracking metrics included as predictors that had a adjusted r-squared of 0.80 and mean square error of 1.2 (on a target 0-10, p value=3.712e-12). Secondly with a larger dataset (n=59), classification showed significant accuracy of 91% (p value=0.0003) in predicting the group (novice vs expert). In the second study a total of 14 participants were recruited, and the study demonstrated that significant differences could be seen in eye tracking
metrics on selected areas of interest between trainee and expert interventionist
cardiologists. Experts had a significantly larger dwell % (11.1 ±4.3 vs 4.7 ±1.6, p = 0.006) and fixation % (8.5 ±3.5 vs 3.5 ±1.4, p = 0.007) on the instruments 14 screen. In addition, experts had a significantly higher totalled dwell % (63 ±10% vs. 42 ±20%, p = 0.03) and fixation % (50.2 ±9.6 vs 33.5 ±17, p = 0.04)

Conclusions: This thesis provides evidence that eye tracking behaviour can measure some of the variation in how competent a user is, whether that involves diagnostic tasks (reading clinical data) or carrying out a clinical intervention
(cardiac treatment). The first study conducts the first ever capturing of visual attention measurements, from a set of nurses (with varying experience and expertise) while reading and interpreting from a simulated vital signs monitor. The data collected, specifically the data analysis of eye tracking metrics, has shown that visual attention and the Performance Level for this specific task (measured by performance score) are not independent of each other. Also, a subset of metrics from the eye tracking data can accurately classify the interpreter as a novice or expert. The second study captured a unique dataset with psychophysiological metrics along with a novel measurement of attentional
capacity recorded during an important highly skilled clinical procedure. Significant differences between groups have been found when using these metrics; most notably the dwell % and fixation % spent on the display screens.
Date of AwardJun 2021
Original languageEnglish
SupervisorRaymond Bond (Supervisor), Paul Mc Cullagh (Supervisor) & Pauline Black (Supervisor)

Keywords

  • Eye-tracking
  • Simulation-based training

Cite this

'