Wearable technology-based metrics for predicting operator performance during cardiac catheterisation

Jonathan Currie, RR Bond, P McCullagh, Pauline Black, D Finlay, Stephen Gallagher, Peter Kearney, Aaron Peace, Danail Stoyanov, Colin D. Bicknell, Stephen Leslie, Anthony G. Gallagher

Research output: Contribution to journalArticle

Abstract

Introduction: Unobtrusive metrics that can auto-assess performance during clinical procedures are of value. Three approaches to deriving wearable technology-based metrics are explored: (1) eye tracking, (2) psychophysiological measurements [e.g. electrodermal activity (EDA)] and (3) arm and hand movement via accelerometry. We also measure attentional capacity by tasking the operator with an additional task to track an unrelated object during the procedure. Methods: Two aspects of performance are measured: (1) using eye gaze and psychophysiology metrics and (2) measuring attentional capacity via an additional unrelated task (to monitor a visual stimulus/playing cards). The aim was to identify metrics that can be used to automatically discriminate between levels of performance or at least between novices and experts. The study was conducted using two groups: (1) novice operators and (2) expert operators. Both groups made two attempts at a coronary angiography procedure using a full-physics virtual reality simulator. Participants wore eye tracking glasses and an E4 wearable wristband. Areas of interest were defined to track visual attention on display screens, including: (1) X-ray, (2) vital signs, (3) instruments and (4) the stimulus screen (for measuring attentional capacity). Results: Experts provided greater dwell time (63% vs 42%, p = 0.03) and fixations (50% vs 34%, p = 0.04) on display screens. They also provided greater dwell time (11% vs 5%, p = 0.006) and fixations (9% vs 4%, p = 0.007) when selecting instruments. The experts’ performance for tracking the unrelated object during the visual stimulus task negatively correlated with total errors (r = − 0.95, p = 0.0009). Experts also had a higher standard deviation of EDA (2.52 µS vs 0.89 µS, p = 0.04). Conclusions: Eye tracking metrics may help discriminate between a novice and expert operator, by showing that experts maintain greater visual attention on the display screens. In addition, the visual stimulus study shows that an unrelated task can measure attentional capacity. Trial registration This work is registered through clinicaltrials.gov, a service of the U.S. National Health Institute, and is identified by the trial reference: NCT02928796.

LanguageEnglish
Pages 645–657
Number of pages13
JournalInternational Journal of Computer Assisted Radiology and Surgery
Volume14
Issue number4
Early online date7 Feb 2019
DOIs
Publication statusPublished - 1 Apr 2019

Fingerprint

Cardiac Catheterization
Display devices
Technology
Psychophysiology
Angiography
Accelerometry
Virtual reality
Vital Signs
Physics
National Institutes of Health (U.S.)
National Health Programs
Simulators
Coronary Angiography
Wear of materials
Health
Glass
X rays
Arm
Hand
X-Rays

Keywords

  • Attentional capacity
  • Eye tracking
  • Simulation-based training
  • Surgical simulation
  • Wearable technology

Cite this

Currie, Jonathan ; Bond, RR ; McCullagh, P ; Black, Pauline ; Finlay, D ; Gallagher, Stephen ; Kearney, Peter ; Peace, Aaron ; Stoyanov, Danail ; Bicknell, Colin D. ; Leslie, Stephen ; Gallagher, Anthony G. / Wearable technology-based metrics for predicting operator performance during cardiac catheterisation. 2019 ; Vol. 14, No. 4. pp. 645–657.
@article{091ef0e22938414cb7127bf0eb05b1ff,
title = "Wearable technology-based metrics for predicting operator performance during cardiac catheterisation",
abstract = "Introduction: Unobtrusive metrics that can auto-assess performance during clinical procedures are of value. Three approaches to deriving wearable technology-based metrics are explored: (1) eye tracking, (2) psychophysiological measurements [e.g. electrodermal activity (EDA)] and (3) arm and hand movement via accelerometry. We also measure attentional capacity by tasking the operator with an additional task to track an unrelated object during the procedure. Methods: Two aspects of performance are measured: (1) using eye gaze and psychophysiology metrics and (2) measuring attentional capacity via an additional unrelated task (to monitor a visual stimulus/playing cards). The aim was to identify metrics that can be used to automatically discriminate between levels of performance or at least between novices and experts. The study was conducted using two groups: (1) novice operators and (2) expert operators. Both groups made two attempts at a coronary angiography procedure using a full-physics virtual reality simulator. Participants wore eye tracking glasses and an E4 wearable wristband. Areas of interest were defined to track visual attention on display screens, including: (1) X-ray, (2) vital signs, (3) instruments and (4) the stimulus screen (for measuring attentional capacity). Results: Experts provided greater dwell time (63{\%} vs 42{\%}, p = 0.03) and fixations (50{\%} vs 34{\%}, p = 0.04) on display screens. They also provided greater dwell time (11{\%} vs 5{\%}, p = 0.006) and fixations (9{\%} vs 4{\%}, p = 0.007) when selecting instruments. The experts’ performance for tracking the unrelated object during the visual stimulus task negatively correlated with total errors (r = − 0.95, p = 0.0009). Experts also had a higher standard deviation of EDA (2.52 µS vs 0.89 µS, p = 0.04). Conclusions: Eye tracking metrics may help discriminate between a novice and expert operator, by showing that experts maintain greater visual attention on the display screens. In addition, the visual stimulus study shows that an unrelated task can measure attentional capacity. Trial registration This work is registered through clinicaltrials.gov, a service of the U.S. National Health Institute, and is identified by the trial reference: NCT02928796.",
keywords = "Attentional capacity, Eye tracking, Simulation-based training, Surgical simulation, Wearable technology",
author = "Jonathan Currie and RR Bond and P McCullagh and Pauline Black and D Finlay and Stephen Gallagher and Peter Kearney and Aaron Peace and Danail Stoyanov and Bicknell, {Colin D.} and Stephen Leslie and Gallagher, {Anthony G.}",
year = "2019",
month = "4",
day = "1",
doi = "10.1007/s11548-019-01918-0",
language = "English",
volume = "14",
pages = "645–657",
number = "4",

}

Wearable technology-based metrics for predicting operator performance during cardiac catheterisation. / Currie, Jonathan; Bond, RR; McCullagh, P; Black, Pauline; Finlay, D; Gallagher, Stephen; Kearney, Peter; Peace, Aaron; Stoyanov, Danail; Bicknell, Colin D.; Leslie, Stephen; Gallagher, Anthony G.

Vol. 14, No. 4, 01.04.2019, p. 645–657.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Wearable technology-based metrics for predicting operator performance during cardiac catheterisation

AU - Currie, Jonathan

AU - Bond, RR

AU - McCullagh, P

AU - Black, Pauline

AU - Finlay, D

AU - Gallagher, Stephen

AU - Kearney, Peter

AU - Peace, Aaron

AU - Stoyanov, Danail

AU - Bicknell, Colin D.

AU - Leslie, Stephen

AU - Gallagher, Anthony G.

PY - 2019/4/1

Y1 - 2019/4/1

N2 - Introduction: Unobtrusive metrics that can auto-assess performance during clinical procedures are of value. Three approaches to deriving wearable technology-based metrics are explored: (1) eye tracking, (2) psychophysiological measurements [e.g. electrodermal activity (EDA)] and (3) arm and hand movement via accelerometry. We also measure attentional capacity by tasking the operator with an additional task to track an unrelated object during the procedure. Methods: Two aspects of performance are measured: (1) using eye gaze and psychophysiology metrics and (2) measuring attentional capacity via an additional unrelated task (to monitor a visual stimulus/playing cards). The aim was to identify metrics that can be used to automatically discriminate between levels of performance or at least between novices and experts. The study was conducted using two groups: (1) novice operators and (2) expert operators. Both groups made two attempts at a coronary angiography procedure using a full-physics virtual reality simulator. Participants wore eye tracking glasses and an E4 wearable wristband. Areas of interest were defined to track visual attention on display screens, including: (1) X-ray, (2) vital signs, (3) instruments and (4) the stimulus screen (for measuring attentional capacity). Results: Experts provided greater dwell time (63% vs 42%, p = 0.03) and fixations (50% vs 34%, p = 0.04) on display screens. They also provided greater dwell time (11% vs 5%, p = 0.006) and fixations (9% vs 4%, p = 0.007) when selecting instruments. The experts’ performance for tracking the unrelated object during the visual stimulus task negatively correlated with total errors (r = − 0.95, p = 0.0009). Experts also had a higher standard deviation of EDA (2.52 µS vs 0.89 µS, p = 0.04). Conclusions: Eye tracking metrics may help discriminate between a novice and expert operator, by showing that experts maintain greater visual attention on the display screens. In addition, the visual stimulus study shows that an unrelated task can measure attentional capacity. Trial registration This work is registered through clinicaltrials.gov, a service of the U.S. National Health Institute, and is identified by the trial reference: NCT02928796.

AB - Introduction: Unobtrusive metrics that can auto-assess performance during clinical procedures are of value. Three approaches to deriving wearable technology-based metrics are explored: (1) eye tracking, (2) psychophysiological measurements [e.g. electrodermal activity (EDA)] and (3) arm and hand movement via accelerometry. We also measure attentional capacity by tasking the operator with an additional task to track an unrelated object during the procedure. Methods: Two aspects of performance are measured: (1) using eye gaze and psychophysiology metrics and (2) measuring attentional capacity via an additional unrelated task (to monitor a visual stimulus/playing cards). The aim was to identify metrics that can be used to automatically discriminate between levels of performance or at least between novices and experts. The study was conducted using two groups: (1) novice operators and (2) expert operators. Both groups made two attempts at a coronary angiography procedure using a full-physics virtual reality simulator. Participants wore eye tracking glasses and an E4 wearable wristband. Areas of interest were defined to track visual attention on display screens, including: (1) X-ray, (2) vital signs, (3) instruments and (4) the stimulus screen (for measuring attentional capacity). Results: Experts provided greater dwell time (63% vs 42%, p = 0.03) and fixations (50% vs 34%, p = 0.04) on display screens. They also provided greater dwell time (11% vs 5%, p = 0.006) and fixations (9% vs 4%, p = 0.007) when selecting instruments. The experts’ performance for tracking the unrelated object during the visual stimulus task negatively correlated with total errors (r = − 0.95, p = 0.0009). Experts also had a higher standard deviation of EDA (2.52 µS vs 0.89 µS, p = 0.04). Conclusions: Eye tracking metrics may help discriminate between a novice and expert operator, by showing that experts maintain greater visual attention on the display screens. In addition, the visual stimulus study shows that an unrelated task can measure attentional capacity. Trial registration This work is registered through clinicaltrials.gov, a service of the U.S. National Health Institute, and is identified by the trial reference: NCT02928796.

KW - Attentional capacity

KW - Eye tracking

KW - Simulation-based training

KW - Surgical simulation

KW - Wearable technology

UR - http://www.scopus.com/inward/record.url?scp=85061330038&partnerID=8YFLogxK

U2 - 10.1007/s11548-019-01918-0

DO - 10.1007/s11548-019-01918-0

M3 - Article

VL - 14

SP - 645

EP - 657

IS - 4

ER -