Computing Eye Gaze Metrics for the Automatic Assessment of Radiographer Performance during X-ray Image Interpretation

Laura McLaughlin, Raymond Bond, Ciara Hughes, Jonathon McConnell, Sonyia McFadden

Research output: Contribution to journalArticle

6 Citations (Scopus)

Abstract

Aim: To investigate image interpretation performance by diagnostic radiography students, diagnostic radiographers and reporting radiographers by computing eye gaze metrics using eye tracking technology. Methods: Three groups of participants were studied during their interpretation of digital 8 radiographic images including the axial and appendicular skeleton, and chest (prevalence of normal images was 12.5%). A total of 464 image interpretations were collected. Participants consisted of 21 radiography students, 19 qualified radiographers and 18 reporting radiographers who were qualified to report on the musculoskeletal (MSK) system.Outcome measures: Eye tracking data was collected using the Tobii X60 eye tracker and subsequently eye gaze metrics were computed. Voice recordings, confidence levels and diagnoses provided a clear demonstration of the image interpretation and the cognitive processes undertaken by each participant. A questionnaire afforded the participants an opportunity to offer information on their experience in image interpretation and their opinion on the eye tracking technology.Results: Reporting radiographers demonstrated a 15% greater accuracy rate (p≤0.001), were more confident (p≤0.001) and took a mean of 2.4s longer to clinically decide on all features compared to students. Reporting radiographers also had a 15% greater accuracy rate (p≤0.001), were more confident (p≤0.001) and took longer to clinically decide on an image diagnosis (p=0.02) than radiographers. Reporting radiographers had a greater mean fixation duration (p=0.01), mean fixation count (p=0.04) and mean visit count (p=0.04) within the areas of pathology compared to students. Eye tracking patterns, presented within heat maps, were a good reflection of group expertise and search strategies. Eye gaze metrics such as time to first fixate, fixation count, fixation duration and visit count within the areas of pathology were indicative of the radiographer’s competency. Conclusion: The accuracy and confidence of each group could be reflected in the variability of their eye tracking heat maps. Participants’ thoughts and decisions were quantified using the eye tracking data. Eye tracking metrics also reflected the different search strategies that each group of participants adopted during their image interpretations. This is the first study to use eye tracking technology to assess image interpretation skills between various groups of different levels of experience in radiography, especially on a combination of the MSK system, chest cavity and a variety of pathologies.
LanguageEnglish
Pages11-21
JournalInternational Journal of Medical Informatics
Volume105
Early online date15 May 2017
DOIs
Publication statusE-pub ahead of print - 15 May 2017

Fingerprint

X-Rays
Radiography
Students
Musculoskeletal System
Pathology
Technology
Thorax
Hot Temperature
Skeleton
Outcome Assessment (Health Care)

Keywords

  • radiography
  • eye tracking
  • interpretation
  • musculoskeletal
  • chest

Cite this

@article{44e2ead798014b7ab3311169902544f4,
title = "Computing Eye Gaze Metrics for the Automatic Assessment of Radiographer Performance during X-ray Image Interpretation",
abstract = "Aim: To investigate image interpretation performance by diagnostic radiography students, diagnostic radiographers and reporting radiographers by computing eye gaze metrics using eye tracking technology. Methods: Three groups of participants were studied during their interpretation of digital 8 radiographic images including the axial and appendicular skeleton, and chest (prevalence of normal images was 12.5{\%}). A total of 464 image interpretations were collected. Participants consisted of 21 radiography students, 19 qualified radiographers and 18 reporting radiographers who were qualified to report on the musculoskeletal (MSK) system.Outcome measures: Eye tracking data was collected using the Tobii X60 eye tracker and subsequently eye gaze metrics were computed. Voice recordings, confidence levels and diagnoses provided a clear demonstration of the image interpretation and the cognitive processes undertaken by each participant. A questionnaire afforded the participants an opportunity to offer information on their experience in image interpretation and their opinion on the eye tracking technology.Results: Reporting radiographers demonstrated a 15{\%} greater accuracy rate (p≤0.001), were more confident (p≤0.001) and took a mean of 2.4s longer to clinically decide on all features compared to students. Reporting radiographers also had a 15{\%} greater accuracy rate (p≤0.001), were more confident (p≤0.001) and took longer to clinically decide on an image diagnosis (p=0.02) than radiographers. Reporting radiographers had a greater mean fixation duration (p=0.01), mean fixation count (p=0.04) and mean visit count (p=0.04) within the areas of pathology compared to students. Eye tracking patterns, presented within heat maps, were a good reflection of group expertise and search strategies. Eye gaze metrics such as time to first fixate, fixation count, fixation duration and visit count within the areas of pathology were indicative of the radiographer’s competency. Conclusion: The accuracy and confidence of each group could be reflected in the variability of their eye tracking heat maps. Participants’ thoughts and decisions were quantified using the eye tracking data. Eye tracking metrics also reflected the different search strategies that each group of participants adopted during their image interpretations. This is the first study to use eye tracking technology to assess image interpretation skills between various groups of different levels of experience in radiography, especially on a combination of the MSK system, chest cavity and a variety of pathologies.",
keywords = "radiography, eye tracking, interpretation, musculoskeletal, chest",
author = "Laura McLaughlin and Raymond Bond and Ciara Hughes and Jonathon McConnell and Sonyia McFadden",
year = "2017",
month = "5",
day = "15",
doi = "10.1016/j.ijmedinf.2017.03.001",
language = "English",
volume = "105",
pages = "11--21",

}

Computing Eye Gaze Metrics for the Automatic Assessment of Radiographer Performance during X-ray Image Interpretation. / McLaughlin, Laura; Bond, Raymond; Hughes, Ciara; McConnell, Jonathon; McFadden, Sonyia.

Vol. 105, 15.05.2017, p. 11-21.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Computing Eye Gaze Metrics for the Automatic Assessment of Radiographer Performance during X-ray Image Interpretation

AU - McLaughlin, Laura

AU - Bond, Raymond

AU - Hughes, Ciara

AU - McConnell, Jonathon

AU - McFadden, Sonyia

PY - 2017/5/15

Y1 - 2017/5/15

N2 - Aim: To investigate image interpretation performance by diagnostic radiography students, diagnostic radiographers and reporting radiographers by computing eye gaze metrics using eye tracking technology. Methods: Three groups of participants were studied during their interpretation of digital 8 radiographic images including the axial and appendicular skeleton, and chest (prevalence of normal images was 12.5%). A total of 464 image interpretations were collected. Participants consisted of 21 radiography students, 19 qualified radiographers and 18 reporting radiographers who were qualified to report on the musculoskeletal (MSK) system.Outcome measures: Eye tracking data was collected using the Tobii X60 eye tracker and subsequently eye gaze metrics were computed. Voice recordings, confidence levels and diagnoses provided a clear demonstration of the image interpretation and the cognitive processes undertaken by each participant. A questionnaire afforded the participants an opportunity to offer information on their experience in image interpretation and their opinion on the eye tracking technology.Results: Reporting radiographers demonstrated a 15% greater accuracy rate (p≤0.001), were more confident (p≤0.001) and took a mean of 2.4s longer to clinically decide on all features compared to students. Reporting radiographers also had a 15% greater accuracy rate (p≤0.001), were more confident (p≤0.001) and took longer to clinically decide on an image diagnosis (p=0.02) than radiographers. Reporting radiographers had a greater mean fixation duration (p=0.01), mean fixation count (p=0.04) and mean visit count (p=0.04) within the areas of pathology compared to students. Eye tracking patterns, presented within heat maps, were a good reflection of group expertise and search strategies. Eye gaze metrics such as time to first fixate, fixation count, fixation duration and visit count within the areas of pathology were indicative of the radiographer’s competency. Conclusion: The accuracy and confidence of each group could be reflected in the variability of their eye tracking heat maps. Participants’ thoughts and decisions were quantified using the eye tracking data. Eye tracking metrics also reflected the different search strategies that each group of participants adopted during their image interpretations. This is the first study to use eye tracking technology to assess image interpretation skills between various groups of different levels of experience in radiography, especially on a combination of the MSK system, chest cavity and a variety of pathologies.

AB - Aim: To investigate image interpretation performance by diagnostic radiography students, diagnostic radiographers and reporting radiographers by computing eye gaze metrics using eye tracking technology. Methods: Three groups of participants were studied during their interpretation of digital 8 radiographic images including the axial and appendicular skeleton, and chest (prevalence of normal images was 12.5%). A total of 464 image interpretations were collected. Participants consisted of 21 radiography students, 19 qualified radiographers and 18 reporting radiographers who were qualified to report on the musculoskeletal (MSK) system.Outcome measures: Eye tracking data was collected using the Tobii X60 eye tracker and subsequently eye gaze metrics were computed. Voice recordings, confidence levels and diagnoses provided a clear demonstration of the image interpretation and the cognitive processes undertaken by each participant. A questionnaire afforded the participants an opportunity to offer information on their experience in image interpretation and their opinion on the eye tracking technology.Results: Reporting radiographers demonstrated a 15% greater accuracy rate (p≤0.001), were more confident (p≤0.001) and took a mean of 2.4s longer to clinically decide on all features compared to students. Reporting radiographers also had a 15% greater accuracy rate (p≤0.001), were more confident (p≤0.001) and took longer to clinically decide on an image diagnosis (p=0.02) than radiographers. Reporting radiographers had a greater mean fixation duration (p=0.01), mean fixation count (p=0.04) and mean visit count (p=0.04) within the areas of pathology compared to students. Eye tracking patterns, presented within heat maps, were a good reflection of group expertise and search strategies. Eye gaze metrics such as time to first fixate, fixation count, fixation duration and visit count within the areas of pathology were indicative of the radiographer’s competency. Conclusion: The accuracy and confidence of each group could be reflected in the variability of their eye tracking heat maps. Participants’ thoughts and decisions were quantified using the eye tracking data. Eye tracking metrics also reflected the different search strategies that each group of participants adopted during their image interpretations. This is the first study to use eye tracking technology to assess image interpretation skills between various groups of different levels of experience in radiography, especially on a combination of the MSK system, chest cavity and a variety of pathologies.

KW - radiography

KW - eye tracking

KW - interpretation

KW - musculoskeletal

KW - chest

U2 - 10.1016/j.ijmedinf.2017.03.001

DO - 10.1016/j.ijmedinf.2017.03.001

M3 - Article

VL - 105

SP - 11

EP - 21

ER -