The impact of AI feedback on the accuracy of diagnosis, decision switching and trust in Radiography.

Clare Rainey; Raymond Bond; Jonathan McConnell; Avneet Gill; Ciara Hughes; Deviner Kumar; Sonyia McFadden

doi:10.1371/journal.pone.0322051

The impact of AI feedback on the accuracy of diagnosis, decision switching and trust in Radiography.

Clare Rainey, Raymond Bond, Jonathan McConnell, Avneet Gill, Ciara Hughes, Deviner Kumar, Sonyia McFadden

Research output: Contribution to journal › Article › peer-review

2 Downloads (Pure)

Abstract

Artificial intelligence decision support systems have been proposed to assist a struggling National Health Service (NHS) workforce in the United Kingdom. Its implementation in UK healthcare systems has been identified as a priority for deployment. Few studies have investigated the impact of the feedback from such systems on the end user. This study investigated the impact of two forms of AI feedback (saliency/heatmaps and AI diagnosis with percentage confidence) on student and qualified diagnostic radiographers’ accuracy when determining binary diagnosis on skeletal radiographs. The AI feedback proved beneficial to accuracy in all cases except when the AI was incorrect and for pathological cases in the student group. The self-reported trust of all participants decreased from the beginning to the end of the study. The findings of this study should guide developers in the provision of the most advantageous forms of AI feedback and direct educators in tailoring education to highlight weaknesses in human interaction with AI-based clinical decision support systems.

Original language	English
Article number	e0322051
Pages (from-to)	1-27
Number of pages	27
Journal	PLoS One
Volume	20
Issue number	5
Early online date	9 May 2025
DOIs	https://doi.org/10.1371/journal.pone.0322051
Publication status	Published online - 9 May 2025

Bibliographical note

Publisher Copyright:
© 2025 Rainey et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Access Statement

Data cannot be shared publicly because of ethical restrictions on sharing sensitive data. Access to the data can be provided following a successful application to Ulster University’s Nursing and Health Research Ethics Filter Committee. Ulster University’s Research Portal contains metadata on the dataset and instructions on how to request access to this dataset. This information can be accessed at https://doi.org/10.21251/50890091-4b54-4644-b980-ea9da646aa0e.

Keywords

AI
trust
decision switching
Humans
Artificial Intelligence
United Kingdom
Decision Support Systems, Clinical
Male
Trust
Radiography
Feedback
Female
Adult

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1371/journal.pone.0322051Licence: CC BY

journal.pone.0322051Final published version, 1.92 MBLicence: CC BY

Cite this

@article{53eadd1aac1647a2a62a8fe842433287,

title = "The impact of AI feedback on the accuracy of diagnosis, decision switching and trust in Radiography.",

abstract = "Artificial intelligence decision support systems have been proposed to assist a struggling National Health Service (NHS) workforce in the United Kingdom. Its implementation in UK healthcare systems has been identified as a priority for deployment. Few studies have investigated the impact of the feedback from such systems on the end user. This study investigated the impact of two forms of AI feedback (saliency/heatmaps and AI diagnosis with percentage confidence) on student and qualified diagnostic radiographers{\textquoteright} accuracy when determining binary diagnosis on skeletal radiographs. The AI feedback proved beneficial to accuracy in all cases except when the AI was incorrect and for pathological cases in the student group. The self-reported trust of all participants decreased from the beginning to the end of the study. The findings of this study should guide developers in the provision of the most advantageous forms of AI feedback and direct educators in tailoring education to highlight weaknesses in human interaction with AI-based clinical decision support systems.",

keywords = "AI, trust, decision switching, Humans, Artificial Intelligence, United Kingdom, Decision Support Systems, Clinical, Male, Trust, Radiography, Feedback, Female, Adult",

author = "Clare Rainey and Raymond Bond and Jonathan McConnell and Avneet Gill and Ciara Hughes and Deviner Kumar and Sonyia McFadden",

note = "Publisher Copyright: {\textcopyright} 2025 Rainey et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.",

year = "2025",

month = may,

day = "9",

doi = "10.1371/journal.pone.0322051",

language = "English",

volume = "20",

pages = "1--27",

journal = "PLoS One ",

issn = "1932-6203",

publisher = "PLoS",

number = "5",

}

TY - JOUR

T1 - The impact of AI feedback on the accuracy of diagnosis, decision switching and trust in Radiography.

AU - Rainey, Clare

AU - Bond, Raymond

AU - McConnell, Jonathan

AU - Gill, Avneet

AU - Hughes, Ciara

AU - Kumar, Deviner

AU - McFadden, Sonyia

N1 - Publisher Copyright: © 2025 Rainey et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

PY - 2025/5/9

Y1 - 2025/5/9

N2 - Artificial intelligence decision support systems have been proposed to assist a struggling National Health Service (NHS) workforce in the United Kingdom. Its implementation in UK healthcare systems has been identified as a priority for deployment. Few studies have investigated the impact of the feedback from such systems on the end user. This study investigated the impact of two forms of AI feedback (saliency/heatmaps and AI diagnosis with percentage confidence) on student and qualified diagnostic radiographers’ accuracy when determining binary diagnosis on skeletal radiographs. The AI feedback proved beneficial to accuracy in all cases except when the AI was incorrect and for pathological cases in the student group. The self-reported trust of all participants decreased from the beginning to the end of the study. The findings of this study should guide developers in the provision of the most advantageous forms of AI feedback and direct educators in tailoring education to highlight weaknesses in human interaction with AI-based clinical decision support systems.

AB - Artificial intelligence decision support systems have been proposed to assist a struggling National Health Service (NHS) workforce in the United Kingdom. Its implementation in UK healthcare systems has been identified as a priority for deployment. Few studies have investigated the impact of the feedback from such systems on the end user. This study investigated the impact of two forms of AI feedback (saliency/heatmaps and AI diagnosis with percentage confidence) on student and qualified diagnostic radiographers’ accuracy when determining binary diagnosis on skeletal radiographs. The AI feedback proved beneficial to accuracy in all cases except when the AI was incorrect and for pathological cases in the student group. The self-reported trust of all participants decreased from the beginning to the end of the study. The findings of this study should guide developers in the provision of the most advantageous forms of AI feedback and direct educators in tailoring education to highlight weaknesses in human interaction with AI-based clinical decision support systems.

KW - AI

KW - trust

KW - decision switching

KW - Humans

KW - Artificial Intelligence

KW - United Kingdom

KW - Decision Support Systems, Clinical

KW - Male

KW - Trust

KW - Radiography

KW - Feedback

KW - Female

KW - Adult

UR - https://pure.ulster.ac.uk/en/publications/53eadd1a-ac16-47a2-a62a-8fe842433287

UR - http://www.scopus.com/inward/record.url?scp=105004933122&partnerID=8YFLogxK

U2 - 10.1371/journal.pone.0322051

DO - 10.1371/journal.pone.0322051

M3 - Article

C2 - 40344152

SN - 1932-6203

VL - 20

SP - 1

EP - 27

JO - PLoS One

JF - PLoS One

IS - 5

M1 - e0322051

ER -

The impact of AI feedback on the accuracy of diagnosis, decision switching and trust in Radiography.

Abstract

Bibliographical note

Data Access Statement

Keywords

UN SDGs

Access to Document

Other files and links

Fingerprint

Cite this