Abstract
Purpose: As artificial intelligence (AI) becomes more integrated into healthcare
it is imperative to deploy this technology responsibly. This research attempts to
add insight into the effect of forms of AI feedback on the interpretation
accuracy of student and qualified radiographers, highlighting optimal aspects of
AI feedback.
Methods or Backround: A study was designed in Qualtrics®, deployed at
ECR 2021 and promoted via social media (Twitter®/LinkedIn®), resulting in 94
respondents. Each participant was presented with a random selection of three
musculoskeletal examinations from a dataset of 21. Diagnostic accuracy under
four conditions (AI correct, AI incorrect, pathological and non-pathological
examinations) was determined by comparison with a known ground truth
diagnosis from three to five reporting radiographers and/or radiologists.
ANOVA, t-tests and descriptive statistics were used to describe the data.
Results or Findings: There is a statistically significant difference in diagnostic
accuracy of all participants before any AI feedback and after provision of binary
diagnosis, when the AI is correct (p=0.007). In pathological cases there was a
statistically significant decrease in accuracy following presentation of the
heatmap (p=0.015) and a subsequent increase in accuracy when the binary
diagnosis was provided (p=0.013).
Conclusion: Heatmap feedback was detrimental to participant accuracy in
pathological cases and did not have a positive impact on accuracy in any
condition. Binary diagnosis was beneficial to both groups across all of the four
conditions with a significant impact on accuracy when the AI was correct and in
pathological cases. Methods of explainability should be developed with
cognisance of the impact on the end-user.
Limitations: Participants were analysed as two broad groups. Impact of years
of professional experience on interaction with the AI will be investigated.
Ethics committee approval: This study was approved by the UU Nursing and
Health research filter committee, approval number FCNUR-20-035.
Funding for this study: Funding was received from the College of
Radiographers Industry Partnership award Scheme (CoRIPS) and Ulster
University
it is imperative to deploy this technology responsibly. This research attempts to
add insight into the effect of forms of AI feedback on the interpretation
accuracy of student and qualified radiographers, highlighting optimal aspects of
AI feedback.
Methods or Backround: A study was designed in Qualtrics®, deployed at
ECR 2021 and promoted via social media (Twitter®/LinkedIn®), resulting in 94
respondents. Each participant was presented with a random selection of three
musculoskeletal examinations from a dataset of 21. Diagnostic accuracy under
four conditions (AI correct, AI incorrect, pathological and non-pathological
examinations) was determined by comparison with a known ground truth
diagnosis from three to five reporting radiographers and/or radiologists.
ANOVA, t-tests and descriptive statistics were used to describe the data.
Results or Findings: There is a statistically significant difference in diagnostic
accuracy of all participants before any AI feedback and after provision of binary
diagnosis, when the AI is correct (p=0.007). In pathological cases there was a
statistically significant decrease in accuracy following presentation of the
heatmap (p=0.015) and a subsequent increase in accuracy when the binary
diagnosis was provided (p=0.013).
Conclusion: Heatmap feedback was detrimental to participant accuracy in
pathological cases and did not have a positive impact on accuracy in any
condition. Binary diagnosis was beneficial to both groups across all of the four
conditions with a significant impact on accuracy when the AI was correct and in
pathological cases. Methods of explainability should be developed with
cognisance of the impact on the end-user.
Limitations: Participants were analysed as two broad groups. Impact of years
of professional experience on interaction with the AI will be investigated.
Ethics committee approval: This study was approved by the UU Nursing and
Health research filter committee, approval number FCNUR-20-035.
Funding for this study: Funding was received from the College of
Radiographers Industry Partnership award Scheme (CoRIPS) and Ulster
University
| Original language | English |
|---|---|
| Pages | 122 |
| Number of pages | 1 |
| Publication status | Published (in print/issue) - 31 Mar 2023 |
| Event | European Congress of Radiology 2023 - Austria, Vienna, Austria Duration: 1 Mar 2023 → 5 Mar 2023 https://www.myesr.org/congress |
Conference
| Conference | European Congress of Radiology 2023 |
|---|---|
| Abbreviated title | ECR 2023 |
| Country/Territory | Austria |
| City | Vienna |
| Period | 1/03/23 → 5/03/23 |
| Internet address |
