Description
Purpose or Learning ObjectiveAs artificial intelligence (AI) becomes more integrated into healthcare it is imperative to deploy this technology responsibly. This research attempts to add insight into the effect of forms of AI feedback on the interpretation accuracy of student and qualified radiographers, highlighting optimal aspects of AI feedback.
Methods or Background
A study was designed in Qualtrics®, deployed at ECR 2021 and promoted via social media (Twitter®/LinkedIn®), resulting in 94 respondents. Each participant was presented with a random selection of three musculoskeletal examinations from a dataset of 21. Diagnostic accuracy under four conditions (AI correct, AI incorrect, pathological and non-pathological examinations) was determined by comparison with a known ground truth diagnosis from three to five reporting radiographers and/or radiologists. ANOVA, t-tests and descriptive statistics were used to describe the data.
Results or Findings
There is a statistically significant difference in diagnostic accuracy of all participants before any AI feedback and after provision of binary diagnosis, when the AI is correct (p=0.007).
In pathological cases there was a statistically significant decrease in accuracy following presentation of the heatmap (p=0.015) and a subsequent increase in accuracy when the binary diagnosis was provided (p=0.013).
Conclusion
Heatmap feedback was detrimental to participant accuracy in pathological cases and did not have a positive impact on accuracy in any condition.
Binary diagnosis was beneficial to both groups across all of the four conditions with a significant impact on accuracy when the AI was correct and in pathological cases.
Methods of explainability should be developed with cognisance of the impact on the end-user.
Limitations
Participants were analysed as two broad groups. Impact of years of professional experience on interaction with the AI will be investigated.
| Period | 1 Mar 2023 → 5 Mar 2023 |
|---|---|
| Event type | Conference |
| Location | Vienna, AustriaShow on map |
| Degree of Recognition | International |