Think Aloud Study for Knowledge (TASK) about AI user interface perceptions

Description

Background


Understanding and comprehending the cognitive behaviours of healthcare staff when interacting with Artificial Intelligence (AI)-User Interfaces (UI) is lacking in the literature. By gathering healthcare staff preferences of chest radiograph AI-based UIs, it is envisaged that this gap can be addressed. Human and machine interaction involves the UI that clinicians use. However, there is currently a lack of UI standardisation within medical imaging and AI (Schalekamp, Klein and van Leeuwen, 2022). The significant benefits to the production of a standardised chest radiograph AI UI, could enhance the adoption of AI technology within clinical environments. AI technology acceptance is currently resisted within radiology departments due to fear of role replacement (Chen et al., 2021). The design and creation of an AI UI output from the feedback of users i.e., those involved in preliminary commenting, pathology diagnoses and decision making, will enhance and encourage integration and acceptance of AI use in clinical departments.

Methods

A qualitative Think-Aloud method was undertaken with diagnostic radiographers’ and radiologists’ on their visual preferences when reviewing four types of chest radiograph AI UIs in a set of 22 chest radiographs. Methods investigated were salience maps, textual reports, area of interest and an abnormality score in UI. Participants were asked to review the images and say how each method impacted their thinking i.e. using the concurrent “Think-Aloud” method. Perceived levels of confidence against the four different interfaces were also collected.

Results/findings:

24 participants took part in the study which allowed an understanding of which components of the chest radiograph UI are preferred. Clear themes on the need for a non-distracting and plain UI e.g. without use of heat maps, to relay results of the AI algorithm were highlighted. Additionally, a dislike for the AI UIs that had too much detail, i.e. large amount of text, was also emphasized. Insights on participant preference were further detailed from the Think-Aloud and post-study confidence data, in which analysis is ongoing.

Conclusion:

Understanding the user preferences for chest radiograph AI EUI is important to ensure appropriate user engagement with the information provided by the technology. In turn, it gives radiographers and radiologists the ability to explain how the AI is being used to contribute to patient management.

Period	10 Oct 2025
Event title	EuSoMII
Event type	Conference