Robustness of Question Answering Systems in the Biomedical Domain: a study of the BioASQ dataset

Robustness is a critical consideration when integrating artificial intelligence (AI) systems into decision-making processes. This concern is particularly relevant for generative AI systems, which are designed to consistently produce convincing outputs but can be prone to hallucinations, risking overconfidence in their outputs. This paper investigates the performance of fine-tuning and retrieval augmented generation (RAG) under external data quality perturbations, including typographical errors and factual inaccuracies. Results from experiments using the BioASQ Task 12b question answering dataset and PubMed articles showed nuanced trade-offs, with either RAG or fine-tuning performing better for different scenarios. Furthermore, an analysis of LLMs’ self-reported confidence scores indicated a tendency toward overconfidence, particularly in the presence of inconsistent or erroneous context data. A novel mitigation strategy, leveraging an LLM for data quality error correction was evaluated, but the results demonstrated limited effectiveness, highlighting the need for more advanced correction techniques.