A clinician’s guide to understanding and critically appraising machine learning studies: a checklist for Ruling Out Bias Using Standard Tools in Machine Learning (ROBUST-ML)

Developing functional machine learning-based models to address unmet clinical needs requires unique considerations for optimal clinical utility. Recent debates about the rigors, transparency, explainability, and reproducibility of machine learning models, terms which are defined in this article, have raised concerns about their clinical utility and suitability for integration in current evidence-based practice paradigms. This featured article focuses on increasing literacy of machine learning among clinicians by providing them with the knowledge and tools needed to understand and critically appraise clinical studies focused on machine learning. A checklist is provided for evaluating the rigor and reproducibility of the four machine learning building blocks: data curation, feature engineering, model development, and clinical deployment. Checklists like this are important for quality assurance and to ensure that machine learning studies are rigorously and confidently reviewed by clinicians and are guided by domain knowledge of the setting in which the findings will be applied. Bridging the gap between clinicians, healthcare scientists, and machine learning engineers can address many shortcomings and pitfalls of machine learning-based solutions and their potential deployment at the bedside.