Practical Strategies for Extreme Missing Data Imputation in Dementia Diagnosis

Research output: Contribution to journalArticlepeer-review

1 Downloads (Pure)

Abstract

Accurate computational models for clinical decision support systems require clean and reliable data but, in clinical practice, data are often incomplete. Hence, missing data could arise not only from training datasets but also test datasets which could consist of a single undiagnosed case, an individual. This work addresses the problem of extreme missingness in both training and test data by evaluating multiple imputation and classification workflows based on both diagnostic classification accuracy and computational cost. Extreme missingness is defined as having ~50% of the total data missing in more than half the data features. In particular, we focus on dementia diagnosis due to long time delays, high variability, high attrition rates and lack of practical data imputation strategies in its diagnostic pathway. We identified and replicated the extreme missingness structure of data from a real-world memory clinic on a larger open dataset, with the original complete data acting as ground truth. Overall, we found that computational cost, but not accuracy, varies widely for various imputation and classification approaches. Particularly, we found that iterative imputation on the training dataset combined with a reduced-feature classification model provides the best approach, in terms of speed and accuracy. Taken together, this work has elucidated important factors to be considered when developing a predictive model for a dementia diagnostic support system.
Original languageEnglish
Number of pages10
JournalIEEE Journal of Biomedical and Health Informatics
Early online date21 Jul 2021
DOIs
Publication statusE-pub ahead of print - 21 Jul 2021

Keywords

  • Clinical decision support systems
  • medical expert systems
  • machine learning
  • missing data
  • data imputation
  • dementia
  • ADNI data
  • Alzheimer’s disease classification
  • data quality

Fingerprint

Dive into the research topics of 'Practical Strategies for Extreme Missing Data Imputation in Dementia Diagnosis'. Together they form a unique fingerprint.

Cite this