Automated Identification of Label Errors in Large Electrocardiogram Datasets

Background: Training and testing Deep Neural Networks
(DNNs) for automated electrocardiogram (ECG)
interpretation requires large datasets. These datasets
are commonly extracted at scale from Electronic Health
Records (EHRs). Typically, a single physician over-reads
the machine generated interpretation as part of standard
care. Incorrect interpretation of the ECG occurs frequently,
reducing the quality of the labels.
Method: We trained a DNN to identify seven ECG
rhythms based on morphology; Sinus Rhythm, Junctional
Rhythm, Ectopic Atrial Rhythm, Atrial Flutter, Atrial Fibrillation,
Ventricular Rhythm and Pacemaker. The DNN
was trained on a dataset of 368,202 ECGs taken from a
proprietary database. We then applied confident learning
techniques using the DNN to identify label errors in the
Physionet PTB-XL database, which is publicly available.
Results: The confident learning algorithm identified 515
potential rhythm label errors in the 21,837 ECGs in PTBXL
database (2.36%). The labels were sorted by the likelihood
of label error based on the self-confidence score, and
the top 200 ECGs were manually reviewed. Of these 200
ECGs, 158 were found to be incorrectly labelled (79%).
Confident learning successfully corrected the label in 156
cases (78%). The estimated labelling error