Clinical Entity Extraction: Comparison between MetaMap, cTAKES, CLAMP and Amazon Comprehend Medical

In recent years, electronic health records (EHRs) have been adopted widely and there is an increasing need to extract useful clinical information from free-text clinical notes. In this work, we compare the performance of the clinical entity extraction tools including MetaMap, cTAKES, CLAMP and Amazon Comprehend Medical. The clinical notes dataset we use in this work is i2b2 Obesity Challenge dataset. The experiments are designed to extract a list of the clinical entities related to obesity symptoms or clinical conditions using four different clinical entity extraction tools. The medical entities were manually annotated by two obesity experts in the dataset which are used as the ground truth. The evaluation has been done by using evaluation metrics including precision, recall, and F1score and comparison has been made of different clinical entity extraction tools and APIs. The results show that MetaMap has the highest recall (0.61) and F1-score (0.70) and CLAMP has the highest precision (0.98) of the averages for all the selected clinical conditions. However, for certain clinical conditions, cTAKES and Amazon Comprehend Medical outperform other tools. The results demonstrate that these clinical entity extraction tools are able to automatically and accurately extract useful information from the clinical notes.