Level set enhanced weakly supervised learning network for segmentation of dense cell nuclei

Cell nucleus segmentation is crucial to qualitative and quantitative pathology analysis, which can assist pathologists to diagnose cancer, monitor the changes in cells, and help to improve the treatment efficacy, etc. Compared with the fully supervised deep learning methods which require a large number of pixel-level annotations, weakly supervised learning methods only need simple and small number of annotations, however, they usually have inferior performance in cell nucleus segmentation. Moreover, effective and accurate segmentation of overlapped and adjacent cell nuclei is a challenge both for fully and weakly supervised learning methods. In this study, we proposed a level set embedded weakly supervised learning network (LS-Net), which incorporates a detection branch for center point localization and bounding box generation, and a segmentation branch with level set function to achieve the accurate nucleus segmentation, where the level set function is evolved by Chan-Vese energy function within the corresponding bounding box. The experiments on the two public datasets (MoNuSeg and CPM-17) and a constructed real-world clinical dataset demonstrate that LS-Net has excellent performance across detection, segmentation, and generalization ability. Especially, it is outstanding in the segmentation of overlapped and adjacent cell nuclei with strong generalization ability. This work would effectively improve the accurate analysis of dense cell nuclei in pathological images, which can assist precision medicine.