Lightweight deep models based on domain adaptation and network pruning for breast cancer HER2 scoring: IHC vs. H&E histopathological images

Human epidermal growth factor receptor 2 (HER2)-positive breast cancer is an aggressive cancer type that requires special diagnosis and treatment methods. Immunohistochemistry (IHC) staining effectively highlights relevant morphological structures within histopathological images yet can be expensive in terms of both labor and required laboratory equipment. Hematoxylin and eosin (H&E) images are more readily available and less expensive than IHC images as they are routinely performed for all patient samples. Lightweight models are well-suited for deployment on resource-constrained devices such as mobile phones and embedded systems, making them ideal for real-time diagnosis in rural regions and developing countries. In this study, IHC images are compared to H&E images for automatic HER2 scoring using lightweight deep models that incorporate several advanced techniques including network pruning, domain adaptation, and attention mechanisms. Two lightweight models are presented: PrunEff4 and ATHER2. PrunEff4 is a subset of EfficientNetV2B0 pruned to reduce the network parameters by ~80%. ATHER2 is a customized lightweight network that employs different sized convolutional filters along with a convolutional block attention module (CBAM). For PrunEff4 and ATHER2, transfer learning (pretraining on ImageNet) and domain-specific pretraining were employed, respectively. Different datasets were utilized in the development and final testing phases in order to effectively evaluate their generalization capability. In all experiments, both networks resulted in accuracies ranging from 97% to 100% for binary classifications and from 95.5% to 98.5% for multiclass classifications regardless of whether IHC or H&E images were utilized. Network pruning significantly reduced the network parameters whilst maintaining reliable performance. Domain-specific pretraining significantly enhanced performance, particularly in complex classification tasks such as HER2 scoring using H&E images and multiclass classifications. Both IHC and H&E stained images were suitable for deep learning-based HER2 scoring, given that the deep networks are efficiently trained for the specified task.