A knowledge distillation-based multi-scale relation-prototypical network for cross-domain few-shot defect classification

Jiaqi Zhao, Xiaolong Qian, Yunzhou Zhang, Dexing Shan, Xiaozheng Liu, Sonya Coleman, Dermot Kerr

Research output: Contribution to journalArticlepeer-review

5 Citations (Scopus)
95 Downloads (Pure)


Surface defect classification plays a very important role in industrial production and mechanical manufacturing. However, there are currently some challenges hindering its use. The first is the similarity of different defect samples makes classification a difficult task. Second, the lack of defect samples leads to poor accuracies when using deep learning methods. In this paper, we first design a novel backbone network, ResMSNet, which draws on the idea of multi-scale feature extraction for small discriminative regions in defect samples. Then, we introduce few-shot learning for defect classification and propose a Relation-Prototypical network (RPNet), which combines the characteristics of ProtoNet and RelationNet and provides classification by linking the prototypes distances and the nonlinear relation scores. Next, we consider a more realistic scenario where the base dataset for training the model and target defect dataset for applying the model are usually obtained from domains with large differences, called cross-domain few-shot learning. Hence, we further improve RPNet to KD-RPNet inspired by knowledge distillation methods. Through extensive comparative experiments and ablation experiments, we demonstrate that either our ResMSNet or RPNet proves its effectiveness and KD-RPNet outperforms other state-of-the-art approaches for few-shot defect classification.

Original languageEnglish
JournalJournal of Intelligent Manufacturing
Early online date5 Feb 2023
Publication statusPublished online - 5 Feb 2023

Bibliographical note

Funding Information:
Funding was provided by National Natural Science Foundation of China (Grant Nos. 61973066 and 61471110), Major Science and Technology Projects of Liaoning (Grant No. 2021JH1/10400049), Fundation of Key Laboratory of Aerospace System Simulation (Grant No. 6142002200301), Fundation of Key Laboratory of Equipment Reliability (Grant No. WD2C20205500306), Open Research Projects of Zhejiang Lab (Grant No. 2019KD0AD01/006), and Major Science and Technology Innovation Engineering Projects of Shandong Province (Grant No. 2019JZZY010128).

Publisher Copyright:
© 2023, The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature.


  • Cross-domain
  • Defect classification
  • Few-shot learning
  • Knowledge distillation
  • Multi-scale feature encoder


Dive into the research topics of 'A knowledge distillation-based multi-scale relation-prototypical network for cross-domain few-shot defect classification'. Together they form a unique fingerprint.

Cite this