CIMTD: Class Incremental Multi-Teacher Knowledge Distillation for Fractal Object Detection

Abstract

In practical industrial vision inspection tasks, acquiring foreign object data proves challenging due to its scarcity and unique characteristics. Industrial foreign objects often exhibit fractal shapes, with rough or fragmented geometrical contours, making them difficult to discern. Given the continuous emergence of new industrial foreign objects on production lines, ensuring the accuracy and speed of detecting both base and novel foreign objects with a limited number of samples is imperative. To address this challenge, we propose an end-to-end multi-teacher knowledge distillation detection framework (CIMTD) based on YOLO. By incorporating the Wasserstein Distance of teacher confidence, the limitation of precise point-to-point matching inherent in KL divergence in traditional knowledge distillation methods has been mitigated. This approach enables students to systematically and logically grasp the knowledge imparted by teachers. Additionally, our design choice of multi-teacher padding instead of cropping allows the distillation network to address the class increment problem at the strategy level. Adaptive regression distillation empowers the model to autonomously determine whether to learn bounding box information from teachers based on discrepancies between teacher and student models, enhancing detection speed without compromising accuracy. Extensive experiments conducted on the IGBT surface foreign body dataset underscore the potential of our module design and strategy implementation, achieving 59.38% mAP, outperforming existing class incremental knowledge distillation methods. Our framework provides a more deployable industrial vision solution for edge-side devices with limited computational resources.

This work was supported by National Key R&D Program of China (2019YFB1705002), National Natural Science Foundation of China (51634002), LiaoNing Revitalization Talents Program (XLYC2002041).