Abstract
Recent advances have been made in applying convolutional neural networks to achieve more precise prediction results for medical image segmentation problems. However, the success of existing methods has highly relied on huge computational complexity and massive storage. To address this problem, we propose a plain and efficient network, named PEDNet. This architecture empowers the lightweight network to get a significant improvement in classification capability while retaining its runtime efficiency. In particular, we propose a cumbersome teacher network that combines ResNeXt and CBAM to achieve high-efficiency feature extraction and classification. Meanwhile, the lightweight ShuffleNetV2 model is designed to trade off performance and efficiency. We further devise a novel adversarial similarity distillation module (ASD) tailored for breast tumor image classification to transfer semantic feature information from teacher to student network. It forces the student network to mimic the extent of difference of representations calculated from different loss functions. The extensive experimental results on Dataset BUSI demonstrate that PEDNet outperforms various knowledge distillation counterparts.
T. Liu and Y. Wang—Contributed equally to this work.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
The International Agency for Research on Cancer, World Health Organization. Latest global cancer data: cancer burden rises to 19.3 million new cases and 10.0 million cancer deaths in 2020
Hooley, R., Scoutt, L., Philpotts, L.: Breast ultrasonography: state of the art. Radiology 268(3), 642–659 (2013)
Xiao, J., Li, J., Gao, H.: FS3DCIoT: a few-shot incremental learning network for skin disease differential diagnosis in the consumer IoT. IEEE Trans. Consum. Electron. 69(4), 786–797 (2023)
Benaouali, M., Bentoumi, M., Touati, M., et al.: Segmentation and classification of benign and malignant breast tumors via texture characterization from ultrasound images. In: 2022 7th International Conference on Image and Signal Processing and their Applications (ISPA), pp. 1–4. IEEE (2022)
Zaychenko, Y., Zaichenko, H., Hamidov, G.: Investigation of convolutional neural networks in the tasks of medical images analysis and classification of breast tumors. In: 2021 14th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI), pp. 1–6. IEEE, Shanghai, China (2021)
Jun, W., Liyuan, Z.: Brain tumor classification based on attention guided deep learning model. Int. J. Comput. Intell. Syst. 15(1), 35 (2022)
Aljuaid, H., Alturki, N., Alsubaie, N., et al.: Computer-aided diagnosis for breast cancer classification using deep neural networks and transfer learning. Comput. Methods Programs Biomed. 223, 106951 (2022)
Araújo, T., Aresta, G., Castro, E., et al.: Classification of breast cancer histology images using convolutional neural networks. PLoS ONE 12(6), e0177544 (2017)
Hinton, G., Vinyals, O., Dean, J.: Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 (2015)
Qin, D., Bu, J., Liu, Z., et al.: Efficient medical image segmentation based on knowledge distillation. IEEE Trans. Med. Imaging 40(12), 3820–3831 (2021)
Tran, M., Do, T., Tran, H., et al.: Light-weight deformable registration using adversarial learning with distilling knowledge. IEEE Trans. Med. Imaging 41(6), 1443–1453 (2022)
Xie, S., Girshick, R., Dollár, P., et al.: Aggregated residual transformations for deep neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5987–5995, Honolulu, HI, USA (2017)
Woo, S., Park, J., Lee, J.-Y., Kweon, I.S.: CBAM: convolutional block attention module. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11211, pp. 3–19. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01234-2_1
Ma, N., Zhang, X., Zheng, H., et al.: ShuffleNet V2: practical guidelines for efficient CNN architecture design. In: Proceedings of the European Conference on Computer Vision, pp. 116–131, Munich, Germany (2018)
Ruder, S.: An overview of gradient descent optimization algorithms. arXiv:1609.04747 (2016)
Zhang, M., Lucas. J.: Lookahead Optimizer: k steps forward, 1 step back. In: Advances in Neural Information Processing Systems (NeurlPS) (2019)
Al-Dhabyani, W., Gomaa, M., Khaled, H., et al.: Dataset of breast ultrasound images, Data in Brief, vol. 28, (2020)
Paszke, A., Gross, S.: Automatic differentiation in PyTorch (2017)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: International Conference on Learning Representations (ICLR) (2015)
Huang, G., Liu, Z., Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4700–4708 (2017)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778 (2016)
Acknowledgments
This study was funded by Henan province key science and technology research projects (grant numbers 212102210565 and 222102210028).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Liu, T. et al. (2024). PEDNet: A Plain and Efficient Knowledge Distillation Network for Breast Tumor Ultrasound Image Classification. In: Huang, DS., Zhang, X., Guo, J. (eds) Advanced Intelligent Computing Technology and Applications. ICIC 2024. Lecture Notes in Computer Science, vol 14866. Springer, Singapore. https://doi.org/10.1007/978-981-97-5594-3_34
Download citation
DOI: https://doi.org/10.1007/978-981-97-5594-3_34
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-97-5593-6
Online ISBN: 978-981-97-5594-3
eBook Packages: Computer ScienceComputer Science (R0)