Skip to main content

Advertisement

SpringerLink
Log in

Automated real-time detection of surface defects in manufacturing processes of aluminum alloy strip using a lightweight network architecture

  • Published:
Journal of Intelligent Manufacturing Aims and scope Submit manuscript

Abstract

The trade-off between detection speed and accuracy and the high hardware requirements of computing equipment have always been two major factors restricting the real-time detection and application of surface defects in aluminum strip. This paper proposes an effective, lightweight detection method for aluminum strip surface defects in industry, which improves the disadvantages of low efficiency and high calculation cost of the YOLOv4 framework. The backbone network GMANet is constructed based on a new convolution Ghost module, in which the union attention module is embedded in the stacked Ghost block. It realizes the compression of the network scale and focuses on the channel information of important feature maps. On this basis, the fusion neck network is redesigned and lightened by utilizing depthwise separable convolution and the sampling blocks of pixelshuffle and passthrough. It can reduce the information loss caused by sampling, and improve the extraction ability to multi-size features and the adaptive learning capability to weights. Moreover, the proposed method is trained and tested on the database of seven types of common defects collected from the quality inspection station of the cold rolling workshop. Experiments demonstrate that the proposed method achieves that the value of mAP is 94.68%, the model volume is reduced by 80.41%, and the detection speed is increased by three times, thereby outperforming the original YOLOv4 model. And it provides a research idea for the subsequent real-time detection of the aluminum strip surface on the embedded system.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  • Bochkovskiy, A., Wang, C.-Y., & Liao, H.-Y. M. (2020). YOLOv4: Optimal Speed and Accuracy of Object Detection. http://arxiv.org/abs/2004.10934.

  • Bustillo, A., Urbikain, G., Perez, J. M., Pereira, O. M., & Lopez de Lacalle, L. N. (2018). Smart optimization of a friction-drilling process based on boosting ensembles. Journal of Manufacturing Systems, 48, 108–121. https://doi.org/10.1016/J.JMSY.2018.06.004

    Article  Google Scholar 

  • Chang, F., Dong, M., Liu, M., Wang, L., & Duan, Y. (2020). A lightweight appearance quality assessment system based on parallel deep learning for painted car body. IEEE Transactions on Instrumentation and Measurement, 69(8), 5298–5307. https://doi.org/10.1109/TIM.2019.2962565

    Article  Google Scholar 

  • Chen, H., Pang, Y., Hu, Q., & Liu, K. (2020). Solar cell surface defect inspection based on multispectral convolutional neural network. Journal of Intelligent Manufacturing, 31(2), 453–468. https://doi.org/10.1007/s10845-018-1458-z

    Article  Google Scholar 

  • Choi, D. C., Jeon, Y. J., Kim, S. H., Moon, S., Yun, J. P., & Kim, S. W. (2017). Detection of pinholes in steel slabs using gabor filter combination and morphological features. ISIJ International, 57(6), 1045–1053. https://doi.org/10.2355/isijinternational.ISIJINT-2016-160

    Article  Google Scholar 

  • Chollet, F. (2017). Xception: Deep learning with depthwise separable convolutions. In Proceedings—30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, 2017-January (pp. 1800–1807). https://doi.org/10.1109/CVPR.2017.195.

  • Cui, L., Jiang, X., Xu, M., Li, W., Lv, P., & Zhou, B. (2021). SDDNet: A fast and accurate network for surface defect detection. IEEE Transactions on Instrumentation and Measurement, 70, 1–13. https://doi.org/10.1109/TIM.2021.3056744

    Article  Google Scholar 

  • Fu, G., Sun, P., Zhu, W., Yang, J., Cao, Y., Yang, M. Y., & Cao, Y. (2019). A deep-learning-based approach for fast and robust steel surface defects classification. Optics and Lasers in Engineering, 121, 397–405. https://doi.org/10.1016/J.OPTLASENG.2019.05.005

    Article  Google Scholar 

  • Girshick, R. (2015). Fast R-CNN. IEEE International Conference on Computer Vision (ICCV), 2015, 1440–1448. https://doi.org/10.1109/ICCV.2015.169

    Article  Google Scholar 

  • Girshick, Ross, Donahue, J., Darrell, T., & Malik, J. (2014). Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (pp. 580–587).https://doi.org/10.1109/CVPR.2014.81.

  • Han, K., Wang, Y., Tian, Q., Guo, J., Xu, C., & Xu, C. (2020). GhostNet: More features from cheap operations. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (pp. 1577–1586).https://doi.org/10.1109/CVPR42600.2020.00165.

  • Hao, R., Lu, B., Cheng, Y., Li, X., & Huang, B. (2021). A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing, 32(7), 1833–1843. https://doi.org/10.1007/s10845-020-01670-2

    Article  Google Scholar 

  • He, K., Gkioxari, G., Dollar, P., & Girshick, R. (2017). Mask R-CNN. In Proceedings of the IEEE International Conference on Computer Vision, 2017-October (pp. 2980–2988). https://doi.org/10.1109/ICCV.2017.322.

  • He, K., Zhang, X., Ren, S., & Sun, J. (2015). Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(9), 1904–1916. https://doi.org/10.1109/TPAMI.2015.2389824

    Article  Google Scholar 

  • Howard, A., Sandler, M., Chen, B., Wang, W., Chen, L. C., Tan, M., Chu, G., Vasudevan, V., Zhu, Y., Pang, R., Le, Q., & Adam, H. (2019). Searching for mobileNetV3. In Proceedings of the IEEE International Conference on Computer Vision, 2019-October (pp. 1314–1324). https://doi.org/10.1109/ICCV.2019.00140.

  • Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., & Adam, H. (2017). MobileNets: Efficient convolutional neural networks for mobile vision applications. ArXiv Pre-Print Server. http://arxiv.org/abs/1704.04861.

  • Huang, X. Q., Luo, X. B., & Wang, R. Z. (2015). A real-time parallel combination segmentation method for aluminum surface defect images. Proceedings—International Conference on Machine Learning and Cybernetics, 2, 544–549. https://doi.org/10.1109/ICMLC.2015.7340612

    Article  Google Scholar 

  • Iandola, F. N., Han, S., Moskewicz, M. W., Ashraf, K., Dally, W. J., & Keutzer, K. (2016). SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size. Revue Medicale Suisse, 8(336), 816. arXiv:1602.07360

  • Ioffe, S., & Szegedy, C. (2015). Batch normalization: Accelerating deep network training by reducing internal covariate shift. Journal of Molecular Structure, 1134, 63–66. https://doi.org/10.1016/j.molstruc.2016.12.061

    Article  Google Scholar 

  • Jiang, Z., Zhao, L., Shuaiyang, L. I., & Yanfei, J. I. A. (2020). Real-time object detection method for embedded devices. ArXiv, 3, 1–11.

    Google Scholar 

  • Jie, H., Li, S., Gang, S., & Albanie, S. (2017). Squeeze-and-excitation networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 99. https://doi.org/10.1109/TPAMI.2019.2913372

  • Kisantal, M., Wojna, Z., Murawski, J., Naruniec, J., & Cho, K. (2019). Augmentation for small object detection. 119–133. https://doi.org/10.5121/csit.2019.91713.

  • Li, D., Li, Y., Xie, Q., Wu, Y., Yu, Z., & Wang, J. (2021). Tiny defect detection in high-resolution aero-engine blade images via a coarse-to-fine framework. IEEE Transactions on Instrumentation and Measurement. https://doi.org/10.1109/TIM.2021.3062175

    Article  Google Scholar 

  • Li, J., Su, Z., Geng, J., & Yin, Y. (2018a). Real-time detection of steel strip surface defects based on improved YOLO detection network. IFAC-PapersOnLine, 51(21), 76–81. https://doi.org/10.1016/J.IFACOL.2018.09.412

    Article  Google Scholar 

  • Li, Y., Huang, H., Xie, Q., Yao, L., & Chen, Q. (2018b). Research on a surface defect detection algorithm based on MobileNet-SSD. Applied Sciences, 8(9), 1678. https://doi.org/10.3390/app8091678

    Article  Google Scholar 

  • Li, Y., Liu, J., & Wang, L. (2018c). Lightweight network research based on deep learning: A review. In 2018c 37th Chinese Control Conference (CCC) (pp. 9021–9026). https://doi.org/10.23919/ChiCC.2018.8483963.

  • Liu, H., He, B., He, Y., & Tao, X. (2021). Lightweight detection algorithm for fine-grained surface defects of aerospace seal rings. Journal of Intelligent and Fuzzy Systems, 40(6), 11423–11440. https://doi.org/10.3233/JIFS-202614

    Article  Google Scholar 

  • Liu, R., Sun, Z., Wang, A., Yang, K., Wang, Y., & Sun, Q. (2020). Real-time defect detection network for polarizer based on deep learning. Journal of Intelligent Manufacturing, 31(8), 1813–1823. https://doi.org/10.1007/s10845-020-01536-7

    Article  Google Scholar 

  • Liu, S., Qi, L., Qin, H., Shi, J., & Jia, J. (2018). Path Aggregation Network for Instance Segmentation. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (pp. 8759–8768).https://doi.org/10.1109/CVPR.2018.00913.

  • Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C. Y., & Berg, A. C. (2016). SSD: Single shot multibox detector. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 9905 LNCS (pp. 21–37). https://doi.org/10.1007/978-3-319-46448-0_2.

  • Liu, X., Xu, K., & Zhou, D. (2019). Improvements for the recognition rate of surface defects of aluminum sheets. Minerals, Metals and Materials Series. https://doi.org/10.1007/978-3-030-05864-7_66

    Article  Google Scholar 

  • Luo, Q., Fang, X., Liu, L., Yang, C., & Sun, Y. (2020). Automated visual defect detection for flat steel surface: A survey. IEEE Transactions on Instrumentation and Measurement, 69(3), 626–644. https://doi.org/10.1109/TIM.2019.2963555

    Article  Google Scholar 

  • Ma, N., Zhang, X., Zheng, H. T., & Sun, J. (2018). Shufflenet V2: Practical guidelines for efficient cnn architecture design. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 11218 LNCS (pp. 122–138). https://doi.org/10.1007/978-3-030-01264-9_8.

  • Qi, H., Xu, T., Wang, G., Cheng, Y., & Chen, C. (2020). MYOLOv3-Tiny: A new convolutional neural network architecture for real-time detection of track fasteners. Computers in Industry. https://doi.org/10.1016/j.compind.2020.103303

    Article  Google Scholar 

  • Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016). You only look once: Unified, real-time object detection. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2016-December (pp. 779–788). https://doi.org/10.1109/CVPR.2016.91.

  • Redmon, J., & Farhadi, A. (2017). YOLO9000: Better, faster, stronger. In Proceedings—30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, 2017-January (pp. 6517–6525). https://doi.org/10.1109/CVPR.2017.690.

  • Redmon, J., & Farhadi, A. (2018). YOLOv3: An incremental improvement. ArXiv Pre-Print Server. http://arxiv.org/abs/1804.02767.

  • Ren, Q., Geng, J., & Li, J. (2019). Slighter Faster R-CNN for real-time detection of steel strip surface defects. In Proceedings 2018 Chinese Automation Congress, CAC 2018 (pp. 2173–2178).https://doi.org/10.1109/CAC.2018.8623407.

  • Ren, S., He, K., Girshick, R., & Sun, J. (2017). Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6), 1137–1149. https://doi.org/10.1109/TPAMI.2016.2577031

    Article  Google Scholar 

  • Sánchez Egea, A. J., Rodríguez, A., Celentano, D., Calleja, A., & López de Lacalle, L. N. (2019). Joining metrics enhancement when combining FSW and ball-burnishing in a 2050 aluminium alloy. Surface and Coatings Technology, 367, 327–335. https://doi.org/10.1016/J.SURFCOAT.2019.04.010

    Article  Google Scholar 

  • Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L. C. (2018). MobileNetV2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (pp. 4510–4520).https://doi.org/10.1109/CVPR.2018.00474.

  • Shi, W., Caballero, J., Huszar, F., Totz, J., Aitken, A. P., Bishop, R., Rueckert, D., & Wang, Z. (2016). Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2016-December (pp. 1874–1883). https://doi.org/10.1109/CVPR.2016.207.

  • Wang, Q., Wu, B., Zhu, P., Li, P., Zuo, W., & Hu, Q. (2020). ECA-Net: Efficient channel attention for deep convolutional neural networks. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (pp. 11531–11539).https://doi.org/10.1109/CVPR42600.2020.01155.

  • Wei, R., & Bi, Y. (2019). Research on recognition technology of aluminum profile surface defects based on deep learning. Materials. https://doi.org/10.3390/MA12101681

    Article  Google Scholar 

  • Weichert, D., Link, P., Stoll, A., Rüping, S., Ihlenfeldt, S., & Wrobel, S. (2019). A review of machine learning for the optimization of production processes. The International Journal of Advanced Manufacturing Technology, 104(5–8), 1889–1902. https://doi.org/10.1007/s00170-019-03988-5

    Article  Google Scholar 

  • Woo, S., Park, J., Lee, J. Y., & Kweon, I. S. (2018). CBAM: Convolutional block attention module. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 11211 LNCS (pp. 3–19). https://doi.org/10.1007/978-3-030-01234-2_1.

  • Wu, D., Lv, S., Jiang, M., & Song, H. (2020). Using channel pruning-based YOLO v4 deep learning algorithm for the real-time and accurate detection of apple flowers in natural environments. Computers and Electronics in Agriculture. https://doi.org/10.1016/j.compag.2020.105742

    Article  Google Scholar 

  • Wu, W.-Y., Hung, C.-W., & Yu, W.-B. (2013). The development of automated solder bump inspection using machine vision techniques. The International Journal of Advanced Manufacturing Technology, 69(1–4), 509–523. https://doi.org/10.1007/s00170-013-4994-x

    Article  Google Scholar 

  • Yao, D., Liu, H., Yang, J., & Zhang, J. (2020). Implementation of a novel algorithm of wheelset and axle box concurrent fault identification based on an efficient neural network with the attention mechanism. Journal of Intelligent Manufacturing, 32(3), 729–743. https://doi.org/10.1007/s10845-020-01701-y

    Article  Google Scholar 

  • Zhai, M., Fu, S., Gu, S., Xie, Z., & Luo, X. (2011). Defect detection in aluminum foil by measurement-residual-based chi-square detector. International Journal of Advanced Manufacturing Technology, 53(5–8), 661–667. https://doi.org/10.1007/s00170-010-2863-4

    Article  Google Scholar 

  • Zhang, J., Wang, H., Tian, Y., & Liu, K. (2020). An accurate fuzzy measure-based detection method for various types of defects on strip steel surfaces. Computers in Industry, 122, 103231. https://doi.org/10.1016/J.COMPIND.2020.103231

    Article  Google Scholar 

  • Zhang, X., Zhou, X., Lin, M., & Sun, J. (2017). ShuffleNet: An extremely efficient convolutional neural network for mobile devices. The Black Studies Reader. https://doi.org/10.4324/9780203491348

    Article  Google Scholar 

  • Zhao, Y. J., Yan, Y. H., & Song, K. C. (2017). Vision-based automatic detection of steel surface defects in the cold rolling process: Considering the influence of industrial liquids and surface textures. The IntErnational Journal of Advanced Manufacturing Technology, 90(5–8), 1665–1678. https://doi.org/10.1007/s00170-016-9489-0

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the Guangxi Specially-invited Experts Foundation of Guangxi Zhuang Autonomous Region, China (GuiRenzi2019(13)). Moreover, we would like to acknowledge support of our work from Guangxi Liuzhou Yinhai Aluminum Company Limited.

Author information

Authors and Affiliations

  1. School of Mechanical and Electrical Engineering, Central South University, Changsha, 410083, Hunan, China

    Zhuxi Ma & Si Tang

  2. Light Alloy Research Institute, Central South University, Changsha, 410083, Hunan, China

    Yibo Li, Minghui Huang & Jie Cheng

  3. Guangxi Liuzhou Yinhai Aluminum Company Limited, Liuzhou, 545006, Guangxi, China

    Qianbin Huang

Authors
  1. Zhuxi Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yibo Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Minghui Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Qianbin Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jie Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Si Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Yibo Li or Minghui Huang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ma, Z., Li, Y., Huang, M. et al. Automated real-time detection of surface defects in manufacturing processes of aluminum alloy strip using a lightweight network architecture. J Intell Manuf 34, 2431–2447 (2023). https://doi.org/10.1007/s10845-022-01930-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10845-022-01930-3

Keywords

Search

Navigation