Skip to main content

Advertisement

Springer Nature Link
Log in

Study on end-to-end detection method for surface defects of automotive sheet metal parts

  • Research
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

Sheet metal parts account for more than 60% of the total automotive parts, and their defects can seriously affect the safety of automobile operations. Therefore, it is very important to detect defects in sheet metal parts during the production process. Due to the small size of defects in sheet metal parts, and high detection precision required, the traditional detection method cannot meet the requirements. And the factory production speed is fast, if the detection speed is low, it will cause defects to escape. Therefore, we propose an end-to-end detection method for automotive sheet metal parts surface defects. To effectively improve the detection speed, the dual regression classification strategy is proposed, which removes the NMS post-processing. Gradient information branch is added to provide rich gradient information for the model and mitigate the information loss during long convolution. Use the SPD-Conv module, optimized for small-size defects detection, to retain complete space information. Finally, the model is evaluated on the automotive sheet metal parts defect dataset. The experimental results show that the proposed method is superior to the benchmark methods in precision and speed, with mAP of 92.32% and FPS of 39.06, which achieves end-to-end detection.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Explore related subjects

Discover the latest articles and news from researchers in related subjects, suggested using machine learning.

Data availability

The algorithm presented in this article is based on a company project. To obtain the corresponding data, please contact the author.

References

  1. Zhao, D., Chen, H., Jia, W.: Examining user satisfaction and vehicle development for Mini EVs and Non-mini EVs. Transp. Res. Part A Policy Pract. 180, 103952 (2024)

    Article  Google Scholar 

  2. Fan, L., Wang, X., Yang, J., Liu, Y., Lv, C., Yu, H., Ma, J., Wang, F.-Y.: Social radars for social vision of intelligent vehicles: a new direction for vehicle research and development. IEEE Trans. Intell. Veh. 9, 4244–4248 (2024)

    Article  Google Scholar 

  3. Trzepieciński, T., Najm, S.M.: Current trends in metallic materials for body panels and structural members used in the automotive industry. Materials 17, 590 (2024)

    Article  Google Scholar 

  4. Li, S., Wang, H., Zhang, Y., Zhou, Y., Liu, C.: A novel method for necking detection and measurement in automotive sheet metal components. Meas. Sci. Technol. 35, 056001 (2024)

    Article  Google Scholar 

  5. Xu, Y.: Universal formability technology and applications. J. Mater. Process. Technol. 151, 119–125 (2004)

    Article  Google Scholar 

  6. Rodil, S.S., Gómez, R.A., Bernárdez, J.M., Rodríguez, F., Miguel, L.J., Perán, J.R.: Laser welding defects detection in automotive industry based on radiation and spectroscopical measurements. Int. J. Adv. Manuf. Technol. 49, 133–145 (2010)

    Article  Google Scholar 

  7. Han, B., Xiang, H., Li, Z., Huang, J.: Defects detection of sheet metal parts based on halcon and region morphology. Appl. Mech. Mater. 365–366, 729–732 (2013)

    Article  Google Scholar 

  8. Guan, S.: Fabric defect delaminating detection based on visual saliency in HSV color space. J. Text. Inst. 109, 1560–1573 (2018)

    Article  Google Scholar 

  9. Li, C., Lan, H.-Q., Sun, Y.-N., Wan, J.-Q.: Detection algorithm of defects on polyethylene gas pipe using image recognition. Int. J. Pressure Vessels Piping 191, 104381 (2021)

    Article  Google Scholar 

  10. Zhang, X., Gao, B., Wu, T., Woo, W., Fan, J., Zhan, S.: Differentiate tensor low rank soft decomposition in thermography defect detection. NDT E Int. 139, 102902 (2023)

    Article  Google Scholar 

  11. Liu, S., Dong, S., Zhang, Y., Zhang, C., Jin, L., Yang, Q., Zhang, C.: Defect detection in cylindrical cavity by electromagnetic ultrasonic creeping wave. IEEE Trans. Magn. 54, 1–5 (2018)

    Google Scholar 

  12. Shao, R., Zhou, M., Li, M., Han, D., Li, G.: TD-Net:tiny defect detection network for industrial products. Complex Intell. Syst. 10, 3943–3954 (2024)

    Article  Google Scholar 

  13. Yixuan, L., Dongbo, W., Jiawei, L., Hui, W.: Aeroengine blade surface defect detection system based on improved faster RCNN. Int. J. Intell. Syst. 2023, 1–14 (2023)

    Article  Google Scholar 

  14. Xu, Y., Li, D., Xie, Q., Wu, Q., Wang, J.: Automatic defect detection and segmentation of tunnel surface using modified Mask R-CNN. Measurement 178, 109316 (2021)

    Article  Google Scholar 

  15. Li, W., Zhang, H., Wang, G., Xiong, G., Zhao, M., Li, G., Li, R.: Deep learning based online metallic surface defect detection method for wire and arc additive manufacturing. Robot. Comput. Integr. Manuf. 80, 102470 (2023)

    Article  Google Scholar 

  16. Lin, J., Yao, Y., Ma, L., Wang, Y.: Detection of a casting defect tracked by deep convolution neural network. Int. J. Adv. Manuf. Technol. 97, 573–581 (2018)

    Article  Google Scholar 

  17. Liu, G., Yan, Y., Meng, J.: Study on the detection technology for inner-wall outer surface defects of the automotive ABS brake master cylinder based on BM-YOLOv8. Meas. Sci. Technol. 35, 055109 (2024)

    Article  Google Scholar 

  18. Allam, A., Moussa, M., Tarry, C., Veres, M.: Detecting teeth defects on automotive gears using deep learning. Sensors 21, 8480 (2021)

    Article  Google Scholar 

  19. Cheng, S., Lu, J., Yang, M., Zhang, S., Xu, Y., Zhang, D., Wang, H.: Wheel hub defect detection based on the DS-Cascade RCNN. Measurement 206, 112208 (2023)

    Article  Google Scholar 

  20. Zhang, J., Xu, J., Zhu, L., Zhang, K., Liu, T., Wang, D., Wang, X.: An improved MobileNet-SSD algorithm for automatic defect detection on vehicle body paint. Multimed. Tools Appl. 79, 23367–23385 (2020)

    Article  Google Scholar 

  21. Jiang, W., Chen, X., He, Y., Wang, X., Hu, S., Lu, M.: Semi-supervised method for visual detection of automotive paint defects. Meas. Sci. Technol. 35, 085902 (2024)

    Article  Google Scholar 

  22. Neubeck, A. and Van Gool, L.: Efficient non-maximum suppression. In: 18th International Conference on Pattern Recognition (ICPR’06) 850–855 (2006)

  23. Zhou, X., Wang, D. and Krähenbühl, P.: Objects as Points. arXiv:1904.07850 (2019)

  24. Hu, H., Gu, J., Zhang, Z., Dai, J. and Wei, Y.: Relation networks for object detection. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition 3588–3597 (2017)

  25. Sun, P., Jiang, Y., Xie, E., Shao, W., Yuan, Z., Wang, C. and Luo, P.: What makes for end-to-end object detection? Int. Conf. Mach. Learn. 139, 9934–9944 (2020)

    Google Scholar 

  26. Wang, J., Song, L., Li, Z., Sun, H., Sun, J. and Zheng, N.: End-to-end object detection with fully convolutional network. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 15844–15853 (2021)

  27. Cai, Y., Zhou, Y., Han, Q., Sun, J., Kong, X., Li, J. and Zhang, X.: Reversible Column Networks. arXiv:2212.11696 (2022)

  28. Lee, C.-Y., Xie, S., Gallagher, P., Zhang, Z. and Tu, Z.: Deeply-Supervised Nets. arXiv:1409.5185 (2014)

  29. Shen, Z., Liu, Z., Li, J., Jiang, Y.-G., Chen, Y., Xue, X.: Object detection from scratch with deep supervision. IEEE Trans. Pattern Anal. Mach. Intell. 42, 398–412 (2020)

    Article  Google Scholar 

  30. Wang, L., Lee, C.-Y., Tu, Z. and Lazebnik, S.: Training deeper convolutional networks with deep supervision. arXiv:1505.02496 (2015)

  31. Dai, J., Qi, H., Xiong, Y., Li, Y., Zhang, G., Hu, H. and Wei, Y.: Deformable convolutional networks. In: IEEE International Conference on Computer Vision (ICCV) 764–773 (2017)

  32. Bell, S., Zitnick, C. L., Bala, K. and Girshick, R.: Inside-Outside Net: Detecting objects in context with skip pooling and recurrent neural networks. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2874–2883 (2015)

  33. Tian, D., Han, Y., Wang, S.: Object feedback and feature information retention for small object detection in intelligent transportation scenes. Expert Syst. Appl. 238, 121811 (2024)

    Article  Google Scholar 

  34. Shin, Y., Shin, H., Ok, J., Back, M., Youn, J., Kim, S.: DCEF2-YOLO: Aerial detection YOLO with deformable convolution–efficient feature fusion for small target detection. Remote Sens. 16, 1071 (2024)

    Article  Google Scholar 

  35. Lin, T.-Y., Goyal, P., Girshick, R., He, K. and Dollár, P.: Focal Loss for dense object detection. In: IEEE International Conference on Computer Vision (ICCV) 2999–3007 (2017)

  36. Project webpage 2020 (available at: https://github.com/ultralytics/YOLOv5) Google Scholar

  37. Project webpage 2023 (available at: https://github.com/ultralytics/YOLOv8) Google Scholar

  38. Zhang, H., Chang, H., Ma, B., Wang, N., Chen, X.: Dynamic R-CNN: towards high quality object detection via dynamic training. Comput. Vis.-ECCV 2020 12360, 260–275 (2020)

    Article  Google Scholar 

  39. Ren, S., He, K., Girshick, R., Sun, J.: Faster r-cnn: towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 38, 1137–1149 (2017)

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to acknowledge the support of the Department of Science and Technology of Guangxi Zhuang Autonomous Region. The project is also supported by the SAIC-GM-Wuling Automobile Co., Ltd. We collected all images data in SAIC-GM-Wuling Automobile Co., Ltd. Restrictions apply to the availability of these data, which were used under license for this study.

Author information

Authors and Affiliations

  1. School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, People’s Republic of China

    Wei Dai & Sun Jin

  2. SAIC-GM-Wuling Automobile Co., Ltd., Liuzhou, People’s Republic of China

    Juncheng lv & Rui Xiang

Authors
  1. Wei Dai
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Juncheng lv
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Rui Xiang
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Sun Jin
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

Wei Dai was responsible for the modeling, experiments, and manuscript writing for this article. Juncheng lv, Rui Xiang was responsible for the production of the datasets used in the article as well as the design and construction of the detection room. Sun Jin was responsible for designing the experiments and for touching up and revising the manuscript. All authors reviewed the manuscript.

Corresponding author

Correspondence to Wei Dai.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

1.1 A. In-line detection room system design

We have developed a highly flexible, low-cost in-line inspection chamber system that realizes full-size part field-of-view coverage by means of a rational camera arrangement. Obtain the optimal field of view and working distance of the camera based on actual test results. The number of cameras required is calculated by distributing them according to the maximum contour size of the sheet metal part. To avoid missed detection due to uncovered detection areas, we configure the corresponding camera working parameters for different parts, so that the system can realize precise detection. The internal and external structure of the in-line detection room is shown in Fig. 11.

Fig. 11
figure 11

Structure of the in-line detection room

Full size image

To acquire a clear image of the parts under high-speed movement, we arrange 6 cameras in the detection room. They are mounted on holders and connected to the detection room frame by connecting rods. In the meantime, the holder can be moved in position along the connecting rod as well as spatially adjusted in angle. The frame of the detection room is made of standard aluminum alloy and is bolted together for flexible adjustment and facilitating disassembly and installation. The lens parameters and camera parameters are shown in Table 6, Table 7 respectively.

To avoid problems such as reflections and darkness of sheet metal parts, we have canceled the direct lighting method of the light source after many experimental tests and replaced it with the diffuse reflection lighting method. The rectangular light source is directed to the soft cloth around the detection room, and the light is illuminated on the parts through reflection, realizing light homogenization and softening. Subsequent to the experiment, the mean light intensity in the 0 ~ 200 mm range along the z-axis direction of the detection area was 1700 LX, which is approximately eight times the ambient light intensity.

1.2 B. Workflow of the detection system

Table 6 Lens parameters
Full size table
Table 7 Camera parameters
Full size table

To ensure that clear and complete images can be obtained for different parts at different production beats, it is necessary to initialize the settings of the operating parameters for taking pictures. The operational parameters of the camera are also entered into the control system. The sheet metal parts to be detected are transported to the in-line detection room by conveyor belt, and the signals are transmitted to the control system for shooting through the trigger grating, as shown in Fig. 12. Immediately after the camera has taken the picture, the image is entered into the computer to complete the defect detection and analyze the results. If a single workpiece is defective, the workpiece is repaired or scrapped at the end of the line. If several consecutive workpieces are defective, the line will be stopped for repair

Fig. 12
figure 12

Sheet metal parts detection process

Full size image

1.3 C. Effectiveness of implementation

The in-line detection system for sheet metal parts has detected 81,670,000 parts in actual production applications, with a total cumulative number of 1160 misdetections and a comprehensive misdetection rate of 1.42%. This covered 35 parts for 10 automotive models, with an average false positive rate of less than 5%.

At the initial stage of research and application, the overall detection rate of the system was around 80% due to the small number of defect samples, inaccurate camera operating parameters, and environmental factors within the factory. With continuous data accumulation, parameter optimization, image processing, and camera optimization, the detection effect is gradually enhanced to meet the technical specifications.

1.4 D. Analysis and discussion

In the computer vision detection of automotive sheet metal parts, weighing the relationship between detection speed and precision is a crucial issue. A contradiction often exists between the two, and the optimal balance must be found when designing a detection system. This balance is based on the application scenario and requirements.

In reality, the pace of production along the production line accelerates in response to escalating output demands. The system must be capable of detecting each component in a brief period, as failure to do so would result in data loss or the inability to maintain pace with production. The precision of the inspection is also particularly important, with the recall rate being the most important precision indicator in the factory. If some critical defects are not detected, the subsequent processing will be seriously affected.

We classify the importance of individual defects by setting a processing priority for them. For important defects, increase the precision required for their detection. For defects like oil stains, which are difficult to detect but have a negligible impact on subsequent processing, the detection requirements can be appropriately relaxed. This approach facilitates both the lightweighting of the model and the more targeted detection of defects. This is the direction of our subsequent research.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dai, W., lv, J., Xiang, R. et al. Study on end-to-end detection method for surface defects of automotive sheet metal parts. J Real-Time Image Proc 22, 80 (2025). https://doi.org/10.1007/s11554-025-01656-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11554-025-01656-4

Keywords