Skip to main content
Springer Nature Link
Log in

The 3D narrow butt weld seam detection system based on the binocular consistency correction

  • Published:
Journal of Intelligent Manufacturing Aims and scope Submit manuscript

Abstract

Detecting narrow butt weld seam with high precision has become an urgent problem with the wide application of laser welding technology. Many previous methods use line laser to locate the welds. However, these methods can only get a single position of the weld seam in each shooting and the detection scope is limited to the laser projection area, leading to low detection efficiency. To extract the narrow butt welds more efficiently, this paper combines the passive methods with the active methods, and proposes a 3D narrow butt weld seam detection system based on the binocular consistency analysis. Specifically, the active light method of fringe projection profilometry is adopted to capture the 3D information of the weldment. The weld seam extraction network based on binocular spatial information mining (BSMNet) is designed to analyze the corresponding passive light data and locate the weld seam position. Besides, a data annotation method based on binocular consistency correction is proposed to achieve more accurate data annotation for the BSMNet training. The experimental results show the max error of the detection is about 0.081mm, and the mean error is about 0.021mm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

References

  • Blomley, R., Weinmann, M., Leitloff, J., & Jutzi, B. (2014). Shape distribution features for point cloud analysis-a geometric histogram approach on multiple scales. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2(3), 9. doi: 10.5194/isprsannals-II-3-9-2014.

    Article  Google Scholar 

  • Cao, K., Wei, C., Gaidon, A., Arechiga, N., & Ma, T. (2019). Learning imbalanced datasets with label-distribution-aware margin loss. Advances in neural information processing systems, 32.

  • Cao, X. J., Jahazi, M., Immarigeon, J. P., & Wallace, W. (2006). A review of laser welding techniques for magnesium alloys. Journal of Materials Processing Technology, 171(2), 188–204. https://doi.org/10.1016/j.jmatprotec.2005.06.068.

  • Chan, S., Huang, C., Bai, C., Ding, W., & Chen, S. (2021). Res2-UNeXt: A novel deep learning framework for few-shot cell image segmentation. Multimedia Tools and Applications,https://doi.org/10.1007/s11042-021-10536-5

    Article  Google Scholar 

  • Cui, Y., Song, Y., Sun, C., Howard, A., & Belongie, S. (2018). Large scale fine-grained categorization and domain-specific transfer learning. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 4109-4118). https://doi.org/10.1109/CVPR.2018.00432

  • Cui, Y., Jia, M., Lin, T. Y., Song, Y., & Belongie, S. (2019). Class-balanced loss based on effective number of samples. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 9268-9277). https://doi.org/10.1109/CVPR.2019.00949

  • Fan, J., Deng, S., Jing, F., Zhou, C., Yang, L., Long, T., & Tan, M. (2019). An initial point alignment and seam-tracking system for narrow weld. IEEE Transactions on Industrial Informatics, 16(2), 877–886. https://doi.org/10.1109/TII.2019.2919658.

  • Guinard, S., & Landrieu, L. (2017). Weakly supervised segmentation-aided classification of urban scenes from 3D LiDAR point clouds. In ISPRS Workshop 2017. https://doi.org/10.5194/isprs-archives-XLII-1-W1-151-2017

  • Gorthi, S. S., & Rastogi, P. (2010). Fringe projection techniques: whither we are?. Optics and lasers in engineering, 48(ARTICLE), 133-140. https://doi.org/10.1016/j.optlaseng.2009.09.001

  • Kang, B., Xie, S., Rohrbach, M., Yan, Z., Gordo, A., Feng, J., & Kalantidis, Y. (2019). Decoupling Representation and Classifier for Long-Tailed Recognition. In International Conference on Learning Representations.

  • Lange, C., & Polthier, K. (2005). Anisotropic smoothing of point sets. Computer Aided Geometric Design, 22(7), 680–692. https://doi.org/10.1016/j.cagd.2005.06.010.

  • Lei, T., Rong, Y., Wang, H., Huang, Y., & Li, M. (2020). A review of vision-aided robotic welding. Computers in Industry, 123, 103326. https://doi.org/10.1016/j.compind.2020.103326.

  • Liu, X., Peng, X., Chen, H., He, D., & Gao, B. Z. (2012). Strategy for automatic and complete three-dimensional optical digitization. Optics letters, 37(15), 3126–3128. https://doi.org/10.1364/OL.37.003126.

  • Li, M., Xiao, Z., & Cheng, J. (2015). Segmentation of Retinal Blood Vessels Based on Regional Connectivity Analysis. Journal of Changchun University of Science and Technology (Natural Science Edition).

  • Mohr, R., Quan, L., & Veillon, F. (1995). Relative 3D reconstruction using multiple uncalibrated images. The International Journal of Robotics Research, 14(6), 619–632. https://doi.org/10.1109/CVPR.1993.341077.

  • Ou, Z., & Sun, Z. (2016). An image processing algorithm based on regional connectivity filtering of sheet weld track. Welding & Joining. https://doi.org/10.3969/j.issn.1001-1382.2016.12.013.

    Article  Google Scholar 

  • Romera, E., Alvarez, J. M., Bergasa, L. M., & Arroyo, R. (2017). Erfnet: Efficient residual factorized convnet for real-time semantic segmentation. IEEE Transactions on Intelligent Transportation Systems, 19(1), 263–272. https://doi.org/10.1109/TITS.2017.2750080.

  • Shao, W. J., Huang, Y., & Zhang, Y. (2018). A novel weld seam detection method for space weld seam of narrow butt joint in laser welding. Optics & Laser Technology, 99, 39–51. https://doi.org/10.1016/j.optlastec.2017.09.037.

  • Shashua, A., & Navab, N. (1994). Relative affine structure: Theory and application to 3D reconstruction from perspective views. In CVPR (Vol. 94, pp. 483-489).

  • Shrivastava, A., Gupta, A., & Girshick, R. (2016). Training region-based object detectors with online hard example mining. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 761-769). https://doi.org/10.1109/CVPR.2016.89

  • Weinmann, M., Urban, S., Hinz, S., Jutzi, B., & Mallet, C. (2015). Distinctive 2D and 3D features for automated large-scale scene analysis in urban areas. Computers & Graphics, 49, 47–57. https://doi.org/10.1016/j.cag.2015.01.006.

  • Xu, Y., Yu, H., Zhong, J., Tao, L., & Chen, S. (2012). Real-time seam tracking control technology during welding robot gtaw process based on passive vision sensor. Journal of Materials Processing Technology, 212(8), 1654–1662. https://doi.org/10.1016/j.jmatprotec.2012.03.007.

  • Xue, B., Chang, B., Peng, G., Gao, Y., Tian, Z., Du, D., & Wang, G. (2019). A vision based detection method for narrow butt joints and a robotic seam tracking system. Sensors, 19(5), 1144. https://doi.org/10.3390/s19051144.

  • Yang, L., Liu, Y., Peng, J., & Liang, Z. (2020). A novel system for off-line 3D seam extraction and path planning based on point cloud segmentation for arc welding robot. Robotics and Computer-Integrated Manufacturing, 64, 101929. https://doi.org/10.1016/j.rcim.2019.101929.

  • Yin, W., Feng, S., Tao, T., Huang, L., Trusiak, M., Chen, Q., & Zuo, C. (2019). High-speed 3D shape measurement using the optimized composite fringe patterns and stereo-assisted structured light system. Optics express, 27(3), 2411–2431. https://doi.org/10.1364/OE.27.002411

    Article  Google Scholar 

  • Yu, H., Chen, X., Zhang, Z., Zhang, Y., Zheng, D., Han, J.. (2019). Dynamic 3-d measurement based on fringe-to-fringe transformation using deep learning. Optics Express, 28: 9405-9418.

    Article  Google Scholar 

  • Zeng, J., Chang, B., Du, D., Peng, G., Chang, S., Hong, Y., & Shan, J. (2017). A vision-aided 3D path teaching method before narrow butt joint welding. Sensors, 17(5), 1099. https://doi.org/10.3390/s17051099.

  • Zhang, Y., You, D., Gao, X., Wang, C., Li, Y., & Gao, P. P. (2020). Real-time monitoring of high-power disk laser welding statuses based on deep learning framework. Journal of Intelligent Manufacturing, 31(4), 799–814. https://doi.org/10.1007/s10845-019-01477-w.

  • Zhang, B., Shi, Y., & Gu, S. (2019). Narrow-seam identification and deviation detection in keyhole deep-penetration TIG welding. The International Journal of Advanced Manufacturing Technology, 101(5), 2051–2064. https://doi.org/10.1007/s00170-018-3089-0.

  • Zhang, L., Xu, Y., Du, S., Zhao, W., Hou, Z., & Chen, S. (2018). Point cloud based three-dimensional reconstruction and identification of initial welding position. In Transactions on Intelligent Welding Manufacturing (pp. 61-77). Springer, Singapore. https://doi.org/10.1007/978-981-10-8330-3_4

  • Zhang, Z. (2000). A flexible new technique for camera calibration. IEEE Transactions on pattern analysis and machine intelligence, 22(11), 1330–1334. https://doi.org/10.1109/34.888718.

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (Grant Nos. 61727802, 61901220) and the China Postdoctoral Science Foundation(2021M691591).

Author information

Authors and Affiliations

  1. Jiangsu Key Laboratory of Spectral Imaging and Intelligent Sense, Nanjing University of Science and Technology, Nanjing, 210094, China

    Xingguo Wang, Tianyun Chen, Yiming Wang, Dongliang Zheng, Xiaoyu Chen & Zhuang Zhao

Authors
  1. Xingguo Wang
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Tianyun Chen
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Yiming Wang
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Dongliang Zheng
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Xiaoyu Chen
    View author publications

    You can also search for this author inPubMed Google Scholar

  6. Zhuang Zhao
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding authors

Correspondence to Xiaoyu Chen or Zhuang Zhao.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, X., Chen, T., Wang, Y. et al. The 3D narrow butt weld seam detection system based on the binocular consistency correction. J Intell Manuf 34, 2321–2332 (2023). https://doi.org/10.1007/s10845-022-01927-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10845-022-01927-y

Keywords