Skip to main content
Log in

Embedded processing methods for online visual analysis of laser welding

  • Original Research Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

Online monitoring and closed-loop control of laser welding offer great possibilities for achieving better weld quality. Earlier work on visual laser welding monitoring has mainly focused on aluminum and fairly thin steel used, for example, in car production. We extend this work by focusing on the automated analysis of the phenomena present in the laser welding of thick steel, where all of the phenomena related to the weld quality are still not well understood or controlled. This paper presents the implementation, test results and analysis for weld monitoring methods implemented on a compact smart camera system. The applied embedded sensor–processor platform allows for high-speed implementation of image capture and dynamic range compression, real-time seam tracking and spatter feature extraction. The paper describes experimental results from implemented real-time algorithms for seam tracking and spatter extraction and additional off-line analysis of methods for spatter tracking and seam widening detection, which are also feasible for future online hardware implementation. The results suggest that it is possible to integrate a compact laser welding analysis system, which achieves analysis rates that are sufficient for real-time process control.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Vargas-Sierra, S., Linan-Cembrano, G., Rodriguez-Vazquez, A.: A 151 dB high dynamic range CMOS image sensor chip architecture with tone mapping compression embedded in-pixel. IEEE Sens. J. 15(1), 180–195 (2015)

    Article  Google Scholar 

  2. Fennander, H., Kyrki, V., Fellman, A., et al.: Visual measurement and tracking in laser hybrid welding. Mach. Vision Appl. 20(2), 103–118 (2009)

    Article  Google Scholar 

  3. Steele, J., Mnich, C., Debrunner, C., et al.: Development of closed-loop control of robotic welding process. Ind Robot Int J 32(4), 350–355 (2005)

    Article  Google Scholar 

  4. Li, Y., Li, Y.F., Wang, Q.L., et al.: Measurement and defect detection of the weld bead based on online vision inspection. IEEE Trans. Instrum. Meas. 59(7), 1841–1849 (2010)

    Article  Google Scholar 

  5. Gao, X.D., Na, S.J.: Detection of weld position and seam tracking based on Kalman filtering of weld pool images. J. Manuf. Syst. 24(1), 1–12 (2005)

    Article  Google Scholar 

  6. Gao, X., You, D., Katayama, S.: Infrared image recognition for seam tracking monitoring during fiber laser welding. Mechatronics 22(4), 370–380 (2012)

    Article  Google Scholar 

  7. Lee, S.K., Na, S.J.: A study on automatic seam tracking in pulsed laser edge welding by using a vision sensor without an auxiliary light source. J. Manuf. Syst. 21(4), 302–315 (2002)

    Article  Google Scholar 

  8. Gao, X., Zhong, X., You, D., et al.: Kalman filtering compensated by radial basis function neural network for seam tracking of laser welding. IEEE Trans. Control Syst. Technol. 21(5), 1916–1923 (2013)

    Article  Google Scholar 

  9. Nicolosi, L., Tetzlaff, R., Abt, F., et. al.: Novel algorithm for the real time multi-feature detection in laser beam welding. In: IEEE International Symposium on Circuits and Systems (ISCAS), pp. 181–184, (2012)

  10. Schweier, M., Heins, J.F., Haubold, M.W., et al.: Spatter formation in laser welding with beam oscillation. Phys. Proc. 41, 20–30 (2013)

    Article  Google Scholar 

  11. Nicolosi, L., Tetzlaff, R., Abt, F., et. al.: New CNN based algorithms for the full penetration hole extraction in laser welding processes: Experimental results. In: International Joint Conference on Neural Networks, pp. 2256–2263, (2009)

  12. Blug, A., Carl, D., Höfler, H., et al.: Closed-loop control of laser power using the full penetration hole image feature in aluminum welding processes. Phys. Proc. 12, 720–729 (2011)

    Article  Google Scholar 

  13. Nicolosi, L., Tetzlaff, R., Abt, F., et. al.: Cellular neural network (CNN) based control algorithms for omnidirectional laser welding processes: Experimental results. In: 12th International Workshop on Cellular Nanoscale Networks and Their Applications (CNNA), (2010)

  14. Lahdenoja, O., Säntti, T., Poikonen, J., et. al.: Characterizing spatters in laser welding of thick steel using motion flow analysis. In: Scandinavian Conference on Image analysis, pp. 675–686, (2013)

  15. Lahdenoja, O., Säntti, T., Laiho M., et. al.: Spatter tracking in laser-and manual arc welding with sensor-level pre-processing. In: International Conference on Computer Graphics, Visualization and Computer Vision (WSCG), (2014)

  16. Lahdenoja, O., Säntti, T., Laiho, M., et. al.: Seam tracking with adaptive image capture for fine-tuning of a high power laser welding process. In: 7th International Conference on Machine Vision (ICMV) International Society for Optics and Photonics (SPIE), (2014)

  17. Säntti, T., Poikonen, J.K., Lahdenoja, O., et. al.: Online seam tracking for laser welding with a vision chip and FPGA enabled camera system. In: IEEE International Symposium on Circuits and Systems (ISCAS), pp. 1985–1988, (2015)

  18. Ion, J.C.: Laser Processing of Engineering Materials: Principles, procedure and industrial applications. Elsevier Butterworth-Heinemann, Oxford (2005)

    Google Scholar 

  19. Steen, W.M.: Laser Material Processing. Springer, London (2003)

    Book  Google Scholar 

  20. Aiello, L.L., Hayne C., Mees, J.A.: Precision joint tracking laser welding system, United States Patent, Patent No. 5,001,324. Wilmingtonh, NC, United States, (1991)

  21. Reitemeyer, D., Schultz, V., Syassen, F., et al.: Laser welding of large scale stainless steel aircraft structures. Phys. Proc. Proc. Lasers Manuf. Conf. 41, 106–111 (2013)

    Google Scholar 

  22. Zain-ul-Abdein, M., Nelias, D., Jullien, J.-F., et al.: Prediction of laser beam welding-induced distortions and residual stresses by numerical simulation for aeronautic application. J. Mater. Process. Technol. 209, 2907–2917 (2009)

    Article  Google Scholar 

  23. Carmignani, C., Mares, R., Toselli, G.: Transient finite element analysis of deep penetration laser welding process in a singlepass butt-welded thick steel plate. Comput. Methods Appl. Mech. Eng. 179, 197–214 (1999)

    Article  MATH  Google Scholar 

  24. Reutzel, E.W., Sullivan, M.J., Mikesic, D.A.: Joining pipe with the hybrid laser-GMAW process: weld test results and cost analysis. Weld. J. 85(6), 66–71 (2006)

    Google Scholar 

  25. http://www.kovilta.fi

  26. Poikonen, J., Laiho, M., Paasio, A.: MIPA4k: mixed-mode cellular processor array. In: Focal-Plane Sensor-Processor Chips (pp. 45–71), Springer New York, (2011)

  27. Ojala, T., Pietikäinen, M., Mäenpää, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. Pattern Anal. Mach. Intell. 24(7), 971–987 (2002)

    Article  MATH  Google Scholar 

  28. Laiho, M., Poikonen, J., Paasio, A.: Object Segmentation and Tracking with Asynchronous Grayscale and Binary Wave Operations on the MIPA4k. In: European Conference on Circuit Theory and Design, Antalya, (2009)

  29. Carey, S.J., Lopichm, A., Barr, D., et. al.: A 100,000 fps vision sensor with embedded 535GOPS/W 256 256 SIMD processor array. In: 2013 Symposium on VLSI Circuits (VLSIC), pp. C182–C183, (2013)

  30. Rodriguez-Vazquez, A., Dominguez-Castro, R., Jimenez-Garrido, F., et al.: The Eye-RIS CMOS Vision System Analog Circuit Design. Springer, Netherlands (2008)

    Google Scholar 

  31. Rodríguez-Vázquez, A., Liñán-Cembrano, G., Carranza, L., et. al.: ACE16k: the third generation of mixed-signal SIMD-CNN ACE chips toward VSoCs. IEEE Trans. Circuits Syst. I Regul. Pap. 51(5), 851–863 (2004)

    Article  Google Scholar 

  32. Poikonen, J., Laiho, M., Paasio, A.: Anisotropic filtering with a resistive fuse network on the MIPA4k processor Array. In: Proceedings of the 12th International Workshop on Cellular Nanoscale Networks and Their Applications (CNNA), 2010, Berkeley, USA, p. 4, 3–5 Feb 2010

  33. Fernández-Berni, J., Carmona-Galán, R., Rodríguez-Vázquez, Á.: Low-Power Smart Imagers for Vision-Enabled Sensor Networks. ISBN 978-1-4614-2391-1, Springer (2012)

  34. Laiho, M., Poikonen, J., Virtanen, K., et. al.: Self-adapting Compressive Image Sensing Scheme. In: IEEE International Workshop on Cellular Neural Networks and Their Applications, pp. 125–128, (2008)

  35. Matas, J., Chum, O., Urban, M., et al.: Robust wide-baseline stereo from maximally stable extremal regions. Image Vis. Comput. 22(10), 761–767 (2004)

    Article  Google Scholar 

  36. Vincent, L.: Morphological grayscale reconstruction in image analysis: applications and efficient algorithms. IEEE Trans. Image Process. 2(2), 176–201 (1993)

    Article  Google Scholar 

  37. Säntti, T., Lahdenoja, O., Paasio, A., et. al.: Line Detection on FPGA with parallel sensor-level segmentation. In: 14th International Workshop on Cellular Nanoscale Networks and their Applications, CNNA (2014)

Download references

Acknowledgments

The authors would like to thank Machine Technology Center, Turku, Finland, for test support. The research was funded by Academy of Finland project PAMOWE (254430 and 254169). Jonne K. Poikonen, Mika Laiho and Ari Paasio are the founders and co-owners of Kovilta Oy (Salo, Finland), who designs and manufactures the KOVA1 pixel-processor ASIC and embedded camera system.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Olli Lahdenoja.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lahdenoja, O., Säntti, T., Poikonen, J.K. et al. Embedded processing methods for online visual analysis of laser welding. J Real-Time Image Proc 16, 1099–1116 (2019). https://doi.org/10.1007/s11554-016-0605-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-016-0605-z

Keywords

Navigation