Abstract
Online monitoring and closed-loop control of laser welding offer great possibilities for achieving better weld quality. Earlier work on visual laser welding monitoring has mainly focused on aluminum and fairly thin steel used, for example, in car production. We extend this work by focusing on the automated analysis of the phenomena present in the laser welding of thick steel, where all of the phenomena related to the weld quality are still not well understood or controlled. This paper presents the implementation, test results and analysis for weld monitoring methods implemented on a compact smart camera system. The applied embedded sensor–processor platform allows for high-speed implementation of image capture and dynamic range compression, real-time seam tracking and spatter feature extraction. The paper describes experimental results from implemented real-time algorithms for seam tracking and spatter extraction and additional off-line analysis of methods for spatter tracking and seam widening detection, which are also feasible for future online hardware implementation. The results suggest that it is possible to integrate a compact laser welding analysis system, which achieves analysis rates that are sufficient for real-time process control.
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11554-016-0605-z/MediaObjects/11554_2016_605_Fig1_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11554-016-0605-z/MediaObjects/11554_2016_605_Fig2_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11554-016-0605-z/MediaObjects/11554_2016_605_Fig3_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11554-016-0605-z/MediaObjects/11554_2016_605_Fig4_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11554-016-0605-z/MediaObjects/11554_2016_605_Fig5_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11554-016-0605-z/MediaObjects/11554_2016_605_Fig6_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11554-016-0605-z/MediaObjects/11554_2016_605_Fig7_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11554-016-0605-z/MediaObjects/11554_2016_605_Fig8_HTML.jpg)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11554-016-0605-z/MediaObjects/11554_2016_605_Fig9_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11554-016-0605-z/MediaObjects/11554_2016_605_Fig10_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11554-016-0605-z/MediaObjects/11554_2016_605_Fig11_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11554-016-0605-z/MediaObjects/11554_2016_605_Fig12_HTML.gif)
Similar content being viewed by others
References
Vargas-Sierra, S., Linan-Cembrano, G., Rodriguez-Vazquez, A.: A 151 dB high dynamic range CMOS image sensor chip architecture with tone mapping compression embedded in-pixel. IEEE Sens. J. 15(1), 180–195 (2015)
Fennander, H., Kyrki, V., Fellman, A., et al.: Visual measurement and tracking in laser hybrid welding. Mach. Vision Appl. 20(2), 103–118 (2009)
Steele, J., Mnich, C., Debrunner, C., et al.: Development of closed-loop control of robotic welding process. Ind Robot Int J 32(4), 350–355 (2005)
Li, Y., Li, Y.F., Wang, Q.L., et al.: Measurement and defect detection of the weld bead based on online vision inspection. IEEE Trans. Instrum. Meas. 59(7), 1841–1849 (2010)
Gao, X.D., Na, S.J.: Detection of weld position and seam tracking based on Kalman filtering of weld pool images. J. Manuf. Syst. 24(1), 1–12 (2005)
Gao, X., You, D., Katayama, S.: Infrared image recognition for seam tracking monitoring during fiber laser welding. Mechatronics 22(4), 370–380 (2012)
Lee, S.K., Na, S.J.: A study on automatic seam tracking in pulsed laser edge welding by using a vision sensor without an auxiliary light source. J. Manuf. Syst. 21(4), 302–315 (2002)
Gao, X., Zhong, X., You, D., et al.: Kalman filtering compensated by radial basis function neural network for seam tracking of laser welding. IEEE Trans. Control Syst. Technol. 21(5), 1916–1923 (2013)
Nicolosi, L., Tetzlaff, R., Abt, F., et. al.: Novel algorithm for the real time multi-feature detection in laser beam welding. In: IEEE International Symposium on Circuits and Systems (ISCAS), pp. 181–184, (2012)
Schweier, M., Heins, J.F., Haubold, M.W., et al.: Spatter formation in laser welding with beam oscillation. Phys. Proc. 41, 20–30 (2013)
Nicolosi, L., Tetzlaff, R., Abt, F., et. al.: New CNN based algorithms for the full penetration hole extraction in laser welding processes: Experimental results. In: International Joint Conference on Neural Networks, pp. 2256–2263, (2009)
Blug, A., Carl, D., Höfler, H., et al.: Closed-loop control of laser power using the full penetration hole image feature in aluminum welding processes. Phys. Proc. 12, 720–729 (2011)
Nicolosi, L., Tetzlaff, R., Abt, F., et. al.: Cellular neural network (CNN) based control algorithms for omnidirectional laser welding processes: Experimental results. In: 12th International Workshop on Cellular Nanoscale Networks and Their Applications (CNNA), (2010)
Lahdenoja, O., Säntti, T., Poikonen, J., et. al.: Characterizing spatters in laser welding of thick steel using motion flow analysis. In: Scandinavian Conference on Image analysis, pp. 675–686, (2013)
Lahdenoja, O., Säntti, T., Laiho M., et. al.: Spatter tracking in laser-and manual arc welding with sensor-level pre-processing. In: International Conference on Computer Graphics, Visualization and Computer Vision (WSCG), (2014)
Lahdenoja, O., Säntti, T., Laiho, M., et. al.: Seam tracking with adaptive image capture for fine-tuning of a high power laser welding process. In: 7th International Conference on Machine Vision (ICMV) International Society for Optics and Photonics (SPIE), (2014)
Säntti, T., Poikonen, J.K., Lahdenoja, O., et. al.: Online seam tracking for laser welding with a vision chip and FPGA enabled camera system. In: IEEE International Symposium on Circuits and Systems (ISCAS), pp. 1985–1988, (2015)
Ion, J.C.: Laser Processing of Engineering Materials: Principles, procedure and industrial applications. Elsevier Butterworth-Heinemann, Oxford (2005)
Steen, W.M.: Laser Material Processing. Springer, London (2003)
Aiello, L.L., Hayne C., Mees, J.A.: Precision joint tracking laser welding system, United States Patent, Patent No. 5,001,324. Wilmingtonh, NC, United States, (1991)
Reitemeyer, D., Schultz, V., Syassen, F., et al.: Laser welding of large scale stainless steel aircraft structures. Phys. Proc. Proc. Lasers Manuf. Conf. 41, 106–111 (2013)
Zain-ul-Abdein, M., Nelias, D., Jullien, J.-F., et al.: Prediction of laser beam welding-induced distortions and residual stresses by numerical simulation for aeronautic application. J. Mater. Process. Technol. 209, 2907–2917 (2009)
Carmignani, C., Mares, R., Toselli, G.: Transient finite element analysis of deep penetration laser welding process in a singlepass butt-welded thick steel plate. Comput. Methods Appl. Mech. Eng. 179, 197–214 (1999)
Reutzel, E.W., Sullivan, M.J., Mikesic, D.A.: Joining pipe with the hybrid laser-GMAW process: weld test results and cost analysis. Weld. J. 85(6), 66–71 (2006)
Poikonen, J., Laiho, M., Paasio, A.: MIPA4k: mixed-mode cellular processor array. In: Focal-Plane Sensor-Processor Chips (pp. 45–71), Springer New York, (2011)
Ojala, T., Pietikäinen, M., Mäenpää, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. Pattern Anal. Mach. Intell. 24(7), 971–987 (2002)
Laiho, M., Poikonen, J., Paasio, A.: Object Segmentation and Tracking with Asynchronous Grayscale and Binary Wave Operations on the MIPA4k. In: European Conference on Circuit Theory and Design, Antalya, (2009)
Carey, S.J., Lopichm, A., Barr, D., et. al.: A 100,000 fps vision sensor with embedded 535GOPS/W 256 256 SIMD processor array. In: 2013 Symposium on VLSI Circuits (VLSIC), pp. C182–C183, (2013)
Rodriguez-Vazquez, A., Dominguez-Castro, R., Jimenez-Garrido, F., et al.: The Eye-RIS CMOS Vision System Analog Circuit Design. Springer, Netherlands (2008)
Rodríguez-Vázquez, A., Liñán-Cembrano, G., Carranza, L., et. al.: ACE16k: the third generation of mixed-signal SIMD-CNN ACE chips toward VSoCs. IEEE Trans. Circuits Syst. I Regul. Pap. 51(5), 851–863 (2004)
Poikonen, J., Laiho, M., Paasio, A.: Anisotropic filtering with a resistive fuse network on the MIPA4k processor Array. In: Proceedings of the 12th International Workshop on Cellular Nanoscale Networks and Their Applications (CNNA), 2010, Berkeley, USA, p. 4, 3–5 Feb 2010
Fernández-Berni, J., Carmona-Galán, R., Rodríguez-Vázquez, Á.: Low-Power Smart Imagers for Vision-Enabled Sensor Networks. ISBN 978-1-4614-2391-1, Springer (2012)
Laiho, M., Poikonen, J., Virtanen, K., et. al.: Self-adapting Compressive Image Sensing Scheme. In: IEEE International Workshop on Cellular Neural Networks and Their Applications, pp. 125–128, (2008)
Matas, J., Chum, O., Urban, M., et al.: Robust wide-baseline stereo from maximally stable extremal regions. Image Vis. Comput. 22(10), 761–767 (2004)
Vincent, L.: Morphological grayscale reconstruction in image analysis: applications and efficient algorithms. IEEE Trans. Image Process. 2(2), 176–201 (1993)
Säntti, T., Lahdenoja, O., Paasio, A., et. al.: Line Detection on FPGA with parallel sensor-level segmentation. In: 14th International Workshop on Cellular Nanoscale Networks and their Applications, CNNA (2014)
Acknowledgments
The authors would like to thank Machine Technology Center, Turku, Finland, for test support. The research was funded by Academy of Finland project PAMOWE (254430 and 254169). Jonne K. Poikonen, Mika Laiho and Ari Paasio are the founders and co-owners of Kovilta Oy (Salo, Finland), who designs and manufactures the KOVA1 pixel-processor ASIC and embedded camera system.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Lahdenoja, O., Säntti, T., Poikonen, J.K. et al. Embedded processing methods for online visual analysis of laser welding. J Real-Time Image Proc 16, 1099–1116 (2019). https://doi.org/10.1007/s11554-016-0605-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11554-016-0605-z