Skip to main content
SpringerLink
Log in

A fast method for monitoring molten pool in infrared image streams using gravitational superpixels.

  • Published:
Journal of Intelligent Manufacturing Aims and scope Submit manuscript

Abstract

Additive manufacturing (AM) is one of the most trending areas in production that allows creating three-dimensional objects according to a predetermined design. AM finds its application in all kinds of niches, from medicine to the aerospace industry, although there are still several technological barriers that must be addressed. For example, monitoring techniques that guarantee decision-making to guarantee quality and repeatability of processes. An imaging-based methodology is presented to monitor and extract thermal and geometric characteristics of molten pool in real-time. A superpixel-based approach is proposed to reduce the dimensionality of the infrared images and facilitate the segmentation and tracking tasks. This algorithm is called gravitational superpixels. Using the color and temperature features, our algorithm groups the pixels. These superpixels have better adherence to the structures that form the images. Facilitating the segmentation tasks. Our algorithm is compared against superpixel-based and saliency-based already reported works. To validate the performance, infrared-image streams (LMD process) and standard datasets are using. The proposed algorithm has a molten pool segmentation uncertainty of \(0.1\; mm\). Reported results show that the performance of our proposal is applicable for tasks that require good precision when segmenting and fast runtime. It is important to highlight the relevance of this work for additive metal manufacturing processes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  • Achanta, Radhakrishna, Shaji, Appu, Smith, Kevin, Lucchi, Aurelien, Fua, Pascal & Süsstrunk Sabine (2010) SLIC Superpixels. EPFL

  • America Makes and ANSI Additive Manufacturing Standardization Collaborative. Standardization roadmap for additive manufacturing. (2017).

  • Bartlett, Jamison L., Heim, Frederick M., Murty, Yellapu V., & Li, Xiaodong. (2018). In situ defect detection in selective laser melting via full-field infrared thermography. Additive Manufacturing, 24, 595–605.

    Article  Google Scholar 

  • Bikas, Harry, Stavropoulos, Panagiotis, & Chryssolouris, George. (2016). Additive manufacturing methods and modelling approaches: A critical review. The International Journal of Advanced Manufacturing Technology, 83(1–4), 389–405.

    Article  Google Scholar 

  • Bruhn, Andrés, Weickert, Joachim, & Schnörr, Christoph. (2005). Lucas/kanade meets horn/schunck: Combining local and global optic flow methods. International Journal of Computer Vision, 61(3), 211–231.

    Article  Google Scholar 

  • Campbell, I., Diegel, Olaf, Kowen, J. & Wohlers, T. (2018). Wohlers report 2018: 3D printing and additive manufacturing state of the industry: annual worldwide progress report. Wohlers Associates.

  • Chabot, A., Laroche, N., Carcreff, E., Rauch, M., & Hascoët, J.-Y. (2020). Towards defect monitoring for metallic additive manufacturing components using phased array ultrasonic testing. Journal of Intelligent Manufacturing, 31, 1191–1201.

    Article  Google Scholar 

  • Chandrasekhar, N., Vasudevan, M., Bhaduri, A. K., & Jayakumar, T. (2015). Intelligent modeling for estimating weld bead width and depth of penetration from infra-red thermal images of the weld pool. Journal of Intelligent Manufacturing, 26(1), 59–71.

    Article  Google Scholar 

  • Cheng, Baokai, Lei, Jincheng, & Xiao, Hai. (2019). A photoacoustic imaging method for in-situ monitoring of laser assisted ceramic additive manufacturing. Optics & Laser Technology, 115, 459–464.

    Article  Google Scholar 

  • Cheng, Ming-Ming, Mitra, Niloy J., Huang, Xiaolei, Torr, Philip H. S., & Hu, Shi-Min. (2015). Global contrast based salient region detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(3), 569–582.

    Article  Google Scholar 

  • Clijsters, Stijn, Craeghs, Tom, Buls, Sam, Kempen, Karolien, & Kruth, J.-P. (2014). In situ quality control of the selective laser melting process using a high-speed real-time melt pool monitoring system. The International Journal of Advanced Manufacturing Technology, 75(5–8), 1089–1101.

    Article  Google Scholar 

  • Craeghs, Tom, Clijsters, Stijn, Kruth, Jean-Pierre, Bechmann, Florian, & Ebert, Marie-Christin. (2012). Detection of process failures in layerwise laser melting with optical process monitoring. Physics Procedia, 39, 753–759.

    Article  Google Scholar 

  • den Bergh, Van, Michael, Boix, Xavier, G. Roig, de Capitani, B., & Van Gool, L. (2012). Seeds: Superpixels extracted via energy-driven sampling. In Andrew Fitzgibbon, Svetlana Lazebnik, Pietro Perona, Yoichi Sato, & Cordelia Schmid (Eds.), European conference on computer vision (pp. 13–26). Berlin: Springer.

    Google Scholar 

  • Erdem, Erkut, & Erdem, Aykut. (2013). Visual saliency estimation by nonlinearly integrating features using region covariances. Journal of Vision, 13(4), 11.

    Article  Google Scholar 

  • Everton, Sarah K., Hirsch, Matthias, Stravroulakis, Petros, Leach, Richard K., & Clare, Adam T. (2016). Review of in-situ process monitoring and in-situ metrology for metal additive manufacturing. Materials & Design, 95, 431–445.

    Article  Google Scholar 

  • Gao, Xiangdong, & Zhang, Yanxi. (2015). Monitoring of welding status by molten pool morphology during high-power disk laser welding. Optik-International Journal for Light and Electron Optics, 126(19), 1797–1802.

    Article  Google Scholar 

  • Gonzalez-Val, Carlos, Pallas, Adrian, Panadeiro, Veronica, & Rodriguez, Alvaro. (2020). A convolutional approach to quality monitoring for laser manufacturing. Journal of Intelligent Manufacturing, 31(3), 789–795.

    Article  Google Scholar 

  • Grasso, Marco, & Colosimo, Bianca Maria. (2017). Process defects and in situ monitoring methods in metal powder bed fusion: A review. Measurement Science and Technology, 28(4), 044005.

    Article  Google Scholar 

  • Hanbury, Allan. (2008). How do superpixels affect image segmentation? In José Ruiz-Shulcloper & Walter G. Kropatsch (Eds.), Iberoamerican congress on pattern recognition (pp. 178–186). Berlin: Springer.

    Google Scholar 

  • Helwig, Nathaniel E., & Ma, Ping. (2015). Fast and stable multiple smoothing parameter selection in smoothing spline analysis of variance models with large samples. Journal of Computational and Graphical Statistics, 24(3), 715–732.

    Article  Google Scholar 

  • Hou, Xiaodi, Harel, Jonathan, & Koch, Christof. (2011). Image signature: Highlighting sparse salient regions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(1), 194–201.

    Google Scholar 

  • Jafari-Marandi, Ruholla, Khanzadeh, Mojtaba, Tian, Wenmeng, Smith, Brian, & Bian, Linkan. (2019). From in-situ monitoring toward high-throughput process control: Cost-driven decision-making framework for laser-based additive manufacturing. Journal of Manufacturing Systems, 51, 29–41.

    Article  Google Scholar 

  • Jurrens, K. (2013). and Energetics Incorporated. Measurement science roadmap for metal-based additive manufacturing. Technical report, National Institute of Standards and Technology.

  • Khanzadeh, Mojtaba, Chowdhury, Sudipta, Marufuzzaman, Mohammad, Tschopp, Mark A., & Bian, Linkan. (2018). Porosity prediction: Supervised-learning of thermal history for direct laser deposition. Journal of Manufacturing Systems, 47, 69–82.

    Article  Google Scholar 

  • Khanzadeh, Mojtaba, Tian, Wenmeng, Yadollahi, Aref, Doude, Haley R., Tschopp, Mark A., & Bian, Linkan. (2018). Dual process monitoring of metal-based additive manufacturing using tensor decomposition of thermal image streams. Additive Manufacturing, 23, 443–456.

    Article  Google Scholar 

  • Kwon, Ohyung, Kim, Hyung Giun, Ham, Min Ji, Kim, Wonrae, Kim, Gun-Hee, Cho, Jae-Hyung, et al. (2020). A deep neural network for classification of melt-pool images in metal additive manufacturing. Journal of Intelligent Manufacturing, 31(2), 375–386.

    Article  Google Scholar 

  • Leach, R. K., Bourell, David, Carmignato, Simone, Donmez, Alkan, Senin, Nicola, & Dewulf, Wim. (2019). Geometrical metrology for metal additive manufacturing. CIRP Annals, 68(2), 677–700.

    Article  Google Scholar 

  • Levinshtein, Alex, Stere, Adrian, Kutulakos, Kiriakos N., Fleet, David J., Dickinson, Sven J., & Siddiqi, Kaleem. (2009). Turbopixels: Fast superpixels using geometric flows. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(12), 2290–2297.

    Article  Google Scholar 

  • Li, Yin, Hou, Xiaodi, Koch, Christof, Rehg, James M. & Yuille, Alan L. (2014). The secrets of salient object segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 280–287.

  • Li, Zhengqin & Chen, Jiansheng (2015). Superpixel segmentation using linear spectral clustering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1356–1363.

  • Lindeberg, Tony. (1993). Discrete derivative approximations with scale-space properties: A basis for low-level feature extraction. Journal of Mathematical Imaging and Vision, 3(4), 349–376.

    Article  Google Scholar 

  • Liu, Yong-Jin, Yu, Cheng-Chi, Yu, Min-Jing & He, Ying (2016) Manifold slic: A fast method to compute content-sensitive superpixels. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 651–659.

  • Luo, Masiyang, & Shin, Yung C. (2015). Vision-based weld pool boundary extraction and width measurement during keyhole fiber laser welding. Optics and Lasers in Engineering, 64, 59–70.

    Article  Google Scholar 

  • Meng, Wei, Yin, Xiaohui, Fang, Junfei, Guo, Lijie, Ma, Qunshuang, & Li, Zhuguo. (2019). Dynamic features of plasma plume and molten pool in laser lap welding based on image monitoring and processing techniques. Optics & Laser Technology, 109, 168–177.

    Article  Google Scholar 

  • Moralejo, S., Penaranda, X., Nieto, S., Barrios, A., Arrizubieta, I., Tabernero, I., et al. (2017). A feedforward controller for tuning laser cladding melt pool geometry in real time. The International Journal of Advanced Manufacturing Technology, 89(1–4), 821–831.

    Article  Google Scholar 

  • Papari, Giuseppe, & Petkov, Nicolai. (2011). Edge and line oriented contour detection: State of the art. Image and Vision Computing, 29(2–3), 79–103.

    Article  Google Scholar 

  • Riche, Nicolas, Mancas, Matei, Duvinage, Matthieu, Mibulumukini, Makiese, Gosselin, Bernard, & Dutoit, Thierry. (2013). Rare 2012: A multi-scale rarity-based saliency detection with its comparative statistical analysis. Signal Processing: Image Communication, 28(6), 642–658.

    Google Scholar 

  • Schauerte, Boris, & Stiefelhagen, Rainer. (2012). Quaternion-based spectral saliency detection for eye fixation prediction. In Andrew Fitzgibbon, Svetlana Lazebnik, Pietro Perona, Yoichi Sato, & Cordelia Schmid (Eds.), European conference on computer vision (pp. 116–129). Berlin: Springer.

    Google Scholar 

  • Shevchik, Sergey A., Masinelli, Giulio Giulio, Kenel, Christoph, Leinenbach, Christian, & Wasmer, Kilian. (2019). Deep learning for in situ and real-time quality monitoring in additive manufacturing using acoustic emission. IEEE Transactions on Industrial Informatics, 15(9), 5194–5203.

    Article  Google Scholar 

  • Song, Lijun, Wang, Fanghua, Li, Simeng, & Han, Xu. (2017). Phase congruency melt pool edge extraction for laser additive manufacturing. Journal of Materials Processing Technology, 250, 261–269.

    Article  Google Scholar 

  • Tavakoli, Hamed Rezazadegan, Rahtu, Esa, & Heikkilä, Janne. (2011). Fast and efficient saliency detection using sparse sampling and kernel density estimation. In Anders Heyden & Fredrik Kahl (Eds.), Scandinavian conference on image analysis (pp. 666–675). Berlin: Springer.

    Chapter  Google Scholar 

  • Veksler, Olga, Boykov, Yuri, & Mehrani, Paria. (2010). Superpixels and supervoxels in an energy optimization framework. In N. Paragios, K. Daniilidis, & P. Maragos (Eds.), European conference on computer vision (pp. 211–224). Berlin: Springer.

    Google Scholar 

  • Wang, Jie, & Wang, Xiaoqiang. (2012). Vcells: Simple and efficient superpixels using edge-weighted centroidal voronoi tessellations. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(6), 1241–1247.

    Article  Google Scholar 

  • Wang, Murong, Liu, Xiabi, Gao, Yixuan, Ma, Xiao, & Soomro, Nouman Q. (2017). Superpixel segmentation: A benchmark. Signal Processing: Image Communication, 56, 28–39.

    Google Scholar 

  • Wang, Wenguan, Shen, Jianbing & Porikli, Fatih (2015). Saliency-aware geodesic video object segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3395–3402. IEEE Computer Society.

  • Wloka, Calden, Kunić, Toni, Kotseruba, Iuliia, Fahimi, Ramin, Frosst, Nicholas, Bruce, Neil D.B. & Tsotsos, John K. (2018). Smiler: Saliency model implementation library for experimental research. arXiv preprint arXiv:1812.08848.

  • Xu, Chenliang & Corso, Jason J. (2012). Evaluation of super-voxel methods for early video processing. In 2012 IEEE Conference on computer vision and pattern recognition, pages 1202–1209. IEEE.

  • Yan, Yijun, Ren, Jinchang, Sun, Genyun, Zhao, Huimin, Han, Junwei, Li, Xuelong, et al. (2018). Unsupervised image saliency detection with gestalt-laws guided optimization and visual attention based refinement. Pattern Recognition, 79, 65–78.

    Article  Google Scholar 

  • Yan, Zhaorui, Liu, Weiwei, Tang, Zijue, Liu, Xuyang, Zhang, Nan, Li, Mingzheng, et al. (2018). Review on thermal analysis in laser-based additive manufacturing. Optics & Laser Technology, 106, 427–441.

    Article  Google Scholar 

  • Zhang, Bin, Liu, Shunyu, & Shin, Yung C. (2019). In-process monitoring of porosity during laser additive manufacturing process. Additive Manufacturing, 28, 497–505.

    Article  Google Scholar 

  • Zhang, Yanxi, You, Deyong, Gao, Xiangdong, Wang, Congyi, Li, Yangjin, & Gao, Perry P. (2019). Real-time monitoring of high-power disk laser welding statuses based on deep learning framework. Journal of Intelligent Manufacturing, 31, 799–814.

    Article  Google Scholar 

Download references

Acknowledgements

Acknowledgments are due to program Cátedras-CONACYT (National Council for Science and Technology of Mexico) for the support provided by generating research opportunities through the project num. 730. Also, the author thanks the financial support provided by CONACYT through the FORDECYT Programs (projects 297265 and 296384). Additionally, thanks are due to the CONACYT Consortium in Additive Manufacturing (CONMAD) for the use of experimental facilities for this work.

Author information

Authors and Affiliations

  1. Center for Engineering and Industrial Development (CIDESI), Av. Playa Pie de la Cuesta No. 702. Desarrollo San Pablo., Querétaro, Qro., Mexico

    Angel-Iván García-Moreno

  2. Dirección de Cátedras-CONACYT, Consejo Nacional de Ciencia y Tecnología (CONACYT), México City, Mexico

    Angel-Iván García-Moreno

Authors
  1. Angel-Iván García-Moreno
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Angel-Iván García-Moreno.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

García-Moreno, AI. A fast method for monitoring molten pool in infrared image streams using gravitational superpixels.. J Intell Manuf 33, 1779–1794 (2022). https://doi.org/10.1007/s10845-021-01761-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10845-021-01761-8

Keywords

Search

Navigation