Abstract
Laser based directed energy deposition (L-DED) is a promising type of additive manufacturing technology. The non-destructive testing technology for the quality monitoring of L-DED processed parts is becoming more and more demanding in terms of accuracy, real-time, and ease of operation. This paper introduces a new image recognition system based on a deep convolutional neural network, which uses multiple lightweight architectures to reduce detection time. In order to eliminate the interference better, it improves the penalty function, which effectively improves the accuracy. Judging from the detection results of the data set, the accuracy of the model training reaches 94.71%, which achieves a very good image segmentation effect and solves the technical problem of in-situ monitoring of the L-DED process. This system realizes the positioning of the spatters for the first time, and at the same time, the number of spatters and area of molten pool are correlated to the laser scanning speed and the laser power.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bartlett, J. L., Heim, F. M., Murty, Y. V., & Li, X. (2018). In situ defect detection in selective laser melting via full-field infrared thermography. Additive Manufacturing, 24, 595–605.
Bouvrie, J. (2006). Notes on convolutional neural networks. In Practice, pp. 47–60.
Cha, Y. J., Choi, M., & Büyüköztürk, O. (2017). Deep learning-based crack damage detection using convolutional neural networks. Computer-Aided Civil and Infrastructure Engineering, 32(5), 361–378.
Eyers, D. R., & Potter, A. T. (2017). Industrial additive manufacturing: A manufacturing systems perspective. Computers in Industry, 92(93), 208–218.
Fang, Q., Tan, Z., Li, H., Liu, S., Song, C., Zhou, X., Yang, Y., & Shen, S. (2020). In-situ capture of melt pool signature in selective laser melting using U-Net based convolutional neural network. Journal of Manufacturing Processes (submitted).
Fox, J. C., Moylan, S. P., & Lane, B. M. (2016). Effect of process parameters on the surface roughness of overhanging structures in laser powder bed fusion additive manufacturing. In 3rd CIRP conference on surface integrity (CIRP CSI), pp. 131–134.
Gobert, C., Reutzel, E. W., Petrich, J., Nassar, A. R., & Phoha, S. (2018). Application of supervised machine learning for defect detection during metallic powder bed fusion additive manufacturing using high resolution imaging. Additive Manufacturing, 21, 517–528.
Grasso, M., Demir, A. G., & Prevital, B. (2018). In situ monitoring of selective laser melting of zinc powder via infrared imaging of the process plume. The Journal Robotics and Computer-Integrated Manufacturing, 49, 229–239.
Jacobsmühlen, J., Kleszczynski, S., Witt, G., & Merhof, D. (2015). Detection of elevated regions in surface images from laser beam melting processes. In IECON 2015—41st annual conference of the IEEE industrial electronics society, pp. 1270–1275.
Kapur, J. N., Sahoo, P. K., & Wong, A. K. C. (1985). A new method for gray-level picture thresholding using the entropy of the histogram. Computer Vision Graphics and Image Processing, 29(3), 273–285.
Kwon, O., Kim, H. G., Ham, M. J., Kim, W., Kim, G. H., Cho, J. H., Kim, N., & Kim, K. (2020). A deep neural network for classification of melt-pool images in metal additive manufacturing. Journal of Intelligent Manufacturing, 31, 375–386.
Long, J., Shelhamer, E., & Darrell, T. (2015). Fully convolutional networks for semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(4), 640–651.
Ngoveni, A. S., Popoola, A. P. I., Arthur, N. K. K., & Pityana, S. L. (2019). Residual stress modelling and experimental analyses of Ti6Al4V ELI additive manufactured by laser engineered net shaping. Procedia Manufacturing., 35, 1001–1006.
Pinkerton, A. J. (2016). Lasers in additive manufacturing. Optics & Laser Technology, 78, 25–32.
Sames, W. J., List, F. A., Pannala, S., Dehoff, R. R., & Babu, S. S. (2016). The metallurgy and processing science of metal additive manufacturing. International Materials Reviews, 61(5), 315–360.
Scime, L., & Beuth, J. (2019). Using machine learning to identify in-situ melt pool signatures indicative of flaw formation in a laser powder bed fusion additive manufacturing process. Additive Manufacturing, 25, 151–165.
Shu, Z., Chen, Z., Wang, L., Wei, X., Li, W., & Zheng, Z. (2020). Microstructure evolution and formation mechanism of a crack-free nickel based superalloy fabricated by laser engineered net shaping. Optics & Laser Technology, 128, 106222.
Soukup, D., & Huber-Mörk, R. (2014). Convolutional neural networks for steel surface defect detection from photometric stereo images. In Advances in visual computing: 11th international symposium, ISVAdv visual computing, ISVC 2014, Lecture notes in computer science, Vol. 8887, pp. 668–77.
Tan, Z., Fang, Q., Li, H., Liu, S., Zhu, W., & Yang, D. (2020). Neural network based image segmentation for spatter extraction during selective laser melting processing. Optics & Laser Technology, 130, 106347.
Tola, E., Lepetit, V., & Fua, P. (2008). A fast local descriptor for dense matching. In 26th IEEE conference on computer vision and pattern recognition (CVPR).
Yang, D., Li, H., Liu, S., Song, C., Yang, Y., Shen, S., Lu, J., Liu, Z., & Zhu, Y. (2020b). In situ capture of spatter signature of SLM process using maximum entropy double threshold image processing method based on genetic algorithm. Optical Laser Technology, 131, 106371.
Yang, Q., Yuan, Z., Zhi, X., Yan, Z., Yang, Y., & Tian, H. (2020a). Real-time width control of molten pool in laser engineered net shaping based on dual-color image. Optics & Laser Technology, 123, 105925.
Ye, D., Fuh, J. Y. H., Zhang, Y., Hong, G. S., & Zhu, K. (2018). In situ monitoring of selective laser melting using plume and spatter signatures by deep belief networks. ISA Transactions, 81, 96–104.
Yin, J., Yang, L., Yang, X., Zhu, H., Wang, D., Ke, L., & Zeng, X. (2019). High-power laser-matter interaction during laser powder bed fusion. Additive Manufacturing, 29, 100778.
Zhang, M., Chen, G., Zhou, Y., Li, S., & Deng, H. (2013). Observation of spatter formation mechanisms in high-power fiber laser welding of thick plate. Applied Surface Science, 280, 868–875.
Zhang, Y., Fuh, J. Y. H., Ye, D., & Hong, G. S. (2019). In-situ monitoring of laser-based PBF via off-axis vision and image processing approaches. Additive Manufacturing, 25, 263–274.
Zhang, Y., Hong, G. S., Ye, D., Zhu, K., & Fuh, J. Y. H. (2018). Extraction and evaluation of melt pool, plume and spatter information for powder-bed fusion AM process monitoring. Materials and Design, 156, 458–469.
Acknowledgements
This work was supported by the Key-Area Research and Development Program of Guangdong Province, China (2018B090905001); the Key Research and Development Program of Sichuan Province, China (2020YFSY0054); the Key Research and Development Program of Hubei province, China (2020BAB045).
Author information
Authors and Affiliations
Corresponding authors
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Mi, J., Zhang, Y., Li, H. et al. In-situ monitoring laser based directed energy deposition process with deep convolutional neural network. J Intell Manuf 34, 683–693 (2023). https://doi.org/10.1007/s10845-021-01820-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10845-021-01820-0