Skip to main content

Advertisement

SpringerLink
Log in

Toward online layer-wise surface morphology measurement in additive manufacturing using a deep learning-based approach

  • Published:
Journal of Intelligent Manufacturing Aims and scope Submit manuscript

A Correction to this article was published on 18 May 2022

This article has been updated

Abstract

Layer-wise surface morphology information plays a critical role in the quality monitoring and control of additive manufacturing (AM) processes. 3D scan technologies can provide effective means to obtain accurate surface morphological data. However, most of the existing 3D scan technologies are time consuming due to either contact mode or algorithm complexity, which are not capable of obtaining the surface morphology data in an online manner during the printing process. To implement online layer-wise surface morphological data acquisition in AM processes, one practical solution is to model the correlation between 2D images and 3D point cloud data. In practice, since this correlation is usually highly complex due to the high dimensionality and non-linearity, it is usually impractical to find an explicit mathematical transfer function to quantify this correlation effectively. To address this issue, a deep learning-based model is developed in this study, in which a powerful deep learning algorithm, namely, convolutional neural network (CNN), is incorporated. With the trained CNN model, the 3D surface data can be predicted directly without the relatively time consuming triangulation computation by the 3D scanner. Thus, the speed of surface data acquisition and morphology measurement can be improved. To validate the effectiveness and efficiency of the proposed methodology, both simulation and real-world AM case studies were performed. The results show that the prediction accuracy using the proposed method is promising. In terms of averaged relative prediction error, it can be mostly lower than 10% in the experiments. Therefore, the proposed method has a great potential for online layer-wise surface morphology measurement in AM.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

Change history

References

  • ACFOAMTSFO Terminology. (2012). Standard Terminology for Additive Manufacturing Technologies. ASTM International.

    Google Scholar 

  • Agarwala, M. K., et al. (1996). Structural quality of parts processed by fused deposition. Rapid Prototyping Journal, 2(4), 4–19.

    Article  Google Scholar 

  • Al-Ahmari, A., & Aalam, J. J. M. (2015). Optimizing parameters of freeform surface reconstruction using CMM. Measurement, 64, 17–28.

    Article  Google Scholar 

  • Arcam Q10 plus. Retrieved from: http://www.arcam.com/technology/products/arcam-q10/.

  • Benesty, J., et al. (2009). Pearson correlation coefficient. Noise Reduction in Speech Processing (pp. 1–4). Springer.

    Google Scholar 

  • Benner, W. (2016). Laser scanners: Technologies and applications; How they work, and how they can work for your product. Pangolin.

  • Besl, P. J., & McKay, N. D. (1992). Method for Registration of 3-D Shapes. In Sensor Fusion IV: Control Paradigms and Data Structures. New York: International Society for Optics and Photonics.

    Google Scholar 

  • Bryan, J., & Carter, D. (1979). Design of a new error-corrected co-ordinate measuring machine. Precision Engineering, 1(3), 125–128.

    Article  Google Scholar 

  • Choy, C.B., et al. (2016). 3d-r2n2: A unified approach for single and multi-view 3d object reconstruction. In European conference on computer vision. Springer.

  • Cui, Y., et al. (2010). 3D shape scanning with a time-of-flight camera. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE.

  • DeGarmo, E. P., et al. (1997). Materials and Process in Manufacturing. Prentice Hall.

    Google Scholar 

  • Debroy, T., et al. (2017). Building digital twins of 3D printing machines. Scripta Materialia, 135, 119–124.

    Article  Google Scholar 

  • Dehoff, R. R., et al. (2018). Electron Beam Melting Technology Improvements. Oak Ridge: Oak Ridge National Lab. (ORNL).

    Google Scholar 

  • DTM Sinterstation 2500 Plus. Retrieved from: https://www.treatstock.com/machines/item/303-dtm-sinterstation-2500-plus.

  • Everton, S. K., et al. (2016). Review of in-situ process monitoring and in-situ metrology for metal additive manufacturing. Materials & Design, 95, 431–445.

    Article  Google Scholar 

  • Fan, H., Su, H., & Guibas, L. J. (2017). A point set generation network for 3d object reconstruction from a single image. In Proceedings of the IEEE conference on computer vision and pattern recognition.

  • Favaro, P., & Soatto, S. (2000). Shape and radiance estimation from the information divergence of blurred images. In European conference on computer vision. Springer.

  • Geng, J. (2011). Structured-light 3D surface imaging: A tutorial. Advances in Optics and Photonics, 3(2), 128–160.

    Article  Google Scholar 

  • Gong, X., Anderson, T., & Chou, K. (2013). Review on powder-based electron beam additive manufacturing technology. In ASME/ISCIE 2012 international symposium on flexible automation. American Society of Mechanical Engineers Digital Collection.

  • Goodfellow, I., et al. (2016). Deep Learning (Vol. 1). Cambridge: MIT Press.

    Google Scholar 

  • Guo, T., et al. (2017). Simple convolutional neural network on image classification. In 2017 IEEE 2nd international conference on big data analysis (ICBDA). IEEE.

  • Han, X.-F., Laga, H., & Bennamoun, M. (2019). Image-based 3D object reconstruction: State-of-the-art and trends in the deep learning era. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(5), 1578–1604.

    Article  Google Scholar 

  • Han, F., & Zhu, S.-C. (2003) Bayesian reconstruction of 3d shapes and scenes from a single image. In First IEEE international workshop on higher-level knowledge in 3D modeling and motion analysis. IEEE.

  • Hassner, T., & Basri, R. (2006). Example based 3D reconstruction from single 2D images. In CVPRW'06, conference on computer vision and pattern recognition workshop. IEEE.

  • Hocken, R. J., & Pereira, P. H. (2016). Coordinate Measuring Machines and Systems. CRC Press.

    Book  Google Scholar 

  • Hoiem, D., Efros, A. A., & Hebert, M. (2007). Recovering surface layout from an image. International Journal of Computer Vision, 75(1), 151–172.

    Article  Google Scholar 

  • Hoiem, D., Efros, A. A., & Hebert, M. (2005) Geometric context from a single image. In ICCV 2005, Tenth IEEE international conference on computer vision. IEEE.

  • International, A., ASTM D638-14. (2015). Standard Test Method for Tensile Properties of Plastics. ASTM International.

    Google Scholar 

  • Jackson, A.S., et al. (2017). Large pose 3D face reconstruction from a single image via direct volumetric CNN regression. In Proceedings of the IEEE international conference on computer vision.

  • Van der Jeught, S., & Dirckx, J. J. J. O. E. (2019). Deep neural networks for single shot structured light profilometry. Optics Express, 27(12), 17091–17101.

    Article  Google Scholar 

  • Ji, S., et al. (2012). 3D convolutional neural networks for human action recognition. IEEE Transactions on Pattern Analysis Machine Intelligence, 35(1), 221–231.

    Article  Google Scholar 

  • Jiang, X. (1973). Precision surface measurement. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 2012(370), 4089–4114.

    Google Scholar 

  • Kasten, Y., Doktofsky, D., & Kovler, I. (2020). End-to-end convolutional neural network for 3D reconstruction of knee bones from bi-planar X-ray images. In International workshop on machine learning for medical image reconstruction. Springer.

  • Kim, J., et al. (2018). Deep CNN-based blind image quality predictor. IEEE Transactions on Neural Networks and Learning Systems, 30(1), 11–24.

    Article  Google Scholar 

  • Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems.

  • Kuhn, A., et al. (2020). DeepC-MVS: Deep confidence prediction for multi-view stereo reconstruction. In 2020 International conference on 3D vision (3DV). IEEE.

  • Laboratory, L.L.N. Powder bed AM. Retrieved from: https://acamm.llnl.gov/am-technology/powder-bed-am.

  • Lague, D., et al. (2013). Accurate 3D comparison of complex topography with terrestrial laser scanner: Application to the Rangitikei canyon (NZ). ISPRS Journal of Photogrammetry and Remote Sensing, 82, 10–26.

    Article  Google Scholar 

  • Li, D., et al. (2019). Lasers structured light with phase-shifting for dense depth perception. Results in Physics, 14, 102433.

    Article  Google Scholar 

  • Li, Y., et al. (2018) Pointcnn: Convolution on x-transformed points. In Advances in neural information processing systems, p. 31.

  • Li, Q., et al. (2014). Medical image classification with convolutional neural network. in 2014 13th international conference on control automation robotics & vision (ICARCV). IEEE.

  • Liu, C., et al. (2019). Image analysis-based closed loop quality control for additive manufacturing with fused filament fabrication. Journal of Manufacturing Systems, 51, 75–86.

    Article  Google Scholar 

  • Liu, C., et al. (2021). An integrated manifold learning approach for high-dimensional data feature extractions and its applications to online process monitoring of additive manufacturing. IISE Transactions. https://doi.org/10.1080/24725854.2020.1849876

    Article  Google Scholar 

  • Liu, J., & Ji, S. (2020). A novel recurrent encoder-decoder structure for large-scale multi-view stereo reconstruction from an open aerial dataset. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition.

  • Lott, P., et al. (2011). Design of an optical system for the in situ process monitoring of selective laser melting (SLM). Physics Procedia, 12, 683–690.

    Article  Google Scholar 

  • Malekipour, E., & El-Mounayri, H. (2018). Defects, process parameters and signatures for online monitoring and control in powder-based additive manufacturing. Mechanics of Additive and Advanced Manufacturing (Vol. 9, pp. 83–90). Springer.

    Google Scholar 

  • Matlab Deep Learning Toolbox. 2018a, MathWorks.

  • Mikhail, E. M., Bethel, J. S., & McGlone, J. C. (2001). Introduction to Modern Photogrammetry. Wiley.

    Google Scholar 

  • Moretti, M., Rossi, A., & Senin, N. J. A. M. (2021). In-process monitoring of part geometry in fused filament fabrication using computer vision and digital twins. Materials Science, 37, 101609.

    Google Scholar 

  • Pawlus, P., Reizer, R., & Wieczorowski, M. J. T. I. (2020). A review of methods of random surface topography modeling. Tribology International, 2020, 106530.

    Article  Google Scholar 

  • Qi, C.R., et al. (2017). Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition.

  • Rao, P. K., et al. (2015). A graph-theoretic approach for quantification of surface morphology variation and its application to chemical mechanical planarization process. Quality & Reliability Engineering, 47(10), 1088–1111.

    Google Scholar 

  • Rocchini, C., et al. (2001). A low cost 3D scanner based on structured light. Computer Graphics Forum, 20, 299–308.

    Article  Google Scholar 

  • Schmidhuber, J. (2015). Deep learning in neural networks: An overview. Neural networks, 61, 85–117.

    Article  Google Scholar 

  • Seitz, S. M., et al. (2006). A comparison and evaluation of multi-view stereo reconstruction algorithms. In IEEE computer society conference on computer vision and pattern recognition (CVPR). IEEE.

  • Shi, Z., et al. (2021). Surface morphology analysis using convolutional autoencoder in additive manufacturing with laser engineered net shaping. Procedia Manufacturing, 53, 16–23.

    Article  Google Scholar 

  • Sing, S. L., et al. (2016). Laser and electron-beam powder-bed additive manufacturing of metallic implants: A review on processes, materials and designs. Journal of Orthopaedic Research, 34(3), 369–385.

    Article  Google Scholar 

  • Snoek, J., Larochelle, H., & Adams, R. P. (2012). Practical bayesian optimization of machine learning algorithms. In Advances in neural information processing systems.

  • Tao, T., et al. (2016). Real-time 3-D shape measurement with composite phase-shifting fringes and multi-view system. Optics Express, 24(18), 20253–20269.

    Article  Google Scholar 

  • Tomasi, C., & Kanade, T. (1992). Shape and motion from image streams under orthography: A factorization method. International Journal of Computer Vision, 9(2), 137–154.

    Article  Google Scholar 

  • Tootooni, M. S., et al. (2016). Online non-contact surface finish measurement in machining using graph theory-based image analysis. Journal of Manufacturing Systems, 41, 266–276.

    Article  Google Scholar 

  • Valente, E. H., et al. (2019). Effect of scanning strategy during selective laser melting on surface topography, porosity, and microstructure of additively manufactured Ti-6Al-4V. Applied Sciences, 9(24), 5554.

    Article  Google Scholar 

  • Wang, R., et al. (2021). Development of structured light 3D-scanner with high spatial resolution and its applications for additive manufacturing quality assurance. The International Journal of Advanced Manufacturing Technology, 117(3), 845–862.

    Google Scholar 

  • Wu, J.-J.J.T.I. (2000). Simulation of rough surfaces with FFT. Tribology International, 33(1), 47–58.

    Article  Google Scholar 

  • Wu, J. (2017). Introduction to Convolutional Neural Networks. National Key Lab for Novel Software Technology. Nanjing University.

    Google Scholar 

  • Yu, K., Xu, W., & Gong, Y. (2009). Deep learning with kernel regularization for visual recognition. in Advances in Neural Information Processing Systems.

  • Zhang, R., et al. (1999). Shape-from-shading: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21(8), 690–706.

    Article  Google Scholar 

  • Zhang, Q., et al. (2019). Recent advances in convolutional neural network acceleration. Neurocomputing, 323, 37–51.

    Article  Google Scholar 

  • Zhang, S., & Li, J. O. (2010). Engineering, Recent progresses on real-time 3D shape measurement using digital fringe projection techniques. Optics and Lasers in Engineering, 48(2), 149–158.

    Article  Google Scholar 

  • Zhang, Z. H., & Li, J. O. (2012). Engineering, Review of single-shot 3D shape measurement by phase calculation-based fringe projection techniques. Optics and Lasers in Engineering, 50(8), 1097–1106.

    Article  Google Scholar 

Download references

Funding

Research reported in this publication was supported by the National Science Foundation under Award Number CMMI 1436592, the Office of Naval Research under Award Number N00014-18–1-2794, and the Department of Defense under award N00014-19–1-2728.

Author information

Authors and Affiliations

  1. Oklahoma State University, Stillwater, USA

    Chenang Liu

  2. Virginia Tech, Blacksburg, USA

    Rongxuan Raphael Wang, Ian Ho, Zhenyu James Kong & Christopher Williams

  3. The University of Tennessee-Knoxville, Knoxville, USA

    Suresh Babu

  4. Manufacturing Demonstration Facility, Oak Ridge National Laboratory, Knoxville, USA

    Suresh Babu & Chase Joslin

Authors
  1. Chenang Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rongxuan Raphael Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ian Ho
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhenyu James Kong
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Christopher Williams
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Suresh Babu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Chase Joslin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhenyu James Kong.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The original online version of this article was revised: The word ‘Oklahoma’ has been corrected in the affiliation of the author Chenang Liu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, C., Wang, R.R., Ho, I. et al. Toward online layer-wise surface morphology measurement in additive manufacturing using a deep learning-based approach. J Intell Manuf 34, 2673–2689 (2023). https://doi.org/10.1007/s10845-022-01933-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10845-022-01933-0

Keywords

Search

Navigation