Skip to main content
SpringerLink
Log in

Detection of inclusion by using 3D laser scanner in composite prepreg manufacturing technique using convolutional neural networks

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

Among different manufacturing techniques available for composite aircraft structures, prepreg-based manual layup is widely used. During the fabrication process, the protective films of the prepregs or other materials used in the process could get inside as a foreign object between the layers. The present method of finding the inclusions during the prepreg layup is by visual inspection in the cleanroom. Carrying out visual inspection is challenging as the layup is usually carried out on large surfaces and reflective by nature. This paper proposes a 3D laser scanner-based approach for the detection of inclusion on flat and curved surfaces. Using the portable laser scanner, the surfaces of each layer are scanned and compared the resulting point clouds using with a reference layer data. Thicknesses between two surfaces are computed with Cloud to Cloud, Mesh to Cloud and Hausdorff distance to enhance the visibility of inclusions. It was found that this approach could enhance the visibility of inclusions over 50 micron and above. These enhanced features are used to train a multiview convolutional neural network to mark the inclusion regions, which can aid the inspector to identify the inclusion regions in a fast and efficient way.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig.12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  1. Foreign object video detection and alert system and method US Patent 6064429 A.

  2. Multispectral imaging system and method for detecting foreign object debris US Patent 20160146741 A1 (Boeing 2014).

  3. Systems and methods for identifying foreign objects and debris and defects during fabrication of a composite structure EP 1503206 A1

  4. http://www.compositesworld.com/articles/laser-projection-envisioning-more-than-template

  5. Non-Contact Ultrasonic Inspection of CFRP Prepregs for Aeronautical Applications During Lay-Up Fabrication” 2013 Joint UFFC, EFTF and PFM Symposium

  6. Bailich, M., et al.: Multidirectional wavelet-based fringe analysis for surface diagnosis by optical profilometry. Procedia Comput. Sci. 148(2019), 627–636 (2019)

    Article  Google Scholar 

  7. Malassioti, S., Strintzis, M.G.: Stereo vision system for precision dimensional inspection of 3D holes. Mach. Vis. Appl. 15(2), 101–113 (2003)

    Article  Google Scholar 

  8. Marani, R., Nitti, M., Cicirelli, G., D’Orazio, T., Stella, E.: High-resolution laser scanning for three-dimensional inspection of drilling tools. Adv. Mech. Eng. (2013). https://doi.org/10.1155/2013/620786

    Article  Google Scholar 

  9. Becker, T., Özkul a M., Stilla, U.: Simulation of close-range photogrammetric systems for industrial surface inspection. In: International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XXXVIII-3/W22, 2011, ISPRS Conference PIA 2011, 5-7 October, Munich, Germany

  10. Wei, Z., Zhou, F., Zhang, G.: 3D coordinates measurement based on structured light sensor. Sens. Actuators A Phys. 120(2), 527–535 (2005)

    Article  Google Scholar 

  11. Jovančević, I.: Exterior inspection of an aircraft using a Pan-Tilt-Zoom camera and a 3D scanner moved by a mobile robot : 2D image processing and 3D point cloud analysis. Optics / Photonic. Ecole des Mines d’Albi-Carmaux, 2016. English

  12. https://www.creaform3d.com/en/metrology-solutions/optical-3d-scanner-metrascan

  13. https://www.cloudcompare.org/doc/wiki/index.php?title=ICP

  14. Distances Computation - CloudCompareWiki “https://www.cloudcompare.org/doc/wiki/index.php?title=Distances_Computation

  15. http://www.cloudcompare.org/doc/wiki/index.php?title=Cloud-to-Mesh_Distance

  16. Bartoň, M., Hanniel, I., Elber, G., Kim, M.-S.: Precise Hausdorff distance computation between polygonal meshes. Comput. Aided Geom. Design 27(8), 580–591 (2010)

    Article  MathSciNet  Google Scholar 

  17. Abdulla, W.: Mask R-CNN for object detection and instance segmentation on Keras and TensorFlow. (2017), https://github.com/matterport/Mask_RCNN

  18. Lin, T.-Y., Dollár, P., Girshick, R., He, K., Hariharan, B., Belongie, S. Feature Pyramid Networks for Object Detection. http://arxiv.org/abs/1612.03144

  19. He, K., Zhang, X., Ren, S., Sun, J: Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778. (2016)

  20. Cignoni, P., Callieri, M., Corsini, M., Dellepiane, M., Ganovelli, F., Ranzuglia, G.: MeshLab: an Open-Source Mesh Processing Tool. In: Sixth Eurographics Italian Chapter Conference, pp. 129–136, (2008)

  21. CloudCompare (version 2.10.1) [GPL software]. (2019). Retrieved from http://www.cloudcompare.org/

Download references

Acknowledgements

Authors thank Department of Science and Technology, Ministry of Science and Technology Government of India, New Delhi, for providing necessary financial support for this work under Advanced Manufacturing Technologies of technology systems development program (TSDP): Sanction order No. DST/TSG/AMT/2015/391 (General). Authors would like to express their gratitude to Dr. Ramesh Sundaram, Head ACD, and Mr. Kotresh Gaddikeri for their useful suggestions and feedback. Authors thank Mr. Kundan Kumar Verma, Mr. Amit Kumar Gupta, Ms. Divyashree Nayak, and Mrs. Athira S. P. for their support during testing.

Author information

Authors and Affiliations

  1. National Aerospace Laboratories CSIR, Bangalore, India

    M. J. Augustin, Vandana Ramesh, R. Krishna Prasad, Nitesh Gupta & M. Ramesh Kumar

Authors
  1. M. J. Augustin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Vandana Ramesh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. R. Krishna Prasad
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nitesh Gupta
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. M. Ramesh Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to M. J. Augustin.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Augustin, M.J., Ramesh, V., Prasad, R.K. et al. Detection of inclusion by using 3D laser scanner in composite prepreg manufacturing technique using convolutional neural networks. Machine Vision and Applications 32, 117 (2021). https://doi.org/10.1007/s00138-021-01241-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00138-021-01241-2

Keywords

Search

Navigation