Skip to main content
SpringerLink
Log in

Automatic recognition of cylinders and planes from unstructured point clouds

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

3D scanning devices are traditionally employed in reverse engineering tasks that can be carried out semi-automatically, with user assistance. However, their application in manufacturing process control requires automatic point cloud segmentation and extraction of geometric primitives. In this paper, we propose a method for automatic recognition of planes and cylinders (most frequently encountered geometric primitives in mechanical engineering) from unstructured point clouds. The method is based on the scatter of data during least squares fitting of second order surfaces. It consists of three phases and the first phase represents automatic point cloud segmentation. The second phase deals with merging of over-segmented regions and surfaces parameters estimation, whereas the final phase provides extraction of recognized geometric primitives. The method is experimentally verified using three real-world case studies, and its performances are compared with two state of the art recognition algorithms. The results have shown that the proposed method outperforms alternative approaches in terms of appropriately recognized planes and cylinders without surface type confusion, as well as when the recognition of non-existent primitives is considered. In addition, the method determines surfaces parameters with high accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

Notes

  1. It should be noted that in the focus of our research is the method for geometric primitives recognition and not the design of scanning devices.

  2. Plane, open b-spline, cone, cylinder, sphere, other, revolution, extrusion, closed b-spline and torus are considered.

  3. Note that the code does not include parameters estimation.

References

  1. Chen, M., Tang, Y., Zou, X., Huang, K., Li, L., He, Y.: High-accuracy multi-camera reconstruction enhanced by adaptive point cloud correction algorithm. Opt. Lasers Eng. 122, 170–183 (2019)

    Article  Google Scholar 

  2. Li, C., Wang, S., Zhuang, Y., Yan, F.: Deep sensor fusion between 2D laser scanner and IMU for mobile robot localization. IEEE Sens. J. 21, 8501–8509 (2019)

    Article  Google Scholar 

  3. Liu, Y., Tian, X.: Robot path planning with two-axis positioner for non-ideal sphere-pipe joint welding based on laser scanning. Int. J. Adv. Manuf. Technol. 105(1), 1295–1310 (2019)

    Article  Google Scholar 

  4. Yan, M., Zhang, K., Liu, D., Yang, H., Li, Z.: Autonomous programming and adaptive filling of lap joint based on three-dimensional welding-seam model by laser scanning. J. Manuf. Process. 53, 396–405 (2020)

    Article  Google Scholar 

  5. Hwang, S., An, Y.K., Yang, J., Sohn, H.: Remote inspection of internal delamination in wind turbine blades using continuous line laser scanning thermography. Int. J. Precis. Eng. Manuf. Green Technol. 7, 1–14 (2020)

    Article  Google Scholar 

  6. Wang, W., Cai, Y., Wang, H.P., Carlson, B.E., Poss, M.: Quality inspection scheme for automotive laser braze joints. Int. J. Adv. Manuf. Technol. 106(3), 1553–1566 (2020)

    Article  Google Scholar 

  7. Khalid, M.U., Hager, J.M., Kraus, W., Huber, M.F., Toussaint, M.: Deep workpiece region segmentation for bin picking. In: 2019 IEEE 15th International Conference on Automation Science and Engineering (CASE), pp. 1138–1144. IEEE (2019)

  8. Tang, Y., Chen, M., Lin, Y., Huang, X., Huang, K., He, Y., Li, L.: Vision-based three-dimensional reconstruction and monitoring of large-scale steel tubular structures. Adv. Civ. Eng. 2020, 1236021 (2020)

    Google Scholar 

  9. Tang, Y., Li, L., Wang, C., Chen, M., Feng, W., Zou, X., Huang, K.: Real-time detection of surface deformation and strain in recycled aggregate concrete-filled steel tubular columns via four-ocular vision. Robot. Comput. Integr. Manuf. 59, 36–46 (2019)

    Article  Google Scholar 

  10. Tang, Y.C., Wang, C., Luo, L., Zou, X., et al.: Recognition and localization methods for vision-based fruit picking robots: a review. Front. Plant Sci. 11, 510 (2020)

    Article  Google Scholar 

  11. Chen, M., Tang, Y., Zou, X., Huang, K., Huang, Z., Zhou, H., Wang, C., Lian, G.: Three-dimensional perception of orchard banana central stock enhanced by adaptive multi-vision technology. Comput. Electron. Agric. 174, 105508 (2020)

    Article  Google Scholar 

  12. Rabbani, T., Heuvel, F., Vosselman, G.: Segmentation of point clouds using smoothness constraint. Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci. 36, 248–253 (2006)

    Google Scholar 

  13. Tsuchie, S.: Reconstruction of adaptive swept surfaces from scanned data for styling design. Vis. Comput. (2021). https://doi.org/10.1007/s00371-020-02030-0

  14. Markovic, V., Jakovljevic, Z., Miljkovic, Z.: Feature sensitive three-dimensional point cloud simplification using support vector regression. Tehnicki Vjesnik 26(4), 985–994 (2019)

    Google Scholar 

  15. Schnabel, R., Wahl, R., Klein, R.: Efficient RANSAC for point-cloud shape detection. Comput. Graph. Forum 26(2), 214–226 (2007)

    Article  Google Scholar 

  16. Tran, T.T., Cao, V.T., Laurendeau, D.: Extraction of reliable primitives from unorganized point clouds. 3D Research 6(4), 44 (2015)

    Article  Google Scholar 

  17. Jakovljevic, Z., Puzovic, R., Pajic, M.: Recognition of planar segments in point cloud based on wavelet transform. IEEE Trans. Ind. Inform. 11(2), 342–352 (2015)

    Google Scholar 

  18. Vieira, M., Shimada, K.: Surface mesh segmentation and smooth surface extraction through region growing. Comput. Aided Geom. Des. 22(8), 771–792 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  19. Angelo, L.D., Stefano, P.D.: Geometric segmentation of 3D scanned surfaces. Comput. Aided Des. 62, 44–56 (2015)

    Article  Google Scholar 

  20. Angelo, L.D., Stefano, P.D., Morabito, A.: Fillets, rounds, grooves and sharp edges segmentation from 3D scanned surfaces. Comput. Aided Des. 110, 78–91 (2019)

    Article  Google Scholar 

  21. Lai, H.C., Chang, Y.H., Lai, J.Y.: Development of feature segmentation algorithms for quadratic surfaces. Adv. Eng. Softw. 40(10), 1011–1022 (2009)

    Article  MATH  Google Scholar 

  22. Wang, J., Gu, D., Yu, Z., Tan, C., Zhou, L.: A framework for 3D model reconstruction in reverse engineering. Comput. Ind. Eng. 63(4), 1189–1200 (2012)

    Article  Google Scholar 

  23. Attene, M., Falcidieno, B., Spagnuolo, M.: Hierarchical mesh segmentation based on fitting primitives. Vis. Comput. 22(3), 181–193 (2006)

    Article  Google Scholar 

  24. Rabbani, T., Heuvel, F.: Efficient Hough transform for automatic detection of cylinders in point clouds. In: Proceedings of ISPRS Workshop Laser Scan 2005, ISPRS Archives, vol. 36 (2005)

  25. Ogundana, O.O., Huntley, J.M., Coggrave, C.R., Burguete, R.L.: Automated detection of planes in 3-D point clouds using fast Hough transforms. Opt. Eng. 50(5), 1–12 (2011)

    Google Scholar 

  26. Ogundana, T., Coggrave, C.R., Burguete, R.L., Huntley, J.M.: Fast Hough transform for automated detection of spheres in three-dimensional point clouds. Opt. Eng. 46(5), 1–11 (2007)

    Google Scholar 

  27. Mukhopadhyay, P., Chaudhuri, B.B.: A survey of Hough transform. Pattern Recognit. 48(3), 993–1010 (2015)

    Article  Google Scholar 

  28. Tran, T.T., Cao, V.T., Laurendeau, D.: Extraction of cylinders and estimation of their parameters from point clouds. Comput. Graph. 46, 345–357 (2015)

    Article  Google Scholar 

  29. Tran, T.T., Cao, V.T., Laurendeau, D.: eSphere: extracting spheres from unorganized point clouds: how to extract multiple spheres accurately and simultaneously. Vis. Comput. 32(10), 1205–1222 (2016)

    Article  Google Scholar 

  30. Oh, I., Ko, K.: Automated recognition of 3D pipelines from point clouds. Vis. Comput. 37, 1385–1400 (2020)

    Article  Google Scholar 

  31. Liu, Y., Zhang, J., Hou, J., Ren, J., Tang, W.: Cylinder detection in large-scale point cloud of pipeline plant. IEEE Trans. Vis. Comput. Graph. 19(10), 1700–1707 (2013)

    Article  Google Scholar 

  32. Patil, A.K., Holi, P., Lee, S.K., Chai, Y.H.: An adaptive approach for the reconstruction and modeling of as-built 3D pipelines from point clouds. Autom. Constr. 75, 65–78 (2017)

    Article  Google Scholar 

  33. Qi, C.R., Yi, L., Su, H., Guibas, L.J.: Pointnet++: deep hierarchical feature learning on point sets in a metric space. In: Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS-17, pp. 5105–5114. Curran Associates Inc., Red Hook (2017)

  34. Morel, J., Bac, A., Kanai, T.: Segmentation of unbalanced and in-homogeneous point clouds and its application to 3D scanned trees. Vis. Comput. 36(10–12), 2419–2431 (2020)

    Article  Google Scholar 

  35. Wang, J., Xu, C., Dai, L., Zhang, J., Zhong, R.Y.: An unequal learning approach for 3D point cloud segmentation. IEEE Trans. Ind. Inform. 17, 7913–7922 (2020)

    Article  Google Scholar 

  36. Li, Y., Ma, L., Zhong, Z., Cao, D., Li, J.: Tgnet: geometric graph CNN on 3-D point cloud segmentation. IEEE Trans. Geosci. Remote Sens. 58(5), 3588–3600 (2020)

    Article  Google Scholar 

  37. Xu, Y., Arai, S., Tokuda, F., Kosuge, K.: A convolutional neural network for point cloud instance segmentation in cluttered scene trained by synthetic data without color. IEEE Access 8, 70262–70269 (2020)

    Article  Google Scholar 

  38. Nagy, B., Benedek, C.: 3D CNN-based semantic labeling approach for mobile laser scanning data. IEEE Sens. J. 19(21), 10034–10045 (2019)

    Article  Google Scholar 

  39. Sharma, G., Liu, D., Maji, S., Kalogerakis, E., Chaudhuri, S., Měch, R.: ParSeNet: a parametric surface fitting network for 3D point clouds. In: European Conference on Computer Vision, pp. 261–276. Springer (2020)

  40. Winiwarter, L., Mandlburger, G., Schmohl, S., Pfeifer, N.: Classification of ALS point clouds using end-to-end deep learnings. PFG J. Photogramm. Remote Sens. Geoinf. Sci. 87(3), 75–90 (2019)

    Google Scholar 

  41. Lalonde, J.F., Vandapel, N., Hebert, M.: Automatic three-dimensional point cloud processing for forest inventory. Tech. Rep. CMU-RI-TR-06-21, Carnegie Mellon University, Pittsburgh (2006)

  42. Kwon, S.W., Bosche, F., Kim, C., Haas, C.T., Liapi, K.A.: Fitting range data to primitives for rapid local 3D modeling using sparse range point clouds. Autom. Constr. 13(1), 67–81 (2004)

    Article  Google Scholar 

  43. Hubert, M., Rousseeuw, P.J., Branden, K.V.: ROBPCA: a new approach to robust principal component analysis. Technometrics 47(1), 64–79 (2005)

    Article  MathSciNet  Google Scholar 

  44. Nurunnabi, A., Sadahiro, Y., Lindenbergh, R.: Robust cylinder fitting in three-dimensional point cloud data. ISPRS Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci. XLII–1/W1, 63–70 (2017)

    Article  Google Scholar 

  45. do Carmo, M.: Differential Geometry of Curves and Surfaces. Prentice-Hall, Inc., Englewood Cliffs (1976)

    MATH  Google Scholar 

  46. Chaperon, T., Goulette, F.: Extracting cylinders in full 3D data using a random sampling method and the Gaussian image. In: Proceedings of the Vision Modeling and Visualization Conference, pp. 35–42 (2001)

  47. Jakovljevic, Z., Markovic, V.: Recognition of one class quadric surfaces from unstructured point cloud. In: Proceedings of the 8th International Working Conference Total Quality Management—Advanced and Intelligent Approaches, pp. 353–360 (2015)

  48. Jakovljevic, Z., Markovic, V.: Recognition of quadrics from 3D point clouds generated by scanning of rotational parts. J. Prod. Eng. 19(1), 65–68 (2016)

    Google Scholar 

  49. Jakovljevic, Z., Markovic, V., Puzovic, R., Majstorovic, V.: Recognition of one class of quadrics from 3D point clouds. Procedia CIRP 57, 292–297 (2016)

    Article  Google Scholar 

  50. Fitzgibbon, A., Pilu, M., Fisher, R.B.: Direct least square fitting of ellipses. IEEE Trans. Pattern Anal. Mach. Intell. 21(5), 476–480 (1999)

    Article  Google Scholar 

  51. Reza, A., Sengupta, A.S.: Least square ellipsoid fitting using iterative orthogonal transformations. Appl. Math. Comput. 314, 349–359 (2017)

    MathSciNet  MATH  Google Scholar 

  52. Ying, X., Yang, L., Kong, J., Hou, Y., Guan, S., Zha, H.: Direct least square fitting of ellipsoids. In: Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012), pp. 3228–3231 (2012)

  53. Al-sharadqah, A., Chernov, N.: Error analysis for circle fitting algorithms. Electron. J. Stat. 3, 886–911 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  54. Xia, S., Chen, D., Wang, R., Li, J., Zhang, X.: Geometric primitives in LiDAR point clouds: a review. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 13, 685–707 (2020)

    Article  Google Scholar 

  55. GOM GmbH: Atos compact scan—the compact class of scanning. http://www.gom.com/metrology-systems/system-overview/atos-compact-scan.html

  56. Range Vision: Pro 3D scanner. https://rangevision.com/en/products/pro

  57. Koch, S., Matveev, A., Jiang, Z., Williams, F., Artemov, A., Burnaev, E., Alexa, M., Zorin, D., Panozzo, D.: ABC: a big CAD model dataset for geometric deep learning. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2019)

  58. Petitjean, S.: A survey of methods for recovering quadrics in triangle meshes. ACM Comput. Surv. 34(2), 211–262 (2002)

    Article  Google Scholar 

  59. Benkő, P., Martin, R.R., Várady, T.: Algorithms for reverse engineering boundary representation models. Comput. Aided Des. 33(11), 839–851 (2001)

    Article  Google Scholar 

  60. CloudCompare: 3D point cloud and mesh processing software. https://www.danielgm.net/cc (2020)

  61. Sharma, G., Liu, D., Kalogerakis, E., Maji, S., Chaudhuri, S., Měch, R.: ParSeNet: a parametric surface fitting network for 3D point clouds. https://github.com/Hippogriff/parsenet-codebase (2021)

Download references

Acknowledgements

This research was supported by the Science Fund of the Republic of Serbia, Grant No. 6523109, AI-MISSION4.0, 2020–2022, as well as by the Ministry of Education, Science and Technological Development of the Serbian Government, Grant No. 451-03-68/2020-14/200105.

Author information

Authors and Affiliations

  1. Faculty of Mechanical Engineering, University of Belgrade, Belgrade, Serbia

    Veljko Markovic & Zivana Jakovljevic

  2. Faculty of Technical Sciences, University of Novi Sad, Novi Sad, Serbia

    Igor Budak

Authors
  1. Veljko Markovic
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zivana Jakovljevic
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Igor Budak
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Veljko Markovic.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Markovic, V., Jakovljevic, Z. & Budak, I. Automatic recognition of cylinders and planes from unstructured point clouds. Vis Comput 38, 4329–4352 (2022). https://doi.org/10.1007/s00371-021-02299-9

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-021-02299-9

Keywords

Search

Navigation