Skip to main content

Advertisement

Springer Nature Link
Log in

Segmentation of unbalanced and in-homogeneous point clouds and its application to 3D scanned trees

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Segmentation of 3D point clouds is still an open issue in the case of unbalanced and in-homogeneous data-sets. In the application context of the modeling of botanical trees, a fundamental challenge consists in separating the leaves from the wood. Based on deep learning and a class decision process, we propose an innovative method designed to separate leaf points from wood points in terrestrial LiDAR point clouds of trees. Although simple, our approach learns trees characteristic point patterns efficiently and robustly. To train our 3D deep learning model, we constructed a 3D labeled point cloud data-set of different tree species. Experiments show that our 3D deep representation together with our geometric approach leads to significant improvement over the state-of-the-art methods in segmentation task.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Ballard, D.H.: Generalizing the Hough transform to detect arbitrary shapes. Pattern Recognit. 13(2), 111–122 (1981)

    MATH  Google Scholar 

  2. Béland, M., Baldocchi, D.D., Widlowski, J.L., Fournier, R.A., Verstraete, M.M.: On seeing the wood from the leaves and the role of voxel size in determining leaf area distribution of forests with terrestrial LiDAR. Agric. For. Meteorol. 184, 82–97 (2014)

    Google Scholar 

  3. Bennett, N.D., Croke, B.F.W., Guariso, G., et al.: Characterising performance of environmental models. Environ. Model. Softw. 40, 1–20 (2013)

    Google Scholar 

  4. Bhanu, B., Lee, S., Ho, C.C., Henderson, T.: Range data processing: representation of surfaces by edges. In: Proceedings of the Eighth International Conference on Pattern Recognition, pp. 236–238. IEEE CS Press (1986)

  5. Biasotti, S., Lavoué, G., Falcidieno, B., Pratikakis, I.: Generalizing discrete convolutions for unstructured point clouds

  6. Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)

    MATH  Google Scholar 

  7. Briechle, S., Krzystek, P., Vosselman, G.: Semantic labeling of als point clouds for tree species mapping using the deep neural network pointnet++. Remote Sensing & Spatial Information Sciences, International Archives of the Photogrammetry (2019)

  8. Burt, A., Disney, M., Calders, K.: Extracting individual trees from Lidar point clouds using treeseg. Methods Ecol. Evol. 10(3), 438–445 (2019)

    Google Scholar 

  9. Chen, J., Chen, B.: Architectural modeling from sparsely scanned range data. Int. J. Comput. Vis. 78(2–3), 223–236 (2008)

    Google Scholar 

  10. Congalton, R.G., Green, K.: Assessing the Accuracy of Remotely Sensed Data: Principles and Practices. CRC Press, Boca Raton (2008)

    Google Scholar 

  11. Côté, J.F., Widlowski, J.L., Fournier, R.A., Verstraete, M.M.: The structural and radiative consistency of three-dimensional tree reconstructions from terrestrial LiDAR. Remote Sens. Environ. 1067–1081 (2009)

  12. Dassot, M., Constant, T., Fournier, M.: The use of terrestrial LiDAR technology in forest science: application fields, benefits and challenges. Ann. For. Sci. 959–974 (2011)

  13. Ester, M., Kriegel, H.P., Sander, J., Xu, X.: A density-based algorithm for discovering clusters a density-based algorithm for discovering clusters in large spatial databases with noise. In: Proceedings of the Second International Conference on Knowledge Discovery and Data Mining, KDD’96, pp. 226–231. AAAI Press (1996)

  14. Ferrara, R., Virdis, S.G., Ventura, A., Ghisu, T., Duce, P., Pellizzaro, G.: An automated approach for wood-leaf separation from terrestrial LiDAR point clouds using the density based clustering algorithm DBSCAN. Agric. For. Meteorol. 262, 434–444 (2018)

    Google Scholar 

  15. Filin, S.: Surface clustering from airborne laser scanning data. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 34(3/A), 119–124 (2002)

    Google Scholar 

  16. Filin, S., Pfeifer, N.: Segmentation of airborne laser scanning data using a slope adaptive neighborhood. ISPRS J. Photogramm. Remote Sens. 60(2), 71–80 (2006)

    Google Scholar 

  17. Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)

    MathSciNet  Google Scholar 

  18. Golovinskiy, A., Funkhouser, T.: Min-cut based segmentation of point clouds. In: 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops, pp. 39–46. IEEE (2009)

  19. Golovinskiy, A., Kim, V.G., Funkhouser, T.: Shape-based recognition of 3D point clouds in urban environments. In: 2009 IEEE 12th International Conference on Computer Vision, pp. 2154–2161. IEEE (2009)

  20. Hackenberg, J., Spiecker, H., Calders, K., Disney, M., Raumonen, P.: Simpletree—an efficient open source tool to build tree models from TLS clouds. Forests 6(11), 4245–4294 (2015)

    Google Scholar 

  21. He, H., Garcia, E.A.: Learning from imbalanced data. IEEE Trans. Knowl. Data Eng. 9, 1263–1284 (2008)

    Google Scholar 

  22. Heinzel, J., Huber, M.O.: Constrained spectral clustering of individual trees in dense forest using terrestrial laser scanning data. Remote Sens. 10(7), 1056 (2018)

    Google Scholar 

  23. Klasing, K., Althoff, D., Wollherr, D., Buss, M.: Comparison of surface normal estimation methods for range sensing applications. In: 2009 IEEE International Conference on Robotics and Automation, pp. 3206–3211. IEEE (2009)

  24. Klokov, R., Lempitsky, V.: Escape from cells: deep KD-networks for the recognition of 3D point cloud models. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 863–872 (2017)

  25. Landrieu, L., Simonovsky, M.: Large-scale point cloud semantic segmentation with superpoint graphs. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2018)

  26. Lang, A.H., Vora, S., Caesar, H., Zhou, L., Yang, J., Beijbom, O.: Pointpillars: fast encoders for object detection from point clouds. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 12,697–12,705 (2019)

  27. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)

    Google Scholar 

  28. Lefsky, M.A., Cohen, W.B., Parker, G.G., Harding, D.J.: LiDAR remote sensing for ecosystem studies. Bioscience 52(1), 19–30 (2002)

    Google Scholar 

  29. Li, Y., Bu, R., Sun, M., Wu, W., Di, X., Chen, B.: Pointcnn: convolution on x-transformed points. In: Advances in Neural Information Processing Systems, pp. 820–830 (2018)

  30. Matthews, B.W.: Comparison of the predicted and observed secondary structure of T4 phage lysozyme. Biochimica et Biophysica Acta (BBA)-Protein Structure 405(2), 442–451 (1975)

    Google Scholar 

  31. Maturana, D., Scherer, S.: VoxNet: a 3D convolutional neural network for real-time object recognition. In: 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 922–928. IEEE (2015)

  32. Momo Takoudjou, S., Ploton, P., Sonké, B., et al.: Using terrestrial laser scanning data to estimate large tropical trees biomass and calibrate allometric models: a comparison with traditional destructive approach. Methods Ecol. Evol. 9(4), 905–916 (2018)

    Google Scholar 

  33. Morel, J., Bac, A., Vega, C.: Surface reconstruction of incomplete datasets: a novel Poisson surface approach based on CSRBF. Comput. Graph. 74, 44–55 (2018)

    Google Scholar 

  34. Nguyen, A., Le, B.: 3D point cloud segmentation: a survey. In: Proc. 6th IEEE Conference on Robotics, Automation and Mechatronics (RAM), pp. 225–230. IEEE (2013)

  35. Niemeyer, J., Rottensteiner, F., Soergel, U.: Classification of urban LiDAR data using conditional random field and random forests. In: Joint Urban Remote Sensing Event 2013, pp. 139–142. IEEE (2013)

  36. Ning, X., Zhang, X., Wang, Y., Jaeger, M.: Segmentation of architecture shape information from 3D point cloud. In: Proceedings of the 8th International Conference on Virtual Reality Continuum and its Applications in Industry, pp. 127–132. ACM (2009)

  37. Olagoke, A., Proisy, C., Féret, J.B., Blanchard, E., Fromard, F., Mehlig, U., de Menezes, M.M., dos Santos, V.F., Berger, U.: Extended biomass allometric equations for large mangrove trees from terrestrial LiDAR data. Trees 30(3), 935–947 (2015)

    Google Scholar 

  38. Pauly, M., Gross, M., Kobbelt, L.P.: Efficient simplification of point-sampled surfaces. In: Proceedings of the Conference on Visualization’02, pp. 163–170. IEEE Computer Society (2002)

  39. Pedregosa, F., Varoquaux, G., Gramfort, A., et al.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)

    MathSciNet  MATH  Google Scholar 

  40. Pfennigbauer, M., Ullrich, A.: Improving quality of laser scanning data acquisition through calibrated amplitude and pulse deviation measurement. In: Laser Radar Technology and Applications XV, Vol. 7684, p. 76841F. International Society for Optics and Photonics (2010)

  41. Pharr, M., Jakob, W., Humphreys, G.: Physically based rendering: From theory to implementation. Morgan Kaufmann, Burlington (2016)

    Google Scholar 

  42. Pu, S., Vosselman, G., et al.: Automatic extraction of building features from terrestrial laser scanning. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 36(5), 25–27 (2006)

    Google Scholar 

  43. Qi, C.R., Liu, W., Wu, C., Su, H., Guibas, L.J.: Frustum pointnets for 3d object detection from rgb-d data. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2018)

  44. Qi, C.R., Su, H., Mo, K., Guibas, L.J.: Pointnet: deep learning on point sets for 3D classification and segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 652–660 (2017)

  45. Qi, C.R., Su, H., Nießner, M., Dai, A., Yan, M., Guibas, L.J.: Volumetric and multi-view CNNs for object classification on 3D data. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5648–5656 (2016)

  46. Qi, C.R., Yi, L., Su, H., Guibas, L.J.: Pointnet++: deep hierarchical feature learning on point sets in a metric space. In: Advances in Neural Information Processing Systems, pp. 5099–5108 (2017)

  47. Raumonen, P., Kaasalainen, M., Åkerblom, M., Kaasalainen, S., Kaartinen, H., Vastaranta, M., Holopainen, M., Disney, M., Lewis, P.: Fast automatic precision tree models from terrestrial laser scanner data. Remote Sens. 5(2), 491–520 (2013)

    Google Scholar 

  48. Ravaglia, J., Bac, A., Fournier, R.: Tree stem reconstruction from terrestrial laser scanner point cloud using Hough transform and open active contours. In: Silvilaser 2015 (2015)

  49. Ravaglia, J., Bac, A., Fournier, R.A.: Extraction of tubular shapes from dense point clouds and application to tree reconstruction from laser scanned data. Comput. Graph. 66, 23–33 (2017)

    Google Scholar 

  50. Ravanbakhsh, S., Schneider, J., Poczos, B.: Deep learning with sets and point clouds. arXiv preprint arXiv:1611.04500 (2016)

  51. Riegler, G., Osman Ulusoy, A., Geiger, A.: Octnet: learning deep 3D representations at high resolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3577–3586 (2017)

  52. Rusu, R.B.: Semantic 3D object maps for everyday manipulation in human living environments. Ph.D. thesis, Computer Science department, Technische Universitaet Muenchen, Germany (2009)

  53. Rusu, R.B., Blodow, N., Beetz, M.: Fast point feature histograms (FPFH) for 3D registration. In: 2009 IEEE International Conference on Robotics and Automation, pp. 3212–3217. IEEE (2009)

  54. Rusu, R.B., Holzbach, A., Blodow, N., Beetz, M.: Fast geometric point labeling using conditional random fields. In: 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 7–12. IEEE (2009)

  55. Sappa, A.D., Devy, M.: Fast range image segmentation by an edge detection strategy. In: Proceedings Third International Conference on 3-D Digital Imaging and Modeling, pp. 292–299. IEEE (2001)

  56. Shao, J., Zhang, W., Mellado, N., Wang, N., Jin, S., Cai, S., Luo, L., Lejemble, T., Yan, G.: Slam-aided forest plot mapping combining terrestrial and mobile laser scanning. ISPRS J. Photogramm. Remote Sens. 163, 214–230 (2020)

    Google Scholar 

  57. Shi, S., Wang, X., Li, H.: Pointrcnn: 3d object proposal generation and detection from point cloud. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2019)

  58. Su, H., Jampani, V., Sun, D., Maji, S., Kalogerakis, E., Yang, M.H., Kautz, J.: Splatnet: sparse lattice networks for point cloud processing. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2530–2539 (2018)

  59. Su, H., Maji, S., Kalogerakis, E., Learned-Miller, E.: Multi-view convolutional neural networks for 3D shape recognition. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 945–953 (2015)

  60. Suykens, J.A., Vandewalle, J.: Least squares support vector machine classifiers. Neural Process. Lett. 9(3), 293–300 (1999)

    Google Scholar 

  61. Tao, S., Guo, Q., Xu, S., Su, Y., Li, Y., Wu, F.: A geometric method for wood-leaf separation using terrestrial and simulated LiDAR data. Photogramm. Eng. Remote Sens. 81(10), 767–776 (2015)

    Google Scholar 

  62. Tao, S., Wu, F., Guo, Q., Wang, Y., Li, W., Xue, B., Hu, X., Li, P., Tian, D., Li, C., et al.: Segmenting tree crowns from terrestrial and mobile Lidar data by exploring ecological theories. ISPRS J. Photogramm. Remote Sens. 110, 66–76 (2015)

    Google Scholar 

  63. Wang, C., Samari, B., Siddiqi, K.: Local spectral graph convolution for point set feature learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 52–66 (2018)

  64. Wang, Y., Sun, Y., Liu, Z., Sarma, S.E., Bronstein, M.M., Solomon, J.M.: Dynamic graph CNN for learning on point clouds. arXiv preprint arXiv:1801.07829 (2018)

  65. Wen, Z., Shi, J., Li, Q., He, B., Chen, J.: ThunderSVM: a fast SVM library on GPUs and CPUs. J. Mach. Learn. Res. 19, 1–5 (2018)

    MathSciNet  Google Scholar 

  66. Wijmans, E.: Pointnet++ pytorch (2018). https://github.com/erikwijmans/Pointnet2_PyTorch

  67. Wu, Z., Song, S., Khosla, A., Yu, F., Zhang, L., Tang, X., Xiao, J.: 3D shapenets: a deep representation for volumetric shapes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1912–1920 (2015)

  68. Wybren van Keulen: The Grove. F12, The Biotope, Haren, The Netherlands (2011). https://www.thegrove3d.com

  69. Xu, Y., Fan, T., Xu, M., Zeng, L., Qiao, Y.: Spidercnn: deep learning on point sets with parameterized convolutional filters. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 87–102 (2018)

  70. Yu, X., Hyyppä, J., Vastaranta, M., Holopainen, M., Viitala, R.: Predicting individual tree attributes from airborne laser point clouds based on the random forests technique. ISPRS J. Photogramm. Remote Sens. 66(1), 28–37 (2011)

    Google Scholar 

  71. Zhang, J., Lin, X., Ning, X.: Svm-based classification of segmented airborne LiDAR point clouds in urban areas. Remote Sens. 5(8), 3749–3775 (2013)

    Google Scholar 

  72. Zhang, W., Wan, P., Wang, T., Cai, S., Chen, Y., Jin, X., Yan, G.: A novel approach for the detection of standing tree stems from plot-level terrestrial laser scanning data. Remote Sens. 11(2), 211 (2019)

    Google Scholar 

  73. Zhou, Y., Tuzel, O.: Voxelnet: end-to-end learning for point cloud based 3d object detection. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2018)

Download references

Acknowledgements

The study was supported by Grant JSPS KAKENHI, Grant Number JP19K11990, from the Japan Society for the Promotion of Science (JSPS) funds. Jules Morel was supported by Grant P18796 from from the Japan Society for the Promotion of Science (JSPS). The authors would like to thank Nicolas Barbier at UMR AMAP (Botanique et Modélisation de l’architecture des plantes et des végétations, France) and S. Momo Takoudjou (IRD-AMAP, ENS-UY1) for providing test data. Those data from Cameroon were collected in collaboration with Alpicam company within the IRD project PPR FTH-AC Changement globaux, biodiversité et santé en zone forestière d’Afrique centrale.

Author information

Authors and Affiliations

  1. Kanai Laboratory, Graduate school of Arts and Sciences, University of Tokyo, Tokyo, Japan

    Jules Morel & Takashi Kanai

  2. Laboratoire d’Informatique et des Systèmes, Aix Marseille University, Marseille, France

    Alexandra Bac

Authors
  1. Jules Morel
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Alexandra Bac
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Takashi Kanai
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Jules Morel.

Ethics declarations

Conflict of interest

To our knowledge, there are no issues that would lead to a conflict of interest or disclosure.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Morel, J., Bac, A. & Kanai, T. Segmentation of unbalanced and in-homogeneous point clouds and its application to 3D scanned trees. Vis Comput 36, 2419–2431 (2020). https://doi.org/10.1007/s00371-020-01966-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-020-01966-7

Keywords