Skip to main content
SpringerLink
Log in

Hyperspectral 3D Point Cloud Segmentation Using RandLA-Net

  • Conference paper
  • First Online:
Intelligent Autonomous Systems 17 (IAS 2022)

Part of the book series: Lecture Notes in Networks and Systems ((LNNS,volume 577))

Included in the following conference series:

Abstract

Point clouds are commonly used in robotics to represent 3D maps. To gain further understanding of their content, it is useful to annotate such maps semantically. To segment 3D point clouds with RGB values, different solutions exist. In machine learning, pre-trained classifiers are used for this purpose. Since it is not always possible to differentiate between entities relying solely on RGB information, hyperspectral histograms can augment the 3D data. The aim of this work is to evaluate, if hyperspectral information can improve the segmentation results for ambiguous objects, e.g., streets, sidewalks, and cars using established deep learning methods. Given the reported performance on geometrical 3D data and the possibility to directly integrate point annotations, we extended the neural network RandLA-Net. In addition to the evaluation of RandLA-Net in this context, we also provide a reference dataset consisting of semantically annotated hyperspectral 3D point clouds.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 259.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 329.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Available in the Robotic 3D Scan Repository: http://kos.informatik.uni-osnabrueck.de/3Dscans/.

References

  1. Guo, Y., Wang, H., Hu, Q., Liu, H., Liu, L., Bennamoun, M.: Deep learning for 3d point clouds: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 43(12), 4338–4364 (2021)

    Article  Google Scholar 

  2. Hu, Q., Yang, B., Xie, L., Rosa, S., Guo, Y., Wang, Z., Trigoni, N., Markham, A.: Randla-net: Efficient semantic segmentation of large-scale point clouds. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2020, June)

    Google Scholar 

  3. Hackel, T., Savinov, N., Ladicky, L., Wegner, J.D., Schindler, K., Pollefeys, M.: Semantic3d. net: a new large-scale point cloud classification benchmark. arXiv preprint arXiv:1704.03847 (2017)

  4. Igelbrink, F., Wiemann, T., Püetz, S., Hertzberg, J.: Markerless Ad-hoc Calibration of a Hyperspectral Camera and a 3D Laser Scanner. Advances in Intelligent Systems and Computing. Springer International Publishing (2018)

    Google Scholar 

  5. Ghamisi, P., Plaza, J., Chen, Y., Li, J., Plaza, A.J.: Advanced spectral classifiers for hyperspectral images: a review. IEEE Geosci. Remote Sens. Mag. 5(1), 8–32 (2017)

    Article  Google Scholar 

  6. Audebert, N., Le Saux, B., Lefèvre, S.: Deep learning for classification of hyperspectral data: a comparative review. IEEE Geosci. Remote Sens. Mag. 7(2), 159–173 (2019)

    Article  Google Scholar 

  7. Dargan, S., Kumar, M., Ayyagari, M.R., Kumar, G.: A survey of deep learning and its applications: a new paradigm to machine learning. Arch. Comput. Methods Eng. 27(4), 1071–1092 (2019)

    Article  MathSciNet  Google Scholar 

  8. Qi, C.R., Su, H., Mo, K., Guibas, L.J.: Pointnet: Deep learning on point sets for 3d classification and segmentation. In: Proceedings of the IEEE Conference on computer Vision and Pattern Recognition, pp. 652–660 (2017)

    Google Scholar 

  9. Qi, C.R., Yi, L., Su, H., Guibas, L.J.: Pointnet++: Deep hierarchical feature learning on point sets in a metric space. arXiv preprint arXiv:1706.02413 (2017)

  10. Armeni, I., Sax, A., Zamir, A.R., Savarese, S.: Joint 2D-3D-Semantic Data for Indoor Scene Understanding. ArXiv e-prints (2017, February)

    Google Scholar 

  11. Roynard, X., Deschaud, J.E., Goulette, F.: Paris-Lille-3d: a large and high-quality ground-truth urban point cloud dataset for automatic segmentation and classification. Int. J. Robot. Res. 37(6), 545–557 (2018)

    Article  Google Scholar 

  12. Behley, J., Garbade, M., Milioto, A., Quenzel, J., Behnke, S., Stachniss, C., Gall, J.: Semantickitti: a dataset for semantic scene understanding of lidar sequences. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) (2019)

    Google Scholar 

  13. Manolakis, D., Lockwood, R., Cooley, T.: Hyperspectral Imaging Remote Sensing. Cambridge University Press (2016)

    Google Scholar 

  14. Xue, J., Su, B.: Significant remote sensing vegetation indices: a review of developments and applications. J. Sens. 2017 (2017)

    Google Scholar 

  15. Wu, D., Sun, D.W.: Advanced applications of hyperspectral imaging technology for food quality and safety analysis and assessment: a review-part i: fundamentals. Innovative Food Sci. Emerg. Technol. 19, 1–14 (2013)

    Article  Google Scholar 

  16. Van der Meer, F.D., Van der Werff, H.M., Van Ruitenbeek, F.J., Hecker, C.A., Bakker, W.H., Noomen, M.F., Van Der Meijde, M., Carranza, E.J.M., De Smeth, J.B., Woldai, T.: Multi-and hyperspectral geologic remote sensing: a review. Int. J. Appl. Earth Obs. Geoinf. 14(1), 112–128 (2012)

    Google Scholar 

  17. Goetz, A.F.: Three decades of hyperspectral remote sensing of the earth: a personal view. Remote Sens. Environ. 113, S5–S16 (2009)

    Article  Google Scholar 

  18. Hege, E.K., O’Connell, D., Johnson, W., Basty, S., Dereniak, E.L.: Hyperspectral imaging for astronomy and space surveillance. In: Imaging Spectrometry IX, Vol. 5159. International Society for Optics and Photonics, pp. 380–391 (2004)

    Google Scholar 

  19. Vaiphasa, C.: Consideration of smoothing techniques for hyperspectral remote sensing. ISPRS J. Photogrammetry Remote Sens. 60(2), 91–99 (2006)

    Article  Google Scholar 

  20. Raczkowska, M.K., Koziol, P., Urbaniak-Wasik, S., Paluszkiewicz, C., Kwiatek, W.M., Wrobel, T.P.: Influence of denoising on classification results in the context of hyperspectral data: high definition FT-IR imaging. Anal. Chimica Acta 1085, 39–47 (2019)

    Article  Google Scholar 

  21. Savitzky, A., Golay, M.J.E.: Smoothing and differentiation of data by simplified least squares procedures. Anal. Chem. 36(8), 1627–1639 (1964)

    Article  Google Scholar 

  22. Green, A., Berman, M., Switzer, P., Craig, M.: A transformation for ordering multispectral data in terms of image quality with implications for noise removal. IEEE Trans. Geosci. Remote Sens. 26(1), 65–74 (1988)

    Article  Google Scholar 

  23. Lee, J., Woodyatt, A., Berman, M.: Enhancement of high spectral resolution remote-sensing data by a noise-adjusted principal components transform. IEEE Trans. Geosci. Remote Sens. 28(3), 295–304 (1990)

    Article  Google Scholar 

  24. Rinnan, A., van den Berg, F., Engelsen, S.B.: Review of the most common pre-processing techniques for near-infrared spectra. TrAC Trends Anal. Chem. 28(10), 1201–1222 (2009)

    Article  Google Scholar 

  25. Barnes, R.J., Dhanoa, M.S., Lister, S.J.: Standard normal variate transformation and de-trending of near-infrared diffuse reflectance spectra. Appl. Spectrosc. 43(5), 772–777 (1989)

    Article  Google Scholar 

  26. Geladi, P., MacDougall, D., Martens, H.: Linearization and scatter-correction for near-infrared reflectance spectra of meat. Appl. Spectrosc. 39(3), 491–500 (1985)

    Article  Google Scholar 

  27. Buckley, S.J., Kurz, T.H., Howell, J.A., Schneider, D.: Terrestrial lidar and hyperspectral data fusion products for geological outcrop analysis. Comput. Geosci. 54, 249–258 (2013)

    Article  Google Scholar 

  28. Weinmann, M.: Fusion of hyperspectral, multispectral, color and 3d point cloud information for the semantic interpretation of urban environments. Int. Arch. Photogrammetry Remote Sens. Spat. Inf. Sci. (2019)

    Google Scholar 

  29. Brell, M., Segl, K., Guanter, L., Bookhagen, B.: 3d hyperspectral point cloud generation: fusing airborne laser scanning and hyperspectral imaging sensors for improved object-based information extraction. ISPRS J. Photogrammetry Remote Sens. 149, 200–214 (2019)

    Article  Google Scholar 

  30. Roy Choudhury, M., Das, S., Christopher, J., Apan, A., Chapman, S., Menzies, N.W., Dang, Y.P.: Improving biomass and grain yield prediction of wheat genotypes on sodic soil using integrated high-resolution multispectral, hyperspectral, 3d point cloud, and machine learning techniques. Remote Sens. 13(17) (2021)

    Google Scholar 

  31. Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press (2016) http://www.deeplearningbook.org

  32. Rusu, R.B., Marton, Z.C., Blodow, N., Dolha, M., Beetz, M.: Towards 3d point cloud based object maps for household environments. Robot. Auton. Syst. 56(11), 927–941 (2008)

    Article  Google Scholar 

  33. Zhou, Q., Park, J., Koltun, V.: Open3d: a modern library for 3d data processing. CoRR abs/1801.09847 (2018)

    Google Scholar 

  34. Wiemann, T., Igelbrink, F., Pütz, S., Hertzberg, J.: A file structure and reference data set for high resolution hyperspectral 3d point clouds. IFAC-PapersOnLine 52(8), 403–408 (2019)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Osnabrück University, Institute of Computer Science, Berghoffstr. 11, 49090, Osnabrück, Germany

    Isaak Mitschke, Thomas Wiemann, Felix Igelbrink & Joachim Hertzberg

  2. DFKI Niedersachsen Lab, Plan Based Robot Control, Berghoffstr. 11, 49090, Osnabrück, Germany

    Thomas Wiemann & Joachim Hertzberg

Authors
  1. Isaak Mitschke
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Thomas Wiemann
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Felix Igelbrink
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Joachim Hertzberg
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Isaak Mitschke .

Editor information

Editors and Affiliations

  1. Faculty of Electrical Engineering, University of Zagreb, Zagreb, Croatia

    Ivan Petrovic

  2. Department of Information Engineering, University of Padua, Padua, Italy

    Emanuele Menegatti

  3. Faculty of Electrical Engineering, University of Zagreb, Zagreb, Croatia

    Ivan Marković

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Mitschke, I., Wiemann, T., Igelbrink, F., Hertzberg, J. (2023). Hyperspectral 3D Point Cloud Segmentation Using RandLA-Net. In: Petrovic, I., Menegatti, E., Marković, I. (eds) Intelligent Autonomous Systems 17. IAS 2022. Lecture Notes in Networks and Systems, vol 577. Springer, Cham. https://doi.org/10.1007/978-3-031-22216-0_21

Download citation

Publish with us

Policies and ethics

Search

Navigation