Skip to main content
SpringerLink
Log in

Concepts and Challenges for 4D Point Clouds as a Foundation of Conscious, Smart City Systems

  • Conference paper
  • First Online:
Computational Science and Its Applications – ICCSA 2022 Workshops (ICCSA 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13377))

Included in the following conference series:

Abstract

Point clouds represent the as-is geometry of indoor and outdoor environments by sets of 3D points. They allow for constructing 3D models of objects, sites, cities, and landscapes and, hence, form the base data for almost any conscious, smart city system and application. For implementing such systems, we need a spatio-temporal data structure that enables efficient storage and access to 4D point clouds. In particular, the data structure should allow continuous updates, change tracking, and support for spatial and spatio-temporal analysis. This paper discusses challenges and approaches for a 4D point cloud data structure. In particular, the challenges arise from repeated scanning of environments in terms of sparsity, data redundancy, and geometric blurring of the corresponding point clouds. We outline a scheme for incremental storage of 4D point clouds via signed distance fields using a sparse, voxel-based representation. To efficiently implement analysis operations, we discuss how the data structure supports access based on both spatial and temporal criteria. In particular, we outline how machine learning-based interpretations used to classify point clouds and derive object-based information can work with the data structure.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Blaha, M., Vogel, C., Richard, A., Wegner, J.D., Pock, T., Schindler, K.: Large-scale semantic 3d reconstruction: An adaptive multi-resolution model for multi-class volumetric labeling. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition, pp. 3176–3184. IEEE Computer Society (2016). https://doi.org/10.1109/CVPR.2016.346

  2. Boulch, A., Guerry, J., Saux, B.L., Audebert, N.: Snapnet: 3d point cloud semantic labeling with 2d deep segmentation networks. Comput. Graph. 71, 189–198 (2018). https://doi.org/10.1016/j.cag.2017.11.010

    Article  Google Scholar 

  3. Brilakis, I., et al.: Built environment digtial twinning. Technical report, International Workshop on Built Environment Digital Twinning presented by TUM Institute for Advanced Study and Siemens AG (2019)

    Google Scholar 

  4. Camuffo, E., Mari, D., Milani, S.: Recent advancements in learning algorithms for point clouds: an updated overview. Sensors 22(4) (2022). https://doi.org/10.3390/s22041357

  5. Curless, B., Levoy, M.: A volumetric method for building complex models from range images. In: Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH 1996, pp. 303–312. Association for Computing Machinery (1996). https://doi.org/10.1145/237170.237269

  6. Daniel, S., Doran, M.A.: Geosmartcity: geomatics contribution to the smart city. In: Proceedings of the 14th Annual International Conference on Digital Government Research, dg.o ’13, pp. 65–71. Association for Computing Machinery (2013). https://doi.org/10.1145/2479724.2479738

  7. El-Saddik, A.: Digital twins: the convergence of multimedia technologies. IEEE Multim. 25(2), 87–92 (2018). https://doi.org/10.1109/MMUL.2018.023121167

    Article  Google Scholar 

  8. Grieves, M.: Digital twin: manufacturing excellence through virtual factory replication. White paper 1, 1–7 (2014)

    Google Scholar 

  9. Hu, Q., et al.: Randla-net: Efficient semantic segmentation of large-scale point clouds. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, pp. 11105–11114. Computer Vision Foundation/IEEE (2020). https://doi.org/10.1109/CVPR42600.2020.01112

  10. Izadi, S., et al.: Kinectfusion: real-time 3d reconstruction and interaction using a moving depth camera. In: Pierce, J.S., Agrawala, M., Klemmer, S.R. (eds.) Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology, pp. 559–568. ACM (2011). https://doi.org/10.1145/2047196.2047270

  11. Jones, M.W.: Distance field compression. In: The 12-th International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision’2004, WSCG 2004, pp. 199–204 (2004)

    Google Scholar 

  12. Khajavi, S.H., Motlagh, N.H., Jaribion, A., Werner, L.C., Holmström, J.: Digital twin: vision, benefits, boundaries, and creation for buildings. IEEE Access 7, 147406–147419 (2019). https://doi.org/10.1109/ACCESS.2019.2946515

    Article  Google Scholar 

  13. Lague, D., Brodu, N., Leroux, J.: Accurate 3d comparison of complex topography with terrestrial laser scanner: Application to the rangitikei canyon (n-z). ISPRS J. Photogramm. Remote. Sens. 82, 10–26 (2013). https://doi.org/10.1016/j.isprsjprs.2013.04.009

    Article  Google Scholar 

  14. Laney, D.E., Bertram, M., Duchaineau, M.A., Max, N.L.: Multiresolution distance volumes for progressive surface compression. In: 1st International Symposium on 3D Data Processing Visualization and Transmission (3DPVT 2002), pp. 470–479. IEEE Computer Society (2002). https://doi.org/10.1109/TDPVT.2002.1024102

  15. Liu, H., Yuan, H., Liu, Q., Hou, J., Liu, J.: A comprehensive study and comparison of core technologies for MPEG 3-d point cloud compression. IEEE Trans. Broadcast. 66(3), 701–717 (2020). https://doi.org/10.1109/TBC.2019.2957652

    Article  Google Scholar 

  16. Milani, S., Polo, E., Limuti, S.: A transform coding strategy for dynamic point clouds. IEEE Trans. Image Process. 29, 8213–8225 (2020). https://doi.org/10.1109/TIP.2020.3011811

    Article  MATH  Google Scholar 

  17. Miller, A., Jain, V., Mundy, J.L.: Real-time rendering and dynamic updating of 3-d volumetric data. GPGPU-4. Association for Computing Machinery (2011). https://doi.org/10.1145/1964179.1964190

  18. Minerva, R., Crespi, N.: Digital twins: properties, software frameworks, and application scenarios. IT Prof. 23(1), 51–55 (2021). https://doi.org/10.1109/MITP.2020.2982896

    Article  Google Scholar 

  19. Minoli, D., Sohraby, K., Occhiogrosso, B.: Iot considerations, requirements, and architectures for smart buildings - energy optimization and next-generation building management systems. IEEE Internet Things J. 4(1), 269–283 (2017). https://doi.org/10.1109/JIOT.2017.2647881

    Article  Google Scholar 

  20. Mitchell, W.J.: Smart cities: Vision. https://smartcities.media.mit.edu/frameset.html. Accessed 11 May 2022

  21. Nebiker, S., Bleisch, S., Christen, M.: Rich point clouds in virtual globes - a new paradigm in city modeling? Comput. Environ. Urban Syst. 34(6), 508–517 (2010). https://doi.org/10.1016/j.compenvurbsys.2010.05.002

    Article  Google Scholar 

  22. Nießner, M., Zollhöfer, M., Izadi, S., Stamminger, M.: Real-time 3d reconstruction at scale using voxel hashing. ACM Trans. Graph. 32(6), 169:1–169:11 (2013). https://doi.org/10.1145/2508363.2508374

  23. Park, J.J., Florence, P., Straub, J., Newcombe, R.A., Lovegrove, S.: Deepsdf: Learning continuous signed distance functions for shape representation. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, pp. 165–174. Computer Vision Foundation/IEEE (2019). https://doi.org/10.1109/CVPR.2019.00025

  24. Qi, C.R., Su, H., Mo, K., Guibas, L.J.: Pointnet: deep learning on point sets for 3d classification and segmentation. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017. pp. 77–85. IEEE Computer Society (2017). https://doi.org/10.1109/CVPR.2017.16

  25. Qi, Q., Tao, F.: Digital twin and big data towards smart manufacturing and industry 4.0: 360 degree comparison. IEEE Access 6, 3585–3593 (2018). https://doi.org/10.1109/ACCESS.2018.2793265

  26. de Queiroz, R.L., Chou, P.A.: Motion-compensated compression of dynamic voxelized point clouds. IEEE Trans. Image Process. 26(8), 3886–3895 (2017). https://doi.org/10.1109/TIP.2017.2707807

    Article  MathSciNet  MATH  Google Scholar 

  27. Richter, R., Discher, S., Döllner, J.: Out-of-Core Visualization of Classified 3D Point Clouds, pp. 227–242. Springer International Publishing, Cham (2015). https://doi.org/10.1007/978-3-319-12181-9_14

  28. Rong, G., Tan, T.S.: Jump flooding in gpu with applications to voronoi diagram and distance transform, vol. 2006, pp. 109–116, January 2006. https://doi.org/10.1145/1111411.1111431

  29. Rosu, R.A., Schütt, P., Quenzel, J., Behnke, S.: Latticenet: fast point cloud segmentation using permutohedral lattices. In: Toussaint, M., Bicchi, A., Hermans, T. (eds.) Robotics: Science and Systems XVI, Virtual Event (2020). https://doi.org/10.15607/RSS.2020.XVI.006

  30. Stojanovic, V., Trapp, M., Richter, R., Hagedorn, B., Döllner, J.: Towards the generation of digital twins for facility management based on 3d point clouds. In: 34th Annual ARCOM Conference, pp. 270–279 (2018)

    Google Scholar 

  31. Tchapmi, L.P., Choy, C.B., Armeni, I., Gwak, J., Savarese, S.: Segcloud: Semantic segmentation of 3d point clouds. In: 2017 International Conference on 3D Vision, 3DV 2017, pp. 537–547. IEEE Computer Society (2017). https://doi.org/10.1109/3DV.2017.00067

  32. Thanou, D., Chou, P.A., Frossard, P.: Graph-based compression of dynamic 3d point cloud sequences. Trans. Img. Proc. 25(4), 1765–1778 (2016). https://doi.org/10.1109/TIP.2016.2529506

  33. Ulusoy, A.O., Biris, O., Mundy, J.L.: Dynamic probabilistic volumetric models. In: IEEE International Conference on Computer Vision, ICCV 2013, pp. 505–512. IEEE Computer Society (2013). https://doi.org/10.1109/ICCV.2013.68

  34. Ulusoy, A.O., Mundy, J.L.: Image-based 4-d reconstruction using 3-d change detection. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8691, pp. 31–45. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10578-9_3

    Chapter  Google Scholar 

  35. Wan, L., Nochta, T., Schooling, J.: Developing a city-level digital twin - propositions and a case study. In: International Conference on Smart Infrastructure and Construction 2019, ICSIC 2019: Driving Data-Informed Decision-Making, pp. 187–193 (01 2019). https://doi.org/10.1680/icsic.64669.187

  36. Wu, B., Wan, A., Yue, X., Keutzer, K.: Squeezeseg: Convolutional neural nets with recurrent CRF for real-time road-object segmentation from 3d lidar point cloud. In: 2018 IEEE International Conference on Robotics and Automation, ICRA 2018, pp. 1887–1893. IEEE (2018). https://doi.org/10.1109/ICRA.2018.8462926

Download references

Acknowledgements

We thank the anonymous reviewers for their valuable feedback. This work was partially supported by the Federal Ministry of Education and Research (BMBF), Germany (PunctumTube, 01IS18090) and by the Federal Ministry for Digital and Transport (BMDV), Germany (TWIN4ROAD, 19F2210).

Author information

Authors and Affiliations

  1. Hasso Plattner Institute, Prof.-Dr.-Helmert-Straße 2-3, 14482, Potsdam, Germany

    Ole Wegen, Jürgen Döllner & Rico Richter

Authors
  1. Ole Wegen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jürgen Döllner
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rico Richter
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ole Wegen .

Editor information

Editors and Affiliations

  1. University of Perugia, Perugia, Italy

    Osvaldo Gervasi

  2. University of Basilicata, Potenza, Potenza, Italy

    Beniamino Murgante

  3. Østfold University College, Halden, Norway

    Sanjay Misra

  4. University of Minho, Braga, Portugal

    Ana Maria A. C. Rocha

  5. University of Cagliari, Cagliari, Italy

    Chiara Garau

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wegen, O., Döllner, J., Richter, R. (2022). Concepts and Challenges for 4D Point Clouds as a Foundation of Conscious, Smart City Systems. In: Gervasi, O., Murgante, B., Misra, S., Rocha, A.M.A.C., Garau, C. (eds) Computational Science and Its Applications – ICCSA 2022 Workshops. ICCSA 2022. Lecture Notes in Computer Science, vol 13377. Springer, Cham. https://doi.org/10.1007/978-3-031-10536-4_39

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-10536-4_39

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-10535-7

  • Online ISBN: 978-3-031-10536-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation