Skip to main content
SpringerLink
Log in

Upsampling 4D Point Clouds of Human Body via Adversarial Generation

  • Conference paper
  • First Online:
Image Analysis and Processing - ICIAP 2023 Workshops (ICIAP 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14365))

Included in the following conference series:

  • 135 Accesses

Abstract

Time varying sequences of 3D point clouds, or 4D point clouds, are acquired at an increasing pace in several applications (e.g., LiDAR in autonomous or assisted driving). In many cases, such volume of data is transmitted, thus requiring that proper compression tools are applied to either reduce the resolution or the bandwidth. In this paper, we propose a new solution for upscaling of time-varying 3D video point clouds. Our model consists of a specifically designed Graph Convolutional Network that combines Dynamic Edge Convolution and Graph Attention Networks for feature aggregation in a Generative Adversarial setting. To make these modules work in synergy, we present a specific way to sample dense point clouds and provide each node with enough features of its neighbourhood to generate new vertices. Compared to other solutions in the literature that address the same task, our proposed model is capable of obtaining similar results in terms of quality of reconstruction, while using a substantially lower number of parameters (\(\simeq \) 300KB).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Alexa, M., Behr, J., Cohen-Or, D., Fleishman, S., Levin, D., Silva, C.: Computing and rendering point set surfaces. IEEE Trans. Visualization Comput. Graph. 9(1), 3–15 (2003). https://doi.org/10.1109/TVCG.2003.1175093

    Article  Google Scholar 

  2. Bogo, F., Romero, J., Pons-Moll, G., Black, M.J.: Dynamic FAUST: registering human bodies in motion. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017)

    Google Scholar 

  3. Fan, H., Su, H., Guibas, L.J.: A point set generation network for 3D object reconstruction from a single image. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017)

    Google Scholar 

  4. Fey, M., Lenssen, J.E.: Fast graph representation learning with pytorch geometric. arXiv preprint arXiv:1903.02428 (2019)

  5. Graziosi, D., Nakagami, O., Kuma, S., Zaghetto, A., Suzuki, T., Tabatabai, A.: An overview of ongoing point cloud compression standardization activities: video-based (v-pcc) and geometry-based (g-pcc). APSIPA Trans. Signal Inf. Process. 9, e13 (2020)

    Article  Google Scholar 

  6. Huang, H., Li, D., Zhang, H., Ascher, U., Cohen-Or, D.: Consolidation of unorganized point clouds for surface reconstruction. ACM Trans. Graph. 28(5), 1–7 (2009)

    Article  Google Scholar 

  7. Huang, H., Wu, S., Gong, M., Cohen-Or, D., Ascher, U., Zhang, H.R.: Edge-aware point set resampling. ACM Trans. Graph. 32(1), 1–12 (2013)

    Article  Google Scholar 

  8. Jang, E.S., et al.: Video-based point-cloud-compression standard in MPEG: from evidence collection to committee draft [standards in a nutshell]. IEEE Signal Process. Mag. 36(3), 118–123 (2019)

    Article  Google Scholar 

  9. Jiang, B., Zhang, Y., Wei, X., Xue, X., Fu, Y.: Learning compositional representation for 4D captures with neural ode. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5340–5350 (2021)

    Google Scholar 

  10. Kazhdan, M., Bolitho, M., Hoppe, H.: Poisson surface reconstruction. In: Eurographics Symposium on Geometry Processing, SGP 2006, pp. 61–70. Eurographics Association, Goslar (2006)

    Google Scholar 

  11. Leroy, V., Franco, J.S., Boyer, E.: Multi-view dynamic shape refinement using local temporal integration. In: IEEE International Conference on Computer Vision (ICCV) (2017)

    Google Scholar 

  12. Li, R., Li, X., Fu, C., Cohen-Or, D., Heng, P.: PU-GAN: a point cloud upsampling adversarial network. CoRR arxiv:1907.10844 (2019)

  13. Li, R., Li, X., Heng, P., Fu, C.: Point cloud upsampling via disentangled refinement. CoRR arxiv:2106.04779 (2021)

  14. Lipman, Y., Cohen-Or, D., Levin, D., Tal-Ezer, H.: Parameterization-free projection for geometry reconstruction. ACM Trans. Graph. 26(3), 22-es (2007)

    Google Scholar 

  15. Mao, X., Li, Q., Xie, H., Lau, R.Y.K., Wang, Z., Smolley, S.P.: Least squares generative adversarial networks. In: IEEE International Conference on Computer Vision (ICCV), pp. 2813–2821 (2016)

    Google Scholar 

  16. Mescheder, L., Oechsle, M., Niemeyer, M., Nowozin, S., Geiger, A.: Occupancy networks: learning 3D reconstruction in function space. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4460–4470 (2019)

    Google Scholar 

  17. Mustafa, A., Kim, H., Guillemaut, J.Y., Hilton, A.: Temporally coherent 4D reconstruction of complex dynamic scenes. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)

    Google Scholar 

  18. Niemeyer, M., Mescheder, L., Oechsle, M., Geiger, A.: Occupancy flow: 4D reconstruction by learning particle dynamics. In: IEEE/CVF International Conference on Computer Vision (ICCV) (2019)

    Google Scholar 

  19. Qi, C.R., Yi, L., Su, H., Guibas, L.J.: PointNet++: deep hierarchical feature learning on point sets in a metric space. Adv. Neural Inf. Process. Syst. (NeurIPS) 30, 1–10 (2017)

    Google Scholar 

  20. Tang, J., Xu, D., Jia, K., Zhang, L.: Learning parallel dense correspondence from spatio-temporal descriptors for efficient and robust 4D reconstruction. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6018–6027 (2021)

    Google Scholar 

  21. Veličković, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., Bengio, Y.: Graph attention networks. arXiv preprint arXiv:1710.10903 (2017)

  22. Vu, T.A., Nguyen, D.T., Hua, B.S., Pham, Q.H., Yeung, S.K.: Rfnet-4D: joint object reconstruction and flow estimation from 4D point clouds. In: European Conference on Computer Vision (ECCV), pp. 36–52. Springer, Heidelberg (2022). https://doi.org/10.1007/978-3-031-20050-2_3

  23. Wang, Y., Wu, S., Huang, H., Cohen-Or, D., Sorkine-Hornung, O.: Patch-based progressive 3D point set upsampling. CoRR arxiv:1811.11286 (2018)

  24. Wang, Y., Sun, Y., Liu, Z., Sarma, S.E., Bronstein, M.M., Solomon, J.M.: Dynamic graph CNN for learning on point clouds. ACM Trans. Graph. 38(5), 1–12 (2019)

    Article  Google Scholar 

  25. Wu, T., Pan, L., Zhang, J., Wang, T., Liu, Z., Lin, D.: Density-aware chamfer distance as a comprehensive metric for point cloud completion. arXiv preprint arXiv:2111.12702 (2021)

  26. Wu, Z., et al.: 3D shapenets: a deep representation for volumetric shapes. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015)

    Google Scholar 

  27. Yu, L., Li, X., Fu, C., Cohen-Or, D., Heng, P.: Pu-net: point cloud upsampling network. CoRR arxiv:1801.06761 (2018)

Download references

Acknowledgments

This work was supported by the European Commission under European Horizon 2020 Programme, grant number 951911-AI4Media.

Author information

Authors and Affiliations

  1. Media Integration and Communication Center (MICC), University of Florence, Florence, Italy

    Lorenzo Berlincioni, Stefano Berretti, Marco Bertini & Alberto Del Bimbo

Authors
  1. Lorenzo Berlincioni
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Stefano Berretti
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marco Bertini
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Alberto Del Bimbo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Stefano Berretti .

Editor information

Editors and Affiliations

  1. University of Udine, Udine, Italy

    Gian Luca Foresti

  2. University of Udine, Udine, Italy

    Andrea Fusiello

  3. University of York, York, UK

    Edwin Hancock

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Berlincioni, L., Berretti, S., Bertini, M., Del Bimbo, A. (2024). Upsampling 4D Point Clouds of Human Body via Adversarial Generation. In: Foresti, G.L., Fusiello, A., Hancock, E. (eds) Image Analysis and Processing - ICIAP 2023 Workshops. ICIAP 2023. Lecture Notes in Computer Science, vol 14365. Springer, Cham. https://doi.org/10.1007/978-3-031-51023-6_38

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-51023-6_38

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-51022-9

  • Online ISBN: 978-3-031-51023-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation