Skip to main content
SpringerLink
Log in

3D-ARNet: An accurate 3D point cloud reconstruction network from a single-image

  • 1177: Advances in Deep Learning for Multimodal Fusion and Alignment
  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Generating a more realistic 3D reconstruction point cloud is an ill-posed problem. It is a challenging task to infer 3D shape from a single image. In this paper, a two-stage training network that can reconstruct point cloud from a single image is proposed, namely, 3D-ARNet. The 3D-ARNet uses the designed image encoder with an attention mechanism to extract image features and output a simple point cloud. To improve the accuracy of point cloud reconstruction, the 3D-ARNet network contains a pre-trained point cloud auto-encoder, which a takes simple point cloud as input, and finally obtains an accurately reconstructed point cloud. The proposed approach is analyzed qualitatively and quantitatively on both synthetic and real-world datasets. Improvements are evidently demonstrated from experimental comparison results in reference to existing related state-of-the-art networks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Achlioptas P, Diamanti O, Mitliagkas I, Guibas LJ (2018) Learning representations and generative models for 3d point clouds. In ICML

  2. Chang AX, Funkhouser TA, Guibas LJ, Hanrahan P, Huang QX, Li Z, Savarese S, Savva M, Song S, Su H, Xiao J, Yi L, Yu F (2015) Shapenet: An information-rich 3d model repository. ArXiv, abs/1512.03012

  3. Choy CB, Xu D, Gwak J, Chen K, Savarese S (2016) 3d-r2n2: A unified approach for single and multi-view 3d object reconstruction. Lect Notes Comput Sci 628–644

  4. Choi S, Nguyen AD, Kim JW, Ahn S, Lee S (2019) Point cloud deformation for single image 3d reconstruction. 2019 IEEE International Conference on Image Processing (ICIP) 2379–2383

  5. Di X, Yu P (2017) 3d reconstruction of simple objects from a single view silhouette image 01

  6. Fan H, Su H, Guibas L (2017) A point set generation network for 3d object reconstruction from a single image. 2017 Proc IEEE Conf Comput Vis Pattern Recognit (CVPR)

  7. Fuentes-Pacheco J, Ruiz-Ascencio J, Rendon-Mancha JM (2015) Visual simultaneous localization and mapping: a survey. Artif Intell Rev 43(1):55–81

  8. Gadelha M, Maji S, Wang R (2017) 3d shape induction from 2d views of multiple objects. 2017 International Conference on 3D Vision (3DV)

  9. Girdhar R, Fouhey DF, Rodriguez M, Gupta A (2016) Learning a predictable and generative vector representation for objects. Lect Notes Comput Sci 484–499

  10. Haming K, Peters G (2010) The structure-from-motion reconstruction pipeline - a survey with focus on short image sequences. Kybernetika 5:01

    MathSciNet  MATH  Google Scholar 

  11. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks

  12. Ian J (2014) Goodfellow, Jean Pouget-Abadie, M. Aaron C. Courville, and Yoshua Bengio. Generative adversarial nets. In NIPS, Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair

    Google Scholar 

  13. Kingma DP, Ba J (2015) Adam: A method for stochastic optimization. CoRR, abs/1412.6980

  14. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In NIPS

  15. Kurenkov A, Ji J, Garg A, Mehta V, Gwak J, Choy C, Savarese S (2018) Deformnet: Free-form deformation network for 3d shape reconstruction from a single image. 2018 IEEE Winter Conference on Applications of Computer Vision (WACV)

  16. Lu Q, Xiao M, Lu Y, Yuan X, Yu Y (2019) Attention-based dense point cloud reconstruction from a single image. IEEE Access 7:137420–137431

    Article  Google Scholar 

  17. Mandikal P, Navaneet KL, Agarwal M, Babu RV (2018) 3d-lmnet: Latent embedding matching for accurate and diverse 3d point cloud reconstruction from a single image

  18. Navaneet KL, Mandikal P, Agarwal M, Babu RV (2019) Capnet: Continuous approximation projection for 3d point cloud reconstruction using 2d supervision. Proceedings of the AAAI Conference on Artificial Intelligence 33:8819–8826

  19. Qi CR, Su H, Mo K, Guibas LJ (2017) Pointnet: Deep learning on point sets for 3d classification and segmentation. 2017 Proc IEEE Conf Comput Vis Pattern Recognit (CVPR)

  20. Ramasinghe S, Khan S, Barnes N, Gould S (2019) Spectral-gans for high-resolution 3d point-cloud generation. ArXiv, abs/1912.01800

  21. Rubner Y, Tomasi C, Guibas LJ (2000) The earth mover’s distance as a metric for image retrieval. Int J Comput Vis 40(2):99–121

    Article  Google Scholar 

  22. Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. CoRR, abs/1409.1556

  23. Sun R, Gao Y, Fang Z, Wang A, Zhong C (2019) Ssl-net: Point-cloud generation network with self-supervised learning. IEEE Access 7:82206–82217

    Article  Google Scholar 

  24. Sun X, Wu J, Zhang X, Zhang Z, Zhang C, Xue T, Tenenbaum JB, Freeman WT (2018) Pix3d: Dataset and methods for single-image 3d shape modeling. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition 2974–2983

  25. Tulsiani S, Zhou T, Efros AA, Malik J (2017) Multi-view supervision for single-view reconstruction via differentiable ray consistency. In Proc IEEE Conf Comput Vis Pattern Recognit 2626–2634

  26. Wei Y, Liu S, Zhao W, Lu J, Zhou J (2019) Conditional single-view shape generation for multi-view stereo reconstruction. 2019 IEEE/CVF Conf Comput Vision Pattern Recognit (CVPR) 9643–9652

  27. Wu Z, Song S, Khosla A, Yu F, Zhang L, Tang X, Xiao J (2015) 3d shapenets: A deep representation for volumetric shapes. 2015 Proc IEEE Conf Comput Vis Pattern Recognit (CVPR)

  28. Yan X, Yang J, Yumer E, Guo Y, Lee H (2016) Perspective transformer nets: Learning single-view 3d object reconstruction without 3d supervision. ArXiv, abs/1612.00814

  29. Zhang W, Long C, Yan Q, Xiao C (2020) Multi-stage point completion network with critical set supervision

Download references

Acknowledgements

This work was conducted during the research year of Shanghai University of Electric Power in 2020 and this work is supported by National Natural Science Foundation of China (Grant No. 51705304), Natural Science Foundation of Shanghai (Grant No. 20ZR1421300).

Author information

Authors and Affiliations

  1. College of Automation Engineering, Shanghai University of Electric Power, Shanghai, 310027, China

    Hui Chen & Yipeng Zuo

Authors
  1. Hui Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yipeng Zuo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hui Chen.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, H., Zuo, Y. 3D-ARNet: An accurate 3D point cloud reconstruction network from a single-image. Multimed Tools Appl 81, 12127–12140 (2022). https://doi.org/10.1007/s11042-021-11433-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-11433-7

Keywords

Search

Navigation