Skip to main content
SpringerLink
Log in

WEmap: Weakness-Enhancement Mapping for 3D Reconstruction with Sparse Image Sequences

  • Conference paper
  • First Online:
Advances in Computer Graphics (CGI 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13443))

Included in the following conference series:

  • 1099 Accesses

Abstract

Previous studies assume that a dense image sequence can be used for 3D reconstruction because the images are easily captured by mobile devices. However, mobile devices are not applicable in some cases, such as smart factories, which require real-time monitoring and site safety. Therefore, conducting 3D reconstruction with sparse image sequences is important to reduce the number of used devices, and thus, lower the cost of image acquisition. In this study, we propose weakness-enhancement mapping (WEmap) to improve the results of 3D reconstruction based on sparse image sequences. After the initial reconstruction, the contribution of each image is evaluated by mapping the 3D point cloud to 2D images. The low-contribution images and corresponding matching images are weighted to enhance the weaknesses of the initial reconstruction. To the best of our knowledge, this is the first study on 3D reconstruction with sparse image sequences. Experimental results on the sparse DTU [1] and sparse Tanks & Temples [3] datasets demonstrate that WEmap can effectively enhance a reconstructed structure.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Aans, H., Jensen, R., Vogiatzis, G., Tola, E., Dahl, A.: Large-scale data for multiple-view stereopsis. Int. J. Comput. Vis. (IJCV) 120(2), 153–168 (2016)

    Google Scholar 

  2. Ali, S.G., et al.: Cost-effective broad learning-based ultrasound biomicroscopy with 3D reconstruction for ocular anterior segmentation. Multimedia Tools Appl. 80(28), 35105–35122 (2021)

    Google Scholar 

  3. Angelova, A., Long, P.M.: Benchmarking large-scale fine-grained categorization. In: IEEE Winter Conference on Applications of Computer Vision, pp. 532–539 (2014)

    Google Scholar 

  4. Cheema, M.N., Nazir, A., Sheng, B., Li, P., Qin, J., Feng, D.D.: Liver extraction using residual convolution neural networks from low-dose CT images. IEEE Trans. Biomed. Eng. 66(9), 2641–2650 (2019)

    Google Scholar 

  5. Cheema, M.N., et al.: Image-aligned dynamic liver reconstruction using intra-operative field of views for minimal invasive surgery. IEEE Trans. Biomed. Eng. 66(8), 2163–2173 (2018)

    Google Scholar 

  6. Cheng, S., et al.: Deep stereo using adaptive thin volume representation with uncertainty awareness. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020

    Google Scholar 

  7. DeGol, J., Bretl, T., Hoiem, D.: Improved structure from motion using fiducial marker matching. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11207, pp. 281–296. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01219-9_17

    Chapter  Google Scholar 

  8. Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)

    Google Scholar 

  9. Ahmad Fuad, N., Yusoff, A.R., Ismail, Z., Majid, Z.: Comparing the performance of point cloud registration methods for landslide monitoring using mobile laser scanning data. ISPRS Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 4249, 11–21 (2018)

    Google Scholar 

  10. Furukawa, Y., Ponce, J.: Accurate, dense, and robust multiview stereopsis. IEEE Trans. Pattern Anal. Mach. Intell. 32(8), 1362–1376 (2010)

    Article  Google Scholar 

  11. Girardeau-Montaut, D.: Cloudcompare. http://www.cloudcompare.org

  12. Gu, X., Fan, Z., Zhu, S., Dai, Z., Tan, F., Tan, P.: Cascade cost volume for high-resolution multi-view stereo and stereo matching. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020

    Google Scholar 

  13. Guo, H., Sheng, B., Li, P., Philip Chen, C.L.: Multiview high dynamic range image synthesis using fuzzy broad learning system. IEEE Trans. Cybern. 51(5), 2735–2747 (2019)

    Google Scholar 

  14. Liu, F., Tran, L., Liu, X.: Fully understanding generic objects: Modeling, segmentation, and reconstruction. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7423–7433, June 2021

    Google Scholar 

  15. Locher, A., Havlena, M., Van Gool, L.: Progressive structure from motion. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11208, pp. 22–38. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01225-0_2

    Chapter  Google Scholar 

  16. Moulon, P., Monasse, P., Perrot, R., Marlet, R.: OpenMVG: open multiple view geometry. In: International Workshop on Reproducible Research in Pattern Recognition (2017)

    Google Scholar 

  17. Muthukrishnan, S., Ramakrishnan, S., Sanjayan, J.: Technologies for improving buildability in 3D concrete printing. Cem. Concr. Compos., 104144 (2021)

    Google Scholar 

  18. Schonberger, J.L., Frahm, J.-M.: Structure-from-motion revisited. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016

    Google Scholar 

  19. Shen, Y., Lindenbergh, R., Wang, J.: Change analysis in structural laser scanning point clouds: the baseline method. Sensors 17(1), 26 (2017)

    Google Scholar 

  20. Sheng, B., Li, P., Fang, X., Tan, P., Enhua, W.: Depth-aware motion deblurring using loopy belief propagation. IEEE Trans. Circuits Syst. Video Technol. 30(4), 955–969 (2019)

    Article  Google Scholar 

  21. Snavely, N., Seitz, S.M., Szeliski, R.: Modeling the world from internet photo collections. Int. J. Comput. Vis. 80(2), 189–210 (2008)

    Article  Google Scholar 

  22. Song, S., Chandraker, M.: Robust scale estimation in real-time monocular SFM for autonomous driving. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2014

    Google Scholar 

  23. Tian, X., Liu, R., Wang, Z., Ma, J.: High quality 3D reconstruction based on fusion of polarization imaging and binocular stereo vision. Inf. Fus. 77, 19–28 (2022)

    Article  Google Scholar 

  24. Yao, Y., Luo, Z., Li, S., Fang, T., Quan, L.: MVSNet: depth inference for unstructured multi-view stereo. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11212, pp. 785–801. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01237-3_47

    Chapter  Google Scholar 

  25. Yu, Z., Gao, S.: Fast-MVSNet: sparse-to-dense multi-view stereo with learned propagation and gauss-newton refinement. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020

    Google Scholar 

Download references

Acknowledgments

This work was supported in part by the Natural Science Foundation of Tianjin of China under Grant No. 21JCZDJC00740.

Author information

Authors and Affiliations

  1. College of Computer Science, Nankai University, Tianjin, China

    Kun Zhang, Chunying Song, Jingzhao Wang & Kai Wang

  2. Tianjin Zero Carbon Co., Ltd., Tianjin, China

    Nan Yun

Authors
  1. Kun Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chunying Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jingzhao Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kai Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Nan Yun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kai Wang .

Editor information

Editors and Affiliations

  1. University of Geneva, Geneva, Switzerland

    Nadia Magnenat-Thalmann

  2. Bournemouth University, Poole, UK

    Jian Zhang

  3. University of Sydney, Sydney, NSW, Australia

    Jinman Kim

  4. University of Crete, Heraklion, Greece

    George Papagiannakis

  5. Shanghai Jiao Tong University, Shanghai, China

    Bin Sheng

  6. Swiss Federal Institute of Technology, Lausanne, Switzerland

    Daniel Thalmann

  7. University of Calgary, Calgary, AB, Canada

    Marina Gavrilova

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhang, K., Song, C., Wang, J., Wang, K., Yun, N. (2022). WEmap: Weakness-Enhancement Mapping for 3D Reconstruction with Sparse Image Sequences. In: Magnenat-Thalmann, N., et al. Advances in Computer Graphics. CGI 2022. Lecture Notes in Computer Science, vol 13443. Springer, Cham. https://doi.org/10.1007/978-3-031-23473-6_15

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-23473-6_15

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-23472-9

  • Online ISBN: 978-3-031-23473-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation