Skip to main content
Springer Nature Link
Log in

Reconstructing 3D Contour Models of General Scenes from RGB-D Sequences

  • Conference paper
  • First Online:
MultiMedia Modeling (MMM 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13142))

Included in the following conference series:

Abstract

General 3D reconstruction methods use voxels, surfels, or meshes to represent the 3D model of a given scene. These surface-based methods are vulnerable to the loss of boundary details, which affects the completeness of the reconstructed model. In this paper, we focus on the boundary information of the scene and propose a novel method to reconstruct 3D models by using 3D contours extracted from input image sequences. We design a robust frame-to-model contour matching algorithm to solve the problem of finding many-to-many contour correspondences between different frames, and use contour-enhanced optimization to obtain more accurate camera poses. In order to make the reconstructed model more expressive of structural information, we propose a contour fusion algorithm that considers the connections between 3D contours. Compared with other methods which use straight lines or curve segments to reconstruct the scene model, our method can generate a more complete and regular 3D contour model with topological relationship. Experiments on several public datasets demonstrate the effectiveness of our method for both modeling and pose estimation.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    http://www.danielgm.net/cc/.

References

  1. Bignoli, A., Romanoni, A., Matteucci, M., di Milano, P.: Multi-view stereo 3D edge reconstruction. In: WACV, pp. 867–875 (2018)

    Google Scholar 

  2. Canny, J.: A computational approach to edge detection. IEEE TPAMI 8(6), 679–698 (1986)

    Article  Google Scholar 

  3. Dai, A., Nießner, M., Zollhöfer, M., Izadi, S., Theobalt, C.: BundleFusion: Real-time globally consistent 3D reconstruction using on-the-fly surface reintegration. ACM TOG 36(4), 1 (2017)

    Article  Google Scholar 

  4. Fabbri, R., Kimia, B.: 3D curve sketch: flexible curve-based stereo reconstruction and calibration. In: CVPR, pp. 1538–1545 (2010)

    Google Scholar 

  5. Grupp, M.: evo: Python package for the evaluation of odometry and SLAM (2017). https://github.com/MichaelGrupp/evo

  6. Handa, A., Whelan, T., McDonald, J., Davison, A.: A benchmark for RGB-D visual odometry, 3D reconstruction and SLAM. In: ICRA, pp. 1524–1531 (2014)

    Google Scholar 

  7. Hofer, M., Maurer, M., Bischof, H.: Improving sparse 3D models for man-made environments using line-based 3D reconstruction. In: 3DV, vol. 1, pp. 535–542 (2014)

    Google Scholar 

  8. Hofer, M., Maurer, M., Bischof, H.: Efficient 3D scene abstraction using line segments. Comput. Vis. Image Underst. 157, 167–178 (2017)

    Article  Google Scholar 

  9. Kim, C., Kim, P., Lee, S., Kim, H.J.: Edge-based robust RGB-D visual odometry using 2D edge divergence minimization. In: IROS, pp. 1–9 (2018)

    Google Scholar 

  10. Kümmerle, R., Grisetti, G., Strasdat, H., Konolige, K., Burgard, W.: g\(\hat{2}\)o: a general framework for graph optimization. In: ICRA, pp. 3607–3613 (2011)

    Google Scholar 

  11. Ladicky, L., Saurer, O., Jeong, S., Maninchedda, F., Pollefeys, M.: From point clouds to mesh using regression. In: ICCV, pp. 3893–3902 (2017)

    Google Scholar 

  12. Li, S., Yao, Y., Fang, T., Quan, L.: Reconstructing thin structures of manifold surfaces by integrating spatial curves. In: CVPR, pp. 2887–2896 (2018)

    Google Scholar 

  13. Liu, L., Chen, N., Ceylan, D., Theobalt, C., Wang, W., Mitra, N.: CurveFusion: reconstructing thin structures from RGB-D sequences. ACM TOG 37(6) (2018)

    Google Scholar 

  14. Liu, L., Ceylan, D., Lin, C., Wang, W., Mitra, N.J.: Image-based reconstruction of wire art. ACM TOG 36(4), 1–11 (2017)

    Article  Google Scholar 

  15. Martin, T., Montes, J., Bazin, J.C., Popa, T.: Topology-aware reconstruction of thin tubular structures. In: ACM SIGGRAPH, vol. 12 (2014)

    Google Scholar 

  16. Mur-Artal, R., Tardós, J.D.: ORB-SLAM2: an open-source SLAM system for monocular, stereo, and RGB-D cameras. IEEE TRO 33(5), 1255–1262 (2017)

    Google Scholar 

  17. Nurutdinova, I., Fitzgibbon, A.: Towards pointless structure from motion: 3D reconstruction and camera parameters from general 3D curves. In: ICCV, pp. 2363–2371 (2015)

    Google Scholar 

  18. Ramalingam, S., Antunes, M., Snow, D., Lee, G.H., Pillai, S.: Line-sweep: cross-ratio for wide-baseline matching and 3D reconstruction. In: CVPR, pp. 1238–1246 (2015)

    Google Scholar 

  19. Rao, D., Chung, S.J., Hutchinson, S.: CurveSLAM: an approach for vision-based navigation without point features. In: IROS, pp. 4198–4204 (2012)

    Google Scholar 

  20. Schenk, F., Fraundorfer, F.: RESLAM: a real-time robust edge-based SLAM system. In: ICRA, pp. 154–160 (2019)

    Google Scholar 

  21. Schöps, T., Sattler, T., Pollefeys, M.: Surfelmeshing: online surfel-based mesh reconstruction. IEEE TPAMI 42(10), 2494–2507 (2019)

    Article  Google Scholar 

  22. Sturm, J., Engelhard, N., Endres, F., Burgard, W., Cremers, D.: A benchmark for the evaluation of RGB-D slam systems. In: IROS, pp. 573–580 (2012)

    Google Scholar 

  23. Tabb, A.: Shape from silhouette probability maps: reconstruction of thin objects in the presence of silhouette extraction and calibration error. In: CVPR, pp. 161–168 (2013)

    Google Scholar 

  24. Takeyama, K.: 3D reconstruction of edge line by ICP-based matching with geometric constraints. In: 2020 Digital Image Computing: Techniques and Applications (DICTA), pp. 1–8 (2020)

    Google Scholar 

  25. Tarrio, J.J., Pedre, S.: Realtime edge-based visual odometry for a monocular camera. In: ICCV, pp. 702–710 (2015)

    Google Scholar 

  26. Usumezbas, A., Fabbri, R., Kimia, B.B.: From multiview image curves to 3D drawings. In: ECCV, pp. 70–87 (2016)

    Google Scholar 

  27. Wang, K., Gao, F., Shen, S.: Real-time scalable dense Surfel mapping. In: ICRA, pp. 6919–6925 (2019)

    Google Scholar 

  28. Wang, P., Liu, L., Chen, N., Chu, H.K., Theobalt, C., Wang, W.: Vid2Curve: simultaneous camera motion estimation and thin structure reconstruction from an RGB video. ACM TOG 39(4), 1–132 (2020)

    Article  Google Scholar 

  29. Whelan, T., Salas-Moreno, R.F., Glocker, B., Davison, A.J., Leutenegger, S.: ElasticFusion: real-time dense SLAM and light source estimation. IJRR 35(14), 1697–1716 (2016)

    Google Scholar 

  30. Zhou, T., Li, J., Wang, S., Tao, R., Shen, J.: Matnet: motion-attentive transition network for zero-shot video object segmentation. IEEE TIP 29, 8326–8338 (2020)

    Google Scholar 

  31. Zhou, T., Wang, S., Zhou, Y., Yao, Y., Li, J., Shao, L.: Motion-attentive transition for zero-shot video object segmentation. In: AAAI, pp. 13066–13073 (2020)

    Google Scholar 

  32. Zhou, T., Wang, W., Qi, S., Ling, H., Shen, J.: Cascaded human-object interaction recognition. In: CVPR, pp. 4263–4272 (2020)

    Google Scholar 

  33. Zhou, Y., Li, H., Kneip, L.: Canny-VO: visual odometry with RGB-D cameras based on geometric 3D–2D edge alignment. IEEE TRO 35(1), 184–199 (2018)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Beijing Laboratory of Intelligent Information Technology, School of Computer Science and Technology, Beijing Institute of Technology, Beijing, 100081, China

    Weiran Wang, Huijun Di & Lingxiao Song

Authors
  1. Weiran Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Huijun Di
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lingxiao Song
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Huijun Di .

Editor information

Editors and Affiliations

  1. IT University of Copenhagen, Copenhagen, Denmark

    Björn Þór Jónsson

  2. Dublin City University, Dublin, Ireland

    Cathal Gurrin

  3. University of Science, VNU-HCM, Ho Chi Minh City, Vietnam

    Minh-Triet Tran

  4. University of Bergen, Bergen, Norway

    Duc-Tien Dang-Nguyen

  5. National Tsing Hua University, Hsinchu, Taiwan

    Anita Min-Chun Hu

  6. Hanoi University of Science and Technology, Hanoi, Vietnam

    Binh Huynh Thi Thanh

  7. Median Technologies, Valbonne, France

    Benoit Huet

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wang, W., Di, H., Song, L. (2022). Reconstructing 3D Contour Models of General Scenes from RGB-D Sequences. In: Þór Jónsson, B., et al. MultiMedia Modeling. MMM 2022. Lecture Notes in Computer Science, vol 13142. Springer, Cham. https://doi.org/10.1007/978-3-030-98355-0_14

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-98355-0_14

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-98354-3

  • Online ISBN: 978-3-030-98355-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics