Skip to main content
SpringerLink
Log in

Volumetric stereo and silhouette fusion for image-based modeling

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

This paper presents a volumetric stereo and silhouette fusion algorithm for acquiring high quality models from multiple calibrated photographs. Our method is based on computing and merging depth maps. Different from previous methods of this category, the silhouette information is also applied in our algorithm to recover the shape information on the textureless and occluded areas. The proposed algorithm starts by computing visual hull using a volumetric method in which a novel projection test method is proposed for visual hull octree construction. Then, the depth map of each image is estimated by an expansion-based approach that returns a 3D point cloud with outliers and redundant information. After generating an oriented point cloud from stereo by rejecting outlier, reducing scale, and estimating surface normal for the depth maps, another oriented point cloud from silhouette is added by carving the visual hull octree structure using the point cloud from stereo to restore the textureless and occluded surfaces. Finally, Poisson Surface Reconstruction approach is applied to convert the oriented point cloud both from stereo and silhouette into a complete and accurate triangulated mesh model. The proposed approach has been implemented and the performance of the approach is demonstrated on several real data sets, along with qualitative comparisons with the state-of-the-art image-based modeling techniques according to the Middlebury benchmark.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Prados, E., Faugeras, O.: Perspective shape from shading and viscosity solutions. In: ICCV 2, pp. 826–831 (2003)

  2. Szeliski, R.: Rapid octree construction from range sequences. Comput. Vis. Graph. Image Process. 58(1), 23–32 (1993)

    Google Scholar 

  3. Yemez, Y., Schmitt, F.: 3D reconstruction of real objects with high resolution shape and texture. Image Vis. Comput. 22(13), 1137–1153 (2004)

    Article  Google Scholar 

  4. Kazhdan, M., Bolithp, M., Hoppe, H.: Poisson surface reconstruction. In: Eurographics Symposium on Geometry Processing, pp. 61–70 (2006)

  5. Seitz, S.M., Curless, B., Diebel, J., Scharstein, D., Szeliski, R.: A comparison and evaluation of multi-view stereo reconstruction algorithms. In: CVPR 1, pp. 519–526 (2006)

  6. Vogiatzis, G., Torr, P., Cipolla, R.: Multi-view stereo via volumetric graph-cuts. In: CVPR, pp. 391–398 (2005)

  7. Sinha, S.N., Mordohai, P., Pollefeys, M.: Multi-view stereo via graph cuts on the dual of an adaptive tetrahedral mesh. In: ICCV (2007)

  8. Vogiatzis, G., Hernandez, C., Torr, P.H.S., Cipolla, R.: Multi-view stereo via volumetric graph-cuts and occlusion robust photo-consistenc. IEEE Trans. Pattern Anal. Mach. Intell. 29(12), 2241–2246 (2007)

    Article  Google Scholar 

  9. Kutulakos, K., Seitz, S.M.: A theory of shape by space carving. Int. J. Comput. Vis. 38(3), 199–218 (2000)

    Article  MATH  Google Scholar 

  10. Hernandez, C., Schmitt, F.: Silhouette and stereo fusion for 3d object modeling. Comput. Vis. Image Underst. 96(3), 367–392 (2004)

    Article  Google Scholar 

  11. Pons, J., Keriven, R., Faugeras, O.: Modelling dynamic scenes by registering multi-view image sequences. In: CVPR, pp. 822–827 (2005)

  12. Furukawa, Y., Ponce, J.: Accurate, dense, and robust multi-view stereopsis. In: CVPR (2007)

  13. Habbecke, M., Kobbelt, L.: A surface-growing approach to multi-view stereo reconstruction. In: CVPR (2007)

  14. Goesele, M., Snavely, N., Curless, B., Hoppe, H., Seitz, S.M.: Multi-view stereo for community photo collections. In: ICCV (2007)

  15. Narayanan, P., Rander, P., Kanade, T.: Constructing virtual worlds using dense stereo. In: ICCV, pp. 3–10 (1998)

  16. Goesele, M., Curless, B., Seitz, S.M.: Multi-view stereo revisited. In: CVPR, pp. 2402–2409 (2006)

  17. Bradley, D., Boubekeur, T., Heidrich, W.: Accurate multi-view reconstruction using robust binocular stereo and surface meshing. In: CVPR (2008)

  18. Liu, Y., Cao, X., Dai, Q., Xu, W.: Continuous depth estimation for multi-view stereo. In: CVPR, pp. 2121–2128 (2009)

  19. Laurentini, A.: The visual hull concept for silhouette based image understanding. IEEE Trans. Pattern Anal. Mach. Intell. 16(2), 150–162 (1994)

    Article  Google Scholar 

  20. Matusik, W., Buehler, C., McMillan, L.: Polyhedral visual hulls for real-time rendering. In: Proc. 12th Eurographics Workshop on Rendering, pp. 115–125 (2001)

  21. Tarini, M., Callieri, M., Montani, C., Rocchini, C., Olsson, K., Persson, T.: Marching intersections: An efficient approach to shape-from-silhouette. In: Proceedings of VMV, pp. 283–290 (2002)

  22. Lorensen, W.E., Cline, H.E.: Marching cubes: A high resolution 3d surface construction algorithm. In: SIGGRAPH 21, pp. 163–169 (1987)

  23. Song, P., Wu, X., Wang, M.Y.: A robust and accurate method for visual hull computation. In: IEEE International Conference on Information and Automation, pp. 784–789 (2009)

  24. Potmesil, M.: Generating octree models of 3d objects from their silhouettes in a sequence of images. Comput. Vis. Graph. Image Process. 40, 1–29 (1987)

    Article  Google Scholar 

  25. Borgefors, G.: Distance transformations in digital images. Comput. Vis. Graph. Image Process. 34, 344–371 (1986)

    Article  Google Scholar 

  26. Jarvis, R.A.: On the identification of the convex hull of a finite set of points in the plane. Inf. Process. Lett. 2, 18–21 (1973)

    Article  MATH  Google Scholar 

  27. Hernandez, C., Schmitt, F.: Multi-stereo 3d object reconstruction. In: 3DPVT, pp. 159–166 (2002)

  28. Hoppe, H., DeRose, T., Duchamp, T., McDonald, J., Stuetzle, W.: Surface reconstruction from unorganized points. In: SIGGRAPH, pp. 71–78 (1992)

  29. Yemez, Y., Wetherilt, C.J.: A volumetric fusion technique for surface reconstruction from silhouette and range data. Comput. Vis. Image Underst. 105(1), 30–41 (2007)

    Article  Google Scholar 

  30. Poisson Surface Reconstruction package. http://www.cs.jhu.edu/misha/Code/PoissonRecon/

  31. BigHead sequence. http://www.tsi.enst.fr/3dmodels/

  32. 3D photography data sets. http://www-cvr.ai.uiuc.edu/yfurukaw/research/mview

  33. Dino and Temple data sets. http://vision.middlebury.edu/mview/

  34. Furukawa, Y., Ponce, J.: Carved visual hulls for imaged-based modeling. In: ECCV, vol. 1, pp. 564–577 (2006)

  35. Vu, H., Keriven, R., Labatut, P., Pons, J.P.: Towards high-resolution large-scale multi-view stereo. In: CVPR (2009)

  36. Lhuillier, M., Quan, L.: Match propagation for image-based modeling and rendering. IEEE Trans. Pattern. Anal. Mach. Intell. 24(8), 1140–1146 (2002)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Division of Control and Mechatronics Engineering, Harbin Institute of Technology Shenzhen Graduate School, Shenzhen, 518055, China

    Peng Song & Xiaojun Wu

  2. Department of Mechanical and Automation Engineering, Chinese University of Hong Kong, Shatin, N.T., Hong Kong, China

    Michael Yu Wang

Authors
  1. Peng Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaojun Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Michael Yu Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Peng Song.

Electronic Supplementary Material

Below is the link to the electronic supplementary material.

(WMV 10.832 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Song, P., Wu, X. & Wang, M.Y. Volumetric stereo and silhouette fusion for image-based modeling. Vis Comput 26, 1435–1450 (2010). https://doi.org/10.1007/s00371-010-0429-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-010-0429-y

Keywords

Search

Navigation