Hanfei Xu
ABSTRACT
3D reconstruction using Struct-from-Motion(SFM) and Multi-View Stereo(MVS), along with image-based rendering (IBR), has been applied to the synthesis of the virtual-viewpoint images. We have investigated the latest works, and we find that the underlying 3D reconstruction methods are quite efficient for buildings and regular structure. However, for scenes with weakly textured or discontinuous-depth regions, these methods do not work well. Such regions do not contain reliable or dense 3D information, and as a result, biases will occur in the procedure of synthesizing point clouds. In this paper, we propose a new method using the image pyramid in the depth-map merging based MVS framework. In this method, we use multi-scale image information to estimate depth. Our method can capture more 3D information in weakly textured or discontinuous-depth regions, and finally obtain the better visible effect in the virtual-viewpoint images.
- Seitz, S. M., Curless, B., Diebel, J., Scharstein, D., & Szeliski, R. (2006). A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms. Computer Vision and Pattern Recognition, 2006 IEEE Computer Society Conference on (Vol.1, pp.519--528). IEEE. Google Scholar
Digital Library
- Seitz, S. M., & Dyer, C. R. (1999). Photorealistic Scene Reconstruction by Voxel Coloring. Kluwer Academic Publishers. Google ScholarDigital Library
- Sinha, S. N., Mordohai, P., & Pollefeys, M. (2007). Multi-View Stereo via Graph Cuts on the Dual of an Adaptive Tetrahedral Mesh. IEEE, International Conference on Computer Vision (pp.1--8). IEEE.Google Scholar
- Faugeras, O., & Keriven, R. (1998). Variational principles, surface evolution, pde's, level set methods, and the stereo problem. IEEE Transactions on Image Processing, 7(3), 336--344. Google ScholarDigital Library
- Esteban, C. H., & Schmitt, F. (2004). Silhouette and stereo fusion for 3d object modeling. Computer Vision & Image Understanding, 96(3), 367--392. Google ScholarDigital Library
- Hiep, V. H., Keriven, R., Labatut, P., & Pons, J. P. (2009). Towards high-resolution large-scale multi-view stereo. Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on (pp.1430--1437). IEEE.Google ScholarCross Ref
- Cremers, D., & Kolev, K. (2011). Multiview stereo and silhouette consistency via convex functionals over convex domains. IEEE Transactions on Pattern Analysis & Machine Intelligence, 33(6), 1161. Google ScholarDigital Library
- Lhuillier, M., & Quan, L. (2005). A Quasi-Dense Approach to Surface Reconstruction from Uncalibrated Images. IEEE Computer Society. Google ScholarDigital Library
- Goesele, M., Snavely, N., Curless, B., Hoppe, H., & Seitz, S. M. (2007). Multi-View Stereo for Community Photo Collections. IEEE, International Conference on Computer Vision (pp.1--8). IEEE.Google Scholar
- Furukawa, Y., & Ponce, J. (2010). Accurate, dense, and robust multiview stereopsis. IEEE Transactions on Pattern Analysis & Machine Intelligence, 32(8), 1362--1376. Google ScholarDigital Library
- Wu, T. P., Yeung, S. K., Jia, J., & Tang, C. K. (2010). Quasi-dense 3D reconstruction using tensor-based multiview stereo. Computer Vision and Pattern Recognition (Vol.119, pp.1482--1489). IEEE.Google Scholar
- Furukawa, Y., Curless, B., Seitz, S. M., & Szeliski, R. (2010). Towards Internet-scale multi-view stereo. Computer Vision and Pattern Recognition (Vol.2010, pp.1434--1441). IEEE.Google ScholarCross Ref
- Goesele, M., Curless, B., & Seitz, S. M. (2006). Multi-View Stereo Revisited. Computer Vision and Pattern Recognition, 2006 IEEE Computer Society Conference on (Vol.2, pp.2402--2409). IEEE. Google ScholarDigital Library
- Strecha, C., Fransens, R., & Gool, L. V. (2006). Combined Depth and Outlier Estimation in Multi-View Stereo. IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Vol.2, pp.2394--2401). IEEE Computer Society. Google ScholarDigital Library
- Merrell, P., Akbarzadeh, A., Wang, L., Mordohai, P., Frahm, J. M., & Yang, R., et al. (2007). Real-time visibility-based fusion of depth maps. Proc.int.conf.on Computer Vision (August), 1--8.Google ScholarCross Ref
- Zach, C., Pock, T., & Bischof, H. (2007). A Globally Optimal Algorithm for Robust TV-L 1, Range Image Integration. IEEE, International Conference on Computer Vision (pp.1--8). IEEE.Google Scholar
- Bradley, D., Boubekeur, T., & Heidrich, W. (2008). Accurate multi-view reconstruction using robust binocular stereo and surface meshing. Computer Vision and Pattern Recognition, 2008. CVPR 2008. IEEE Conference on (pp.1--8). IEEE.Google ScholarCross Ref
- Campbell, N. D., Vogiatzis, G., & Cipolla, R. (2008). Using Multiple Hypotheses to Improve Depth-Maps for Multi-View Stereo. European Conference on Computer Vision (Vol.5302, pp.766--779). Springer-Verlag. Google ScholarDigital Library
- Liu, Y., Cao, X., Dai, Q., & Xu, W. (2009). Continuous depth estimation for multi-view stereo. Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on (pp.2121--2128). IEEE.Google ScholarCross Ref
- Li, J., Li, E., Chen, Y., & Xu, L. (2010). Bundled depth-map merging for multi-view stereo. Computer Vision and Pattern Recognition (Vol.26, pp.2769--2776). IEEE.Google ScholarCross Ref
- Tola, E., Strecha, C., & Fua, P. (2012). Efficient large-scale multi-view stereo for ultra high-resolution image sets. Machine Vision & Applications, 23(5), 903--920. Google ScholarDigital Library
- Tola, E., Lepetit, V., & Fua, P. (2010). Daisy: an efficient dense descriptor applied to wide-baseline stereo. IEEE Trans Pami, 32(5), 815--30. Google ScholarDigital Library
- Schönberger, J. L., Zheng, E., Frahm, J. M., & Pollefeys, M. (2016). Pixelwise View Selection for Unstructured Multi-View Stereo. Computer Vision - ECCV 2016. Springer International Publishing.Google Scholar
- Chaurasia, G., Duchene, S., Sorkine-Hornung, O., & Drettakis, G. (2013). Depth synthesis and local warps for plausible image-based navigation. Acm Transactions on Graphics, 32(3), 1--12. Google ScholarDigital Library
Index Terms
- Depth Estimation in Multi-View Stereo Based on Image Pyramid
Recommendations
A DAISY descriptor based multi-view stereo method for large-scale scenes
Display Omitted An improved patch based MVS method by a novel photometric discrepancy function.Proposed a new corresponding point matching method based on the DAISY descriptor.Proposed a new photometric discrepancy function based on DAISY descriptor. ...
Adaptive depth estimation for pyramid multi-view stereo
Highlights- The iterative adaptive depth estimation for learning-based MVS to avoid excessive computation on well-estimated regions during each refinement stage.
Graphical abstractIn this paper, we propose a Multi-View Stereo (MVS) network which can perform high-quality depth estimation with low memory consumption. The proposed MVS network is constructed based on the pyramid architecture to gradually ...
AbstractIn this paper, we propose a Multi-View Stereo (MVS) network which can perform efficient high-resolution depth estimation with low memory consumption. Classical learning-based MVS approaches typically construct 3D cost volumes to ...
PM-MVS: PatchMatch multi-view stereo
AbstractPatchMatch Stereo is a method for generating a depth map from stereo images by repeating spatial propagation and view propagation. The concept of PatchMatch Stereo can be easily extended to Multi-View Stereo (MVS). In this paper, we present ...
Comments