Abstract
Given a set of images acquired from known viewpoints, we describe a method for synthesizing the image which would be seen from a new viewpoint. In contrast to existing techniques, which explicitly reconstruct the 3D geometry of the scene, we transform the problem to the reconstruction of colour rather than depth. This retains the benefits of geometric constraints, but projects out the ambiguities in depth estimation which occur in textureless regions.
On the other hand, regularization is still needed in order to generate high-quality images. The paper’s second contribution is to constrain the generated views to lie in the space of images whose texture statistics are those of the input images. This amounts to an image-based prior on the reconstruction which regularizes the solution, yielding realistic synthetic views. Examples are given of new view generation for cameras interpolated between the acquisition viewpoints—which enables synthetic steadicam stabilization of a sequence with a high level of realism.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles and news from researchers in related subjects, suggested using machine learning.References
2d3 Ltd. http://www.2d3.com, 2002.
Besag, J. 1986. On the statistical analysis of dirty pictures. J. Royal Stat. Soc. B, 48(3):259–302.
Broadhurst, A. and Cipolla, R. 2001. A statistical consistency check for the space carving algorithm. In Proc. ICCV.
Debevec, P.E., Taylor, C.J., and Malik, J. 1996. Modeling and rendering architecture from photographs: A hybrid geometry- and image-based approach. In Proceedings, ACM SIGGRAPH, pp. 11–20.
Efros, A. and Leung, T. 1999. Texture synthesis by non-parametric sampling. In Proc. ICCV, pp. 1039–1046.
Freeman, W. and Pasztor, E. 1999. Learning low-level vision. In ICCV, pp. 1182–1189.
Gortler, S.J., Grzeszczuk, R., Szeliski, R., and Cohen, M.F. 1996. The lumigraph. In SIG-GRAPH96.
Grenander, U. and Srivastava, A. 2001. Probability models for clutter in natural images. IEEE PAMI, 23(4):424–429.
Huang, J. and Mumford, D. 1999. Statistics of natural images and models. In Proc. CVPR, pp. 1541–1547.
Irani, M., Hassner, T., and Anandan, P. 2002. What does the scene look like from a scene point? In Proc. ECCV.
Koch, R. 1995. 3D surface reconstruction from stereoscopic image sequences. In Proc. ICCV, pp. 109–114.
Koch, R., Heigl, B., and Pollefeys, Marc 2001. Image-based rendering from uncalibrated light-fields with scalable geometry. In G. Gimel’farb (Eds.) R. Klette, T. Huang, {Multi-Image Analysis, Springer LNCS 2032}, pp. 51–66.
Kutulakos, K. and Seitz, S. 1999. A theory of shape by space carving. In Proc. ICCV, pp. 307–314.
Levoy, M. and Hanrahan, P. 1996. Light field rendering. In SIGGRAPH96.
Matusik, W., Buehler, C., Raskar, R., McMillan, L., and Gortler, S. 2000. Image-based visual hulls. In Proc. ACMSIGGRAPH, pp. 369–374.
Matusik, W., Pfister, H., Beardsley, P.A., Ngan, A., Ziegler, R., and McMillan, L. 2002. Image-based 3d photography using opacity hulls. In Proc. ACMSIGGRAPH.
McMillan, L. and Bishop, G. 1995. Plenoptic modeling: An image-based rendering system. In SIGGRAPH95.
Rademacher, P. 1999. View-dependent geometry. In Proc. ACM. SIGGRAPH pp. 439–446.
Scharstein, D. 1999. View Synthesis Using Stereo Vision, Vol. 1583 of LNCS. Springer-Verlag.
Scharstein, D. and Szeliski, R. 2002. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. IJCV, 47(l):7–42.
Seitz, S.M. and Dyer, C.R. 1997. Photorealistic scene reconstruction by voxel coloring. In Proc. CVPR, pp. 1067–1073.
Sminchisescu, C. and Triggs, B. 2002. Building roadmaps of local minima of visual models. In Proc. ECCV, Vol. 1, pp. 566–582.
Srivastava, A., Lee, A., Simoncelli, E., and Zhu, S. 2003. On advances in statistical modeling of natural images, 18(l):17–33.
Szeliski, R. and Golland, P. 1998. Stereo matching with transparency and matting. In Proc. ICCV, pp. 517–524.
Wei, L.-Y. and Levoy, M. 2000. Fast texture synthesis using tree-structured vector quantization. In Proc. ACM SIGGRAPH, pp. 479–488.
Wexler, Y. and Chellappa, R. 2001. View synthesis using convex and visual hulls. In Proc. BMVC.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Fitzgibbon, A., Wexler, Y. & Zisserman, A. Image-Based Rendering Using Image-Based Priors. Int J Comput Vision 63, 141–151 (2005). https://doi.org/10.1007/s11263-005-6643-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11263-005-6643-9