Skip to main content
SpringerLink
Log in

Generation of Temporally Consistent Multiple Virtual Camera Views from Stereoscopic Image Sequences

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

The emergence of a new generation of 3D auto stereoscopic displays is driving the requirement for multi-baseline images. The dominant form of this display technology requires multiple views of the same scene, captured at a single instance in time along a common baseline in order to project stereoscopic images to the viewer. The direct acquisition of multiple views (typically 8 or 16 for the current generation of such displays) is problematic due to the difficulty of configuring, calibrating and controlling multiple cameras simultaneously.

This paper describes a technique that alleviates these problems by generating the required views from binocular images. Considering each stereo pair in isolation leads to inconsistency across image sequences. By incorporating a motion-tracking algorithm this problem is significantly reduced. In this paper we describe a novel approach to stereo matching of image sequences for the purpose of generating multiple virtual cameraviews. Results of extensive tests on stereo image sequences will be documented indicating that this approach is promising both in terms of the speed of execution and the quality of the results produced.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  • Alvarez, L., Deriche, R., Sanchez J., and Weickert, J. 2000. Dense disparity map estimation respecting image discontinuities: A PDE and scale-space based approach. INRIA, Technical Report-No. 3874.

  • Barron, J.L. and Eagleson, R. 1995. Binocular estimation of motion and structure from long sequences using optical flow without correspondence. In Proceedings of International on Image Processing, vol. II, pp. 193–196.

    Google Scholar 

  • Beall, A., Loomis, J., Philbeck, J., and Fikes, T. 1995. Absolute motion parallax weakly determines visual scale in real and virtual environments. In Proceedings of The International Society for Optical Engineering, vol. 2411, pp. 288–297.

    Google Scholar 

  • Beier, T. and Neely, S. 1992. Feature based image metamorphingosis. Computer Graphics (SIGGRAPH), 26:35–42.

    Google Scholar 

  • Black, M. and Jepson, A. 1996. Estimating optical flowin segmented images using variable-order parametric models with local deformations. IEEE Trans. Pattern Analysis and Machine Intelligence, 18(10):972–986.

    Google Scholar 

  • Cai, Q. and Aggarwal, J. 1999. Tracking human motion in structured environments using a distributed-camera system. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21(11):1241–1247.

    Google Scholar 

  • Charbonnier, P., Blance-Feraud, L., Aubert, G., and Barlaud, M. 1994. Two deterministic half-quadratic regularization algorithms for computed imaging. In Proceedings of International Conference of Image Processing, vol. 2, pp. 168-172.

    Google Scholar 

  • 4D-Vision Gmbh. http://www.4d-vision.de/presse.htm. Retrieved on 02 November 2001.

  • Delagnes, P., Benois, J., and Barba, D. 1995. Active contours approach to object tracking in image sequences with complex background. Pattern Recognition Letters, 15:171–178.

    Google Scholar 

  • Dynamic Digital Depth. http://www.ddd.com/, Retrieved on 05 November 2001.

  • Falkenhagen, L. 1995. 3D object-based depth estimation from stereo-scopic image sequences. In Proceedings of International Workshop on Stereoscopic and Three Dimensional Imaging, Sept., Greece.

  • Geman, D. and Reynolds, G. 1992. Constrained restoration and the recovery of discontinuities. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14(3):367–383.

    Google Scholar 

  • Gil, S., Milanese, R., and Pun, T. 1996. Combining multiple motion estimates for vehicle tracking. In Proceedings of the 4th European Conference Computer Vision, vol. II, Springer-Verlag, pp. 307–320, April 1996.

    Google Scholar 

  • Huang, H., Kao, C., Lin, Y., and Hung, Y. 2000. Disparity-based view interpolation for multiple perspective stereoscopic displays. SPIE, vol. 3957, pp. 102–113.

    Google Scholar 

  • Jelinek, D. and Taylor, C. 2001. Quasidense motion stereo for 3D view morphing. In Proceedings International Symposium on Virtual and Augmented Architecture, June 2001, pp. 219–229.

  • Jenkin, M. and Jepson, A. 1994. Recovering local surface structure through local phase difference measurements. CVGIP: Image Understanding, 59(1):72–93.

    Google Scholar 

  • Jones, G. 1997. Constraint, optimization and hierarchy: Reviewing stereoscopic correspondence of complex features. Computer Vision and Image Understanding, 65(1):57–78.

    Google Scholar 

  • Kang, S.B. 1999. A survey of image-based rendering techniques. SPIE, vol. 3641, pp. 2–16.

    Google Scholar 

  • Koch, R. 1994. Model-based 3-D scene analysis from stereoscopic image sequences. ISPRS Journal of Photogrammetry and Remote Sensing, 49(5):23–30.

    Google Scholar 

  • Koch, R. 1995. 3-D surface reconstruction from stereoscopic image sequences. In Proceedings of International Conference of Computer Vision, Cambridge, MA, USA, June 1995.

  • Koller, D., Daniilidis, K., and Nagel, H.-H. 1993. Model-based object tracking in monocular image sequences of road traffic scenes. International Journal of Computer Vision, 10(3):257–281.

    Google Scholar 

  • La Cascia, M., Sclaroff, S., and Athitsos, V. 2000. Fast, reliable head tracking under varying illumination: An approach based on robust registration of texture-mapped 3D models. IEEE Trans. Pattern Analysis and Machine Intelligence, 22(4):322–336.

    Google Scholar 

  • Landy, M. and Movshon, J. (Eds.). 1991. The plenoptic function and the elements of early vision. Computational Models of Visual Processing. MIT Press: Cambridge, MA.

  • Matthies, L., Szeliski, R., and Kanade, T. 1989. Kalman filtering-based algorithms for estimating depth from image sequences. International Journal of Computer Vision, 3(3):209–238.

    Google Scholar 

  • Moravec, K., Harvey, R., and Bangham, J. 1998. Improving stereo performance in regions of low texture. In Proceedings of British Machine Vision Conference.

  • Nagel, H.-H. and Enkelmann, W. 1986. An investigation of smoothness constraints for the estimation of displacement vector fields from image sequences. IEEE Trans. Pattern Analysis and Machine Intelligence, 8(5):565–593.

    Google Scholar 

  • NHK R&D. http://www.strl.nhk.or.jp/publica/rd/rd38.html. No. 38, November 1995. NHK R&D Report. Retrieved on 01 Aug. 2001.

  • Niem, W. and Buschmann, R. 1994. Automatic modeling of 3D natural objects from multiple views. In Proceedings of Image Processing for Broadcast and Video Production, Y. Paker and S. Wilbur (Eds.), Springer: Berlin.

    Google Scholar 

  • Perona, P. and Malik, J. 1990. Scale space and edge detection using anisotropic diffusion. IEEE Trans. Pattern Analysis and Machine Intelligence, 12(7):629–639.

    Google Scholar 

  • Rogers, B. and Graham, M. 1979. Motion parallax as an independent cue for depth perception. Perception, 8:125–134.

    Google Scholar 

  • Rudin, L., Osher, S., and Fatemi, E. 1992. Nonlinear total variation based noise removal algorithms. Physica D, 60:259–268.

    Google Scholar 

  • Saito, H., Kawamura, H., and Nakajima, M. 1995. 3Dshape measurement of underwater objects using motion stereo. In Proceedings of 21st International Conference on Industrial Electronics, Control, and Instrumentation, Nov. 1995, pp. 1231–1235.

  • Seitz, S. and Dyer, C. 1996. View morphing. In Proceedings of SIGGRAPH, pp. 21–30.

  • Shao, J. and Fraser, C. 1998. Network vision. In Proceedings of 14th International Conference on Pattern Recognition, pp. 471–475.

  • Tomasi, C. and Kanade, T. 1991. Detection and tracking of point features. Technical Report CMU-CS-91-132, The Robotics Institute, Carnegie Mellon University, Pittsburgh.

    Google Scholar 

  • Vedula, S. 1999. Multi-view spatial and temporal interpolation for dynamic event visualization. Technical Report CMU-RI-99-15, The Robotics Institute, Carnegie Mellon University, Pittsburgh.

    Google Scholar 

  • Veksler, O. 2001. Stereo matching by compact windows via mini-mumratio cycle. In Proceedings of IEEE International Conference on Computer Vision, July 2001, vol. I, pp. 540–547.

    Google Scholar 

  • Waxman, A.M. and Duncan, J.H. 1986. Binocular image flows: Steps toward stereo-motion fusion. IEEE Trans. Pattern Analysis and Machine Intelligence, 8(6):715–729.

    Google Scholar 

  • Waxman, A.M. and Sinha, S.S. 1986. Dynamic stereo: Passive ranging to moving objects from relative image flows. IEEE Trans. Pattern Analysis and Machine Intelligence, 8(4):406–412.

    Google Scholar 

  • Weik, S. and Niemeyer, O. 1999. 3Dmotion estimation for articulated human templates using a sequence of stereoscopic image pairs. In Proceedings of Visual Communications and Image Processing, SPIE, vol. 3653.

  • Zhang, Z. and Faugeras, O. 1992. 3D dynamic scene analysis: A stereo based approach. Springer: Berlin.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Dynamic Digital Depth Research, P.O. Box 1169, Bentley, WA, 6983, Australia

    Juliang Shao

Authors
  1. Juliang Shao
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Shao, J. Generation of Temporally Consistent Multiple Virtual Camera Views from Stereoscopic Image Sequences. International Journal of Computer Vision 47, 171–180 (2002). https://doi.org/10.1023/A:1014545908590

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1014545908590

Search

Navigation