Skip to main content
SpringerLink
Log in

An efficient approach for 2D to 3D video conversion based on structure from motion

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

With the popularity of 3D films, the conversion of existing 2D videos to 3D videos has attracted a wide interest in 3D content production. In this paper, we present an efficient approach for 2D to 3D video conversion based on structure from motion (SFM). The key contributions include a piece-wise SFM approach and a novel nonlinear depth warping considering the characteristics of stereoscopic 3D. The dense depth maps are generated and further refined with color segmentation. Experiments show that the proposed approach can yield more visually satisfactory results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Knorr, S., Smolic, A., Sikora, T.: From 2D-to stereo-to multi-view video. In: Proceedings of 3DTV Conference, Kos Island, Greece, pp. 1–4 (2007)

  2. Rotem, E., Wolowelsky, K., Pelz, D.: Automatic video to stereoscopic video conversion. In: Proceedings of SPIE Conference on Stereoscopic Displays and Virtual Reality Systems, vol. 5664, pp. 198–206 (2005)

  3. Zhang, G., Hua, W., Qin, X., Wong, T., Bao, H.: Stereoscopic video synthesis from a monocular video. IEEE Trans. Vis. Comput. Graph. 13(4), 686–696 (2007)

    Article  Google Scholar 

  4. Fehn, C.: Depth-image-based rendering (DIBR), compression and transmission for a new approach on 3D-TV. In: Proceedings of SPIE Conference on Stereoscopic Displays and Virtual Reality Systems, vol. 5291, pp. 93–104 (2004)

  5. Zhang, L., Tam, W.J.: Stereoscopic image generation based on depth images for 3D TV. IEEE Trans. Broadcast. 51(2), 191–199 (2005)

    Article  Google Scholar 

  6. Schnyder, L., Lang, M., Wang, O., Smolic, A.: Depth image based compositing for stereo 3D. In: Proceedings of 3DTV Conference, pp. 1–4 (2012)

  7. Li, P., Farin, D., Gunnewiek, R.K., With, P.: On creating depth maps from monoscopic video using structure from motion. In: Proceedings of Workshop on Content Generation and Coding for 3DTV, pp. 508–515 (2006)

  8. Liao, M., Gao, J., Yang, R., Gong, M.: Video stereolization: combining motion analysis with user interaction. IEEE Trans. Vis. Comput. Graph. 18(7), 1079–1088 (2012)

    Article  Google Scholar 

  9. Moshe, G., Lior, W., Daniel, C.: Semi-automatic stereo extraction from video footage. In: Proceedings of IEEE International Conference on Computer Vision, pp. 136–142 (2009)

  10. Karsch, K., Liu, C., Kang, S.: Depth extraction from video using non-parametric sampling. In: Proceedings of European Conference on Computer Vision, pp. 775–788 (2012)

  11. Li, T., Dai, Q., Xie, X.: An efficient method for automatic stereoscopic conversion. In: 5th International Conference on Visual Information Engineering, pp. 256–260 (2008)

  12. Nam, S., Kim, H., Ban, Y., Chien, S.: Real-time 2D to 3D conversion for 3DTV using time coherent depth map generation method. In: Proceedings of IEEE International Conference on Consumer Electronics, pp. 187–188 (2013)

  13. Feng, Y., Ren, J., Jiang, J.: Object-based 2D-to-3D video conversion for effective stereoscopic content generation in 3D-TV application. IEEE Trans. Broadcast. 57(2), 500–509 (2011)

    Article  Google Scholar 

  14. Tsai, Y., Chang, Y., Chen, L.: Block-based vanishing line and vanishing point detection for 3D scene reconstruction. In: International Symposium on Intelligent Signal Processing and Communications, pp. 586–589 (2006)

  15. Zheng, F., Yuan, Z.: Depth estimation from single image based on vanishing point. J. Info. Tech. Appl. 1(3), 229–235 (2006)

    MathSciNet  Google Scholar 

  16. Guo, G., Zhang, N., Huo, L., Gao, W.: 2D to 3D conversion based on edge defocus and segmentation. In: Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 2181–2184 (2008)

  17. Kim, C., Park, J., Lee, J., Hwang, J.N.: Fast extraction of objects of interest from images with low depth of field. ETRI J. 29(3), 353–362 (2007)

    Article  Google Scholar 

  18. Ko, J., Kim, M., Kim, C.: 2D-To-3D stereoscopic conversion: depth-map estimation in a 2D single-view image. In: Proceedings of SPIE Conference on Applications of Digital Image Processing, vol. 66962A (2007)

  19. Jung, Y.J., Baik, A., Kim, J., Park, D.: A novel 2D-to-3D conversion technique based on relative height-depth cue. In: Proceedings of SPIE Conference on Stereoscopic Displays and Applications, vol. 72371U (2009)

  20. Cigla, C., Alatan, A.: Real-time stereo matching algorithm for 3DTV. In: 20th Signal Processing and Communications Applications Conference, pp. 1–4 (2012)

  21. Liu, T., Kender, J.R.: Computational approaches to temporal sampling of video sequences. ACM Trans. Multi. Comput. Commun. Appl. 2(2), 7–29 (2007)

    Article  Google Scholar 

  22. Ahmed, M.T., Dailey, M.N.: Robust key frame extraction for 3D reconstruction from video streams. In: Proceedings of Computer Vision Theory and Applications, pp. 231–236 (2010)

  23. Pollefeys, M., Verbiest, F., Gool, V.L.: Surviving dominant planes in uncalibrated structure and motion recovery. In: Proceedings of the 7th European Conference on Computer Vision, vol. 2351, pp. 837–851 (2002)

  24. Repko, J., Pollefeys, M.: 3D models from extended uncalibrated video sequences: addressing key-frame selection and projective drift. In: Proceedings of the 5th International Conference on 3-D Digital Imaging and Modeling, pp. 150–157 (2005)

  25. Hartley, R., Zisserman, A.: Multiple View Geometry. Cambridge University Press, Cambridge (2003)

  26. Lang, M., Hornung, A., Wang, O., Poulakos, S., Smolic, A., Gross, M.: Nonlinear disparity mapping for stereoscopic 3D. ACM Trans. Graph. 29(3), 75:1–75:10 (2010)

    Google Scholar 

  27. Kim, J., Baik, A., Jung, Y.J., Park, D.: 2D-to-3D conversion by using visual attention analysis. In: Proceedings of SPIE Conference on Stereoscopic Displays and Applications, vol. 752412 (2012)

  28. Felzenszwalb, P.F., Huttenlocher, D.P.: Efficient graph-based image segmentation. Int. J. Comput. Vis. 59(2), 167–181 (2004)

    Article  Google Scholar 

  29. Wu, C., Er, G., Xie, X., et al.: A novel method for semi-automatic 2D to 3D video conversion. In: Proceedings of 3DTV Conference: The True Vision—Capture, Transmission and Display of 3D Video, pp. 65–68 (2008)

Download references

Acknowledgments

This work was supported by the National Basic Research Program of China (973 Program) under grant No. 2012CB316302 and the National Natural Science Foundation of China under grant No. 61070107.

Author information

Authors and Affiliations

  1. Center for Internet of Things, Institute of Microelectronics of Chinese Academy of Sciences, Beijing, 100029, China

    Wei Liu

  2. National Laboratory of Pattern Recognition (NLPR), Institute of Automation of Chinese Academy of Sciences, Beijing, 100190, China

    Wei Liu, Yihong Wu, Fusheng Guo & Zhanyi Hu

Authors
  1. Wei Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yihong Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fusheng Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhanyi Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wei Liu.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Liu, W., Wu, Y., Guo, F. et al. An efficient approach for 2D to 3D video conversion based on structure from motion. Vis Comput 31, 55–68 (2015). https://doi.org/10.1007/s00371-013-0904-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-013-0904-3

Keywords

Search

Navigation