Skip to main content
SpringerLink
Log in

Scene Flow Estimation from Light Fields via the Preconditioned Primal-Dual Algorithm

  • Conference paper
  • First Online:
Pattern Recognition (GCPR 2014)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 8753))

Included in the following conference series:

Abstract

In this paper we present a novel variational model to jointly estimate geometry and motion from a sequence of light fields captured with a plenoptic camera. The proposed model uses the so-called sub-aperture representation of the light field. Sub-aperture images represent images with slightly different viewpoints, which can be extracted from the light field. The sub-aperture representation allows us to formulate a convex global energy functional, which enforces multi-view geometry consistency, and piecewise smoothness assumptions on the scene flow variables. We optimize the proposed scene flow model by using an efficient preconditioned primal-dual algorithm. Finally, we also present synthetic and real world experiments.

This research was supported by the FWF-START project Bilevel optimization for Computer Vision, No. Y729 and the Vision\(+\) project Integrating visual information with independent knowledge, No. 836630.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    www.lytro.com

  2. 2.

    www.raytrix.de

  3. 3.

    www.povray.org

  4. 4.

    Scenes are taken from www.oyonale.com.

References

  1. Adelson, E.H., Wang, J.Y.A.: Single lens stereo with a plenoptic camera. IEEE Trans. Pattern Anal. Mach. Intell. 14(2), 99–106 (1992)

    Article  Google Scholar 

  2. Alvertos, P., Patras, I., Alvertos, N., Tziritas, G.: Joint disparity and motion field estimation in stereoscopic image sequences. In: 13th International Conference on Pattern Recognition, pp. 359–362 (1996)

    Google Scholar 

  3. Basha, T., Moses, Y., Kiryati, N.: Multi-view scene flow estimation: a view centered variational approach. Int. J. Comput. Vis. 101(1), 6–21 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  4. Bishop, T., Favaro, P.: Plenoptic depth estimation from multiple aliased views. In: 12th International Conference on Computer Vision Workshops (ICCV Workshops), pp. 1622–1629. IEEE (2009)

    Google Scholar 

  5. Bishop, T.E., Favaro, P.: The light field camera: extended depth of field, aliasing, and superresolution. IEEE Trans. Pattern Anal. Mach. Intell. 34(5), 972–986 (2012)

    Article  Google Scholar 

  6. Bredies, K., Kunisch, K., Pock, T.: Total generalized variation. SIAM J. Imaging Sci. 3(3), 492–526 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  7. Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: Pajdla, T., Matas, J.G. (eds.) ECCV 2004. LNCS, vol. 3024, pp. 25–36. Springer, Heidelberg (2004)

    Chapter  Google Scholar 

  8. Carceroni, R.L., Kutulakos, K.N.: Multi-view scene capture by surfel sampling: from video streams to non-rigid 3d motion, shape and reflectance. Int. J. Comput. Vis. 49, 175–214 (2002)

    Article  MATH  Google Scholar 

  9. Chambolle, A., Pock, T.: A first-order primal-dual algorithm for convex problems with applications to imaging. J. Math. Imaging Vis. 40, 120–145 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  10. Courchay, J., Pons, J.-P., Monasse, P., Keriven, R.: Dense and accurate spatio-temporal multi-view stereovision. In: Zha, H., Taniguchi, R., Maybank, S. (eds.) ACCV 2009, Part II. LNCS, vol. 5995, pp. 11–22. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  11. Furukawa, Y., Ponce, J.: Dense 3d motion capture from synchronized video streams. In: 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2008). IEEE Computer Society, Anchorage, 24–26 June 2008

    Google Scholar 

  12. Goldluecke, B., Wanner, S.: The variational structure of disparity and regularization of 4d light fields. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2013)

    Google Scholar 

  13. Gortler, S.J., Grzeszczuk, R., Szeliski, R., Cohen, M.F.: The lumigraph. In: SIGGRAPH, pp. 43–54 (1996)

    Google Scholar 

  14. Heber, S., Ranftl, R., Pock, T.: Variational shape from light field. In: International Conference on Energy Minimization Methods in Computer Vision and Pattern Recognition (2013)

    Google Scholar 

  15. Isaksen, A., McMillan, L., Gortler, S.J.: Dynamically reparameterized light fields. In: SIGGRAPH, pp. 297–306 (2000)

    Google Scholar 

  16. Keriven, R., Faugeras, O.: Multi-view stereo reconstruction and scene flow estimation with a global image-based matching score. Int. J. Comput. Vis. 72, 2007 (2006)

    Google Scholar 

  17. Lumsdaine, A., Georgiev, T.: The focused plenoptic camera. In. Proceedings of IEEE ICCP, pp. 1–8 (2009)

    Google Scholar 

  18. Neumann, J., Aloimonos, Y.: Spatio-temporal stereo using multi-resolution subdivision surfaces. Int. J. Comput. Vis. 47, 2002 (2002)

    Article  Google Scholar 

  19. Ng, R.: Digital light field photography. Ph.D. thesis, Stanford University (2006). http://www.lytro.com/renng-thesis.pdf

  20. Pock, T., Chambolle, A.: Diagonal preconditioning for first order primal-dual algorithms in convex optimization. In: International Conference on Computer Vision (ICCV), pp. 1762–1769. IEEE (2011)

    Google Scholar 

  21. Ranftl, R., Gehrig, S., Pock, T., Bischof, H.: Pushing the limits of stereo using variational stereo estimation. In: Intelligent Vehicles Symposium, pp. 401–407. IEEE (2012)

    Google Scholar 

  22. Vedula, S., Baker, S., Rander, P., Collins, R.T., Kanade, T.: Three-dimensional scene flow. In: ICCV, pp. 722–729 (1999)

    Google Scholar 

  23. Wanner, S., Goldluecke, B.: Globally consistent depth labeling of 4D lightfields. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2012)

    Google Scholar 

  24. Wanner, S., Goldluecke, B.: Spatial and angular variational super-resolution of 4D light fields. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part V. LNCS, vol. 7576, pp. 608–621. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  25. Wanner, S., Straehle, C., Goldluecke, B.: Globally consistent multi-label assignment on the ray space of 4d light fields. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2013)

    Google Scholar 

  26. Wedel, A., Pock, T., Zach, C., Bischof, H., Cremers, D.: An Improved Algorithm for TV-\({L}^{1}\) Optical Flow. In: Cremers, D., Rosenhahn, B., Yuille, A.L., Schmidt, F.R. (eds.) Statistical and Geometrical Approaches to Visual Motion Analysis. LNCS, vol. 5604, pp. 23–45. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  27. Wedel, A., Rabe, C., Vaudrey, T., Brox, T., Franke, U., Cremers, D.: Efficient dense 3d scene flow from sparse or dense stereo data, Oct 2008

    Google Scholar 

  28. Wilburn, B., Joshi, N., Vaish, V., Talvala, E.V., Antunez, E., Barth, A., Adams, A., Horowitz, M., Levoy, M.: High performance imaging using large camera arrays. ACM Trans. Graph. 24(3), 765–776 (2005)

    Article  Google Scholar 

  29. Zhang, Y., Kambhamettu, C., Kambhamettu, R.: On 3d scene flow and structure estimation. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 778–785 (2001)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute for Computer Graphics and Vision, Graz University of Technology, Graz, Austria

    Stefan Heber & Thomas Pock

  2. Safety and Security Department, AIT Austrian Institute of Technology, Seibersdorf, Austria

    Thomas Pock

Authors
  1. Stefan Heber
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Thomas Pock
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Stefan Heber .

Editor information

Editors and Affiliations

  1. Department of Mathematics and Computer Science, University of Münster, Münster, Germany

    Xiaoyi Jiang

  2. Computer Science Department 5, University of Erlangen-Nürnberg, Erlangen, Germany

    Joachim Hornegger

  3. Department of Computer Science, University of Kiel, Kiel, Germany

    Reinhard Koch

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer International Publishing Switzerland

About this paper

Cite this paper

Heber, S., Pock, T. (2014). Scene Flow Estimation from Light Fields via the Preconditioned Primal-Dual Algorithm. In: Jiang, X., Hornegger, J., Koch, R. (eds) Pattern Recognition. GCPR 2014. Lecture Notes in Computer Science(), vol 8753. Springer, Cham. https://doi.org/10.1007/978-3-319-11752-2_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-11752-2_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-11751-5

  • Online ISBN: 978-3-319-11752-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation