Skip to main content
Springer Nature Link
Log in

Discontinuity-Preserving Computation of Variational Optic Flow in Real-Time

  • Conference paper
Scale Space and PDE Methods in Computer Vision (Scale-Space 2005)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 3459))

Included in the following conference series:

  • 1576 Accesses

Abstract

Variational methods are very popular for optic flow computation: They yield dense flow fields and perform well if they are adapted such that they respect discontinuities in the image sequence or the flow field. Unfortunately, this adaptation results in high computational complexity. In our paper we show that it is possible to achieve real-time performance for these methods if bidirectional multigrid strategies are used. To this end, we study two prototypes: i) For the anisotropic image-driven technique of Nagel and Enkelmann that results in a linear system of equations we derive a regular full multigrid scheme. ii) For an isotropic flow-driven approach with total variation (TV) regularisation that requires to solve a nonlinear system of equations we develop a full multigrid strategy based on a full approximation scheme (FAS). Experiments for sequences of size 160 × 120 demonstrate the excellent performance of the proposed numerical schemes. With frame rates of 6 and 12 dense flow fields per second, respectively, both implementations outperform corresponding modified explicit schemes by two to three orders of magnitude. Thus, for the first time ever, real-time performance can be achieved for these high quality methods.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Alvarez, L., Esclarín, J., Lefébure, M., Sánchez, J.: A PDE model for computing the optical flow. In: Proc. XVI Congreso de Ecuaciones Diferenciales y Aplicaciones, Las Palmas de Gran Canaria, Spain, pp. 1349–1356 (September 1999)

    Google Scholar 

  2. Anandan, P.: A computational framework and an algorithm for the measurement of visual motion. International Journal of Computer Vision 2, 283–310 (1989)

    Article  Google Scholar 

  3. Barron, J.L., Fleet, D.J., Beauchemin, S.S.: Performance of optical flow techniques. International Journal of Computer Vision 12(1), 43–77 (1994)

    Article  Google Scholar 

  4. Borzi, A., Ito, K., Kunisch, K.: Optimal control formulation for determining optical flow. SIAM Journal on Scientific Computing 24(3), 818–847 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  5. Brandt, A.: Multi-level adaptive solutions to boundary-value problems. Mathematics of Computation 31(138), 333–390 (1977)

    Article  MATH  MathSciNet  Google Scholar 

  6. Briggs, W.L., Henson, V.E., McCormick, S.F.: A Multigrid Tutorial, 2nd edn. SIAM, Philadelphia (2000)

    Book  MATH  Google Scholar 

  7. Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optic flow estimation based on a theory for warping. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3024, pp. 25–36. Springer, Heidelberg (2004)

    Chapter  Google Scholar 

  8. Bruhn, A., Weickert, J., Feddern, C., Kohlberger, T., Schnörr, C.: Variational optical flow computation in real-time. IEEE Transactions on Image Processing (2005) (to appear)

    Google Scholar 

  9. Bruhn, A., Weickert, J., Schnörr, C.: Lucas/Kanade meets Horn/Schunck: Combining local and global optic flow methods. International Journal of Computer Vision 61(3), 1–21 (2005) (in press)

    Article  Google Scholar 

  10. Chan, R.H., Chan, T.F., Wan, W.L.: Multigrid for differential-convolution problems arising from image processing. In: Golub, G., Lui, S.H., Luk, F., Plemmons, R. (eds.) Proc. Workshop on Scientific Computing, Hong Kong, September 1997, pp. 58–72 (1997)

    Google Scholar 

  11. Cohen, I.: Nonlinear variational method for optical flow computation. In: Proc. Eighth Scandinavian Conference on Image Analysis, Tromsø, Norway, May 1993, vol. 1, pp. 523–530 (1993)

    Google Scholar 

  12. Deriche, R., Kornprobst, P., Aubert, G.: Optical-flow estimation while preserving its discontinuities: a variational approach. In: Li, S., Teoh, E.-K., Mital, D., Wang, H. (eds.) ACCV 1995. LNCS, vol. 1035, pp. 290–295. Springer, Heidelberg (1996)

    Google Scholar 

  13. El Kalmoun, M., Rüde, U.: A variational multigrid for computing the optical flow. In: Ertl, T., Girod, B., Greiner, G., Niemann, H., Seidel, H.-P., Steinbach, E., Westermann, R. (eds.) Vision, Modelling and Visualization, pp. 577–584. IOS Press, Amsterdam (2003)

    Google Scholar 

  14. Elsgolc, L.E.: Calculus of Variations. Pergamon, Oxford (1961)

    MATH  Google Scholar 

  15. Enkelmann, W.: Investigation of multigrid algorithms for the estimation of optical flow fields in image sequences. Computer Vision, Graphics and Image Processing 43, 150–177 (1987)

    Article  Google Scholar 

  16. Frohn-Schnauf, C., Henn, S., Witsch, K.: Nonlinear multigrid methods for total variation denosing. Computating and Visualization in Sciene (2004) (to appear)

    Google Scholar 

  17. Ghosal, S., Vaněk, P.Č.: Scalable algorithm for discontinuous optical flow estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence 18(2), 181–194 (1996)

    Article  Google Scholar 

  18. Glazer, F.: Multilevel relaxation in low-level computer vision. In: Rosenfeld, A. (ed.) Multiresolution Image Processing and Analysis, pp. 312–330. Springer, Berlin (1984)

    Google Scholar 

  19. Hackbusch, W.: Multigrid Methods and Applications. Springer, New York (1985)

    Google Scholar 

  20. Horn, B., Schunck, B.: Determining optical flow. Artificial Intelligence 17, 185–203 (1981)

    Article  Google Scholar 

  21. Huber, P.J.: Robust Statistics. Wiley, New York (1981)

    Book  MATH  Google Scholar 

  22. Kimmel, A., Yavneh, I.: An algebraic multigrid approach for image analysis. SIAM Journal on Scientific Computing 24(4), 1218–1231 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  23. Kumar, A., Tannenbaum, A.R., Balas, G.J.: Optic flow: a curve evolution approach. IEEE Transactions on Image Processing 5(4), 598–610 (1996)

    Article  Google Scholar 

  24. Luettgen, M.R., Karl, W.C., Willsky, A.S.: Efficient multiscale regularization with applications to the computation of optical flow. IEEE Transactions on Image Processing 3(1), 41–64 (1994)

    Article  Google Scholar 

  25. Mémin, E., Pérez, P.: A multigrid approach for hierarchical motion estimation. In: Proc. 6th International Conference on Computer Vision, Bombay, India, January 1998, pp. 933–938 (1998)

    Google Scholar 

  26. Nagel, H.-H., Enkelmann, W.: An investigation of smoothness constraints for the estimation of displacement vector fields from image sequences. IEEE Transactions on Pattern Analysis and Machine Intelligence 8, 565–593 (1986)

    Article  Google Scholar 

  27. Ortega, J.M., Rheinboldt, W.C.: Iterative Solution of Nonlinear Equations in Several Variables. In: Classics in Applied Mathematics. vol. 30, SIAM, Philadelphia (2000)

    Google Scholar 

  28. Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60, 259–268 (1992)

    Article  MATH  Google Scholar 

  29. Schnörr, C.: Segmentation of visual motion by minimizing convex non-quadratic functionals. In: Proc. Twelfth International Conference on Pattern Recognition, Jerusalem, Israel, October 1994, vol. A, pp. 661–663. IEEE Computer Society Press, Los Alamitos (1994)

    Chapter  Google Scholar 

  30. Terzopoulos, D.: Image analysis using multigrid relaxation. IEEE Transactions on Pattern Analysis and Machine Intelligence 8(2), 129–139 (1986)

    Article  Google Scholar 

  31. Trottenberg, U., Oosterlee, C., Schüller, A.: Multigrid. Academic Press, San Diego (2001)

    MATH  Google Scholar 

  32. Vogel, C.R.: A multigrid method for total variation-based image denosing. Computation and Control IV 20, 323–331 (1995)

    Google Scholar 

  33. Weickert, J., Schnörr, C.: A theoretical framework for convex regularizers in PDE-based computation of image motion. International Journal of Computer Vision 45(3), 245–264 (2001)

    Article  MATH  Google Scholar 

  34. Weickert, J., Schnörr, C.: Variational optic flow computation with a spatio-temporal smoothness constraint. Journal of Mathematical Imaging and Vision 14(3), 245–255 (2001)

    Article  MATH  Google Scholar 

  35. Wesseling, P.: An Introduction to Multigrid Methods. Wiley, Chichester (1992)

    MATH  Google Scholar 

  36. Young, D.M.: Iterative Solution of Large Linear Systems. Academic Press, New York (1971)

    MATH  Google Scholar 

  37. Zini, G., Sarti, A., Lamberti, C.: Application of continuum theory and multi-grid methods to motion evaluation from 3D echocardiography. IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control 44(2), 297–308 (1997)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Mathematical Image Analysis Group, Faculty of Mathematics and Computer Science, Building 27.1, Saarland University, 66041, Saarbrücken, Germany

    Andrés Bruhn & Joachim Weickert

  2. Computer Vision, Graphics, and Pattern Recognition Group, Department of Mathematics and Computer Science, University of Mannheim, 68131, Mannheim, Germany

    Timo Kohlberger & Christoph Schnörr

Authors
  1. Andrés Bruhn
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Joachim Weickert
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Timo Kohlberger
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Christoph Schnörr
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Dept. of Computer Science, Technion – Israel Institute of Technology, 32000, Haifa, Israel

    Ron Kimmel

  2. School of Mathematical Sciences, Tel Aviv University, 69978, Tel Aviv, Israel

    Nir A. Sochen

  3. Mathematical Image Analysis Group, Faculty of Mathematics and Computer Science, Saarland University, Building E1.1, 66041, Saarbrücken, Germany

    Joachim Weickert

Rights and permissions

Reprints and permissions

Copyright information

© 2005 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Bruhn, A., Weickert, J., Kohlberger, T., Schnörr, C. (2005). Discontinuity-Preserving Computation of Variational Optic Flow in Real-Time. In: Kimmel, R., Sochen, N.A., Weickert, J. (eds) Scale Space and PDE Methods in Computer Vision. Scale-Space 2005. Lecture Notes in Computer Science, vol 3459. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11408031_24

Download citation

  • DOI: https://doi.org/10.1007/11408031_24

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-25547-5

  • Online ISBN: 978-3-540-32012-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics