Skip to main content
SpringerLink
Log in

Can Variational Models for Correspondence Problems Benefit from Upwind Discretisations?

  • Published:
Journal of Mathematical Imaging and Vision Aims and scope Submit manuscript

Abstract

Optic flow and stereo reconstruction are important examples of correspondence problems in computer vision. Correspondence problems have been studied for almost 30 years, and energy-based methods such as variational approaches have become popular for solving this task. However, despite the long history of research in this field, only little attention has been paid to the numerical approximation of derivatives that naturally occur in variational approaches.

In this paper we show that strategies from hyperbolic numerics can lead to a significant quality gain in computational results. Starting from a basic formulation of correspondence problems, we take on a novel perspective on the mathematical model. Switching the roles of known and unknown with respect to image data and displacement field, we use the arising hyperbolic colour equation as a basis for a refined numerical approach. For its discretisation, we propose to use one-sided differences in the correct direction identified via a smooth predictor solution. The one-sided differences that are first-order accurate are blended with higher-order central schemes. Thereby the blending mechanism interpolates between the following two situations: The one-sided method is employed at image edges which often coincide with edges in the displacement field. In smooth image parts the higher-order scheme is used. We apply our new scheme to several prototypes of variational models for optic flow and stereo reconstruction, where we achieve significant qualitative improvements compared to standard discretisations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Baker, S., Roth, S., Scharstein, D., Black, M.J., Lewis, J.P., Szeliski, R.: A database and evaluation methodology for optical flow. In: Proc. 2007 IEEE International Conference on Computer Vision. Rio de Janeiro, Brazil. IEEE Computer Society Press, Los Alamitos (2007)

    Google Scholar 

  2. Barron, J.L., Fleet, D.J., Beauchemin, S.S.: Performance of optical flow techniques. Int. J. Comput. Vis. 12(1), 43–77 (1994)

    Article  Google Scholar 

  3. Ben-Ari, R., Sochen, N.: Variational stereo vision with sharp discontinuities and occlusion handling. In: Proc. 2007 IEEE International Conference on Computer Vision. Rio de Janeiro, Brazil. IEEE Computer Society Press, Los Alamitos (2007)

    Google Scholar 

  4. Bertero, M., Poggio, T.A., Torre, V.: Ill-posed problems in early vision. Proc. IEEE 76(8), 869–889 (1988)

    Article  Google Scholar 

  5. Bigün, J., Granlund, G.H., Wiklund, J.: Multidimensional orientation estimation with applications to texture analysis and optical flow. IEEE Trans. Pattern Anal. Mach. Intell. 13(8), 775–790 (1991)

    Article  Google Scholar 

  6. Black, M.J., Anandan, P.: The robust estimation of multiple motions: parametric and piecewise smooth flow fields. Comput. Vis. Image Underst. 63(1), 75–104 (1996)

    Article  Google Scholar 

  7. Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: Pajdla, T., Matas, J. (eds.) Computer Vision—ECCV 2004, Part IV. Lecture Notes in Computer Science, vol. 3024, pp. 25–36. Springer, Berlin (2004)

    Chapter  Google Scholar 

  8. Brown, M., Burschka, D., Hager, G.: Advances in computational stereo. IEEE Trans. Pattern Anal. Mach. Intell. 25(8), 993–1008 (2003)

    Article  Google Scholar 

  9. Bruhn, A., Weickert, J., Kohlberger, T., Schnörr, C.: A multigrid platform for real-time motion computation with discontinuity-preserving variational methods. Int. J. Comput. Vis. 70(3), 257–277 (2006)

    Article  Google Scholar 

  10. Courant, R., Friedrichs, K., Lewy, H.: Über die partiellen Differenzengleichungen der mathematischen Physik. Math. Ann. 100(1), 32–74 (1928)

    Article  MATH  MathSciNet  Google Scholar 

  11. Elsgolc, L.: Calculus of Variations. Pergamon Press, Oxford (1962)

    Google Scholar 

  12. Evans, L.C.: Partial Differential Equations. Oxford University Press, Oxford (1998)

    MATH  Google Scholar 

  13. Fleet, D.J., Weiss, Y.: Optical flow estimation. In: Paragios, N., Chen, Y., Faugeras, O. (eds.) Handbook of Mathematical Models in Computer Vision, Chap. 15, pp. 239–258. Springer, Berlin (2006)

    Google Scholar 

  14. Godlewski, E., Raviart, P.-A.: Hyperbolic Systems of Conservation Laws. Mathématiques et Applications. Ellipses, Paris (1991)

    MATH  Google Scholar 

  15. Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2000)

    MATH  Google Scholar 

  16. Horn, B., Schunck, B.: Determining optical flow. Artif. Intell. 17, 185–203 (1981)

    Article  Google Scholar 

  17. Klette, R., Schlüns, K., Koschan, A.: Computer Vision: Three-Dimensional Data from Images. Springer, Singapore (1998)

    Google Scholar 

  18. LeVeque, R.J.: Numerical Methods for Conservation Laws. Birkhäuser, Basel (1992)

    MATH  Google Scholar 

  19. LeVeque, R.J.: Finite Volume Methods for Hyperbolic Problems. Cambridge University Press, Cambridge (2002)

    Book  MATH  Google Scholar 

  20. Lucas, B., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: Proc. Seventh International Joint Conference on Artificial Intelligence. Vancouver, Canada, pp. 674–679 (1981)

    Google Scholar 

  21. Mansouri, A.R., Mitiche, A., Konrad, J.: Selective image diffusion: application to disparity estimation. In: Proc. 1998 IEEE International Conference on Image Processing, vol. 3. Chicago, IL, pp. 284–288 (1998)

    Google Scholar 

  22. Marquina, A., Osher, S.: Explicit algorithms for a new time dependent model based on level set motion for nonlinear deblurring and noise removal. SIAM J. Sci. Comput. 22(2), 387–405 (2000)

    Article  MATH  MathSciNet  Google Scholar 

  23. Mémin, E., Pérez, P.: Hierarchical estimation and segmentation of dense motion fields. Int. J. Comput. Vis. 46(2), 129–155 (2002)

    Article  MATH  Google Scholar 

  24. Morton, K.W., Mayers, L.M.: Numerical Solution of Partial Differential Equations. Cambridge University Press, Cambridge (1994)

    MATH  Google Scholar 

  25. Nir, T., Bruckstein, A.M., Kimmel, R.: Over-parameterized variational optical flow. Int. J. Comput. Vis. 76(2), 205–216 (2008)

    Article  Google Scholar 

  26. Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60, 259–268 (1992)

    MATH  Google Scholar 

  27. Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vis. 47(1–3), 7–42 (2002)

    Article  MATH  Google Scholar 

  28. Slesareva, N., Bruhn, A., Weickert, J.: Optic flow goes stereo: A variational method for estimating discontinuity-preserving dense disparity maps. In: Kropatsch, W., Sablatnig, R., Hanbury, A. (eds.) Pattern Recognition. Lecture Notes in Computer Science, vol. 3663, pp. 33–40. Springer, Berlin (2005)

    Chapter  Google Scholar 

  29. Toro, E.F.: Riemann Solvers and Numerical Methods for Fluid Dynamics, 2nd edn. Springer, Berlin (1999)

    MATH  Google Scholar 

  30. Trucco, E., Verri, A.: Introductory Techniques for 3-D Computer Vision. Prentice Hall, Englewood Cliffs (1998)

    Google Scholar 

  31. Wedel, A., Cremers, D., Pock, T., Bischof, H.: Structure- and motion-adaptive regularization for high accuracy optic flow. In: Proc. 2009 IEEE International Conference on Computer Vision. Kyoto, Japan. IEEE Computer Society Press, Los Alamitos (2009)

    Google Scholar 

  32. Werlberger, M., Pock, T., Bischof, H.: Motion estimation with non-local total variation regularization. In: Proc. 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Francisco, CA, USA. IEEE Computer Society Press, Los Alamitos (2010)

    Google Scholar 

  33. Xu, L., Jia, J., Matsushita, Y.: Motion detail preserving optical flow estimation. In: Proc. 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Francisco, CA, USA. IEEE Computer Society Press, Los Alamitos (2010)

    Google Scholar 

  34. Young, D.M.: Iterative Solution of Large Linear Systems. Dover, New York (2003)

    MATH  Google Scholar 

  35. Zimmer, H., Breuß, M., Weickert, J., Seidel, H.-P.: Hyperbolic numerics for variational approaches to correspondence problems. In: Tai, X.-C., et al. (eds.) Scale Space and Variational Methods in Computer Vision. Lecture Notes in Computer Science, vol. 5567, pp. 636–647. Springer, Berlin (2009)

    Chapter  Google Scholar 

  36. Zimmer, H., Bruhn, A., Weickert, J., Valgaerts, L., Salgado, A., Rosenhahn, B., Seidel, H.-P.: Complementary optic flow. In: Cremers, D., Boykov, Y., Blake, A., Schmidt, F.R. (eds.) Energy Minimization Methods in Computer Vision and Pattern Recognition—EMMCVPR. Lecture Notes in Computer Science, vol. 5681, pp. 207–220. Springer, Berlin (2009)

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Mathematical Image Analysis Group, Faculty of Mathematics and Computer Science, Saarland University, Campus E1.1, 66123, Saarbrücken, Germany

    Michael Breuß, Henning Zimmer & Joachim Weickert

Authors
  1. Michael Breuß
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Henning Zimmer
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Joachim Weickert
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michael Breuß.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Breuß, M., Zimmer, H. & Weickert, J. Can Variational Models for Correspondence Problems Benefit from Upwind Discretisations?. J Math Imaging Vis 39, 230–244 (2011). https://doi.org/10.1007/s10851-010-0237-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10851-010-0237-z

Keywords

Search

Navigation