Skip to main content
SpringerLink
Log in

Symmetrical Dense Optical Flow Estimation with Occlusions Detection

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

Traditional techniques of dense optical flow estimation do not generally yield symmetrical solutions: the results will differ if they are applied between images I 1 and I 2 or between images I 2 and I 1. In this work, we present a method to recover a dense optical flow field map from two images, while explicitely taking into account the symmetry across the images as well as possible occlusions in the flow field. The idea is to consider both displacements vectors from I 1 to I 2 and I 2 to I 1 and to minimise an energy functional that explicitely encodes all those properties. This variational problem is then solved using the gradient flow defined by the Euler–Lagrange equations associated to the energy. To prove the importance of the concepts of symmetry and occlusions for optical flow computation, we have extended a classical approach to handle those. Experiments clearly show the added value of these properties to improve the accuracy of the computed flows.

Figures appear in color in the online version of this paper.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Alvarez, L., Deriche, R., Papadopoulo, T., and Sanchez, J. 2002a. Symmetrical dense optical flow estimation with occlusion detection. In Proceedings of the 7th European Conference on Computer Vision, A. Heyden, G. Sparr, M. Nielsen, and P. Johansen (Eds.). Springer-Verlag: Copenhagen, Denmark, pp. 721–735.

  • Alvarez, L., Deriche, R., Weickert, J., and Sànchez, J. 2002b. Dense disparity map estimation respecting image discontinuities: A PDE and scale-space based approach. Journal of Visual Communication and Image Representation, Special Issue on Partial Differential Equations in Image Processing, Computer Vision and Computer Graphics, 13(1/2):3–21.

    Google Scholar 

  • Alvarez, L., Lions, P., and Morel, J. 1992. Image selective smoothing and edge detection by nonlinear diffusion (II). SIAM Journal of Numerical Analysis, 29:845–866.

    Article  MATH  MathSciNet  Google Scholar 

  • Alvarez, L., Weickert, J., and Sànchez, J. 1999. A scale-space approach to nonlocal optical flow calculations. In Scale-Space Theories in Computer Vision, Vol. 1682 of Lecture Notes in Computer Science, M. Nielsen, P. Johansen, O. Olsen, and J. Weickert (Eds.). Springer–Verlag, pp. 235–246.

  • Alvarez, L., Weickert, J., and Sànchez, J. 2000. Reliable estimation of dense optical flow fields with large displacements. The International Journal of Computer Vision, 39(1):41–56.

    Article  MATH  Google Scholar 

  • Aubert, G., Deriche, R., and Kornprobst, P. 1999. Computing optical flow via variational techniques. SIAM Journal of Applied Mathematics, 60(1):156–182.

    Article  MATH  MathSciNet  Google Scholar 

  • Barron, J., Fleet, D., and Beauchemin, S. 1994. Performance of optical flow techniques. The International Journal of Computer Vision, 12(1):43–77.

    Article  Google Scholar 

  • Beg, M.F., Miller, M.I., Alain, T., and Younes, L. 2005. Computing large deformation metric mappings via geodesic flows of diffeomorphisms. International Journal of Computer Vision, 61(2):139–157.

    Article  Google Scholar 

  • Black, M. and Rangarajan, P. 1996. On the unification of line processes, outlier rejection, and robust statistics with applications in early vision. The International Journal of Computer Vision, 19(1):57–91.

    Article  Google Scholar 

  • Brox, T., Bruhn, A., Papenberg, N., and Weickert, J. 2004. High accuracy optical flow estimation based on a theory for warping. In Proceedings of the 8th European Conference on Computer Vision, T. Pajdla and J. Matas (Eds.). Springer-Verlag: Prague, Czech Republic, Vol. 4, pp. 25–36.

  • Brox, T. and Weickert, J. 2002. Nonlinear matrix diffusion for optic flow estimation. In DAGM-Symposium, pp. 446–453.

  • Cachier, P. and Rey, D. 2000. Symmetrization of the non-rigid registration probem using inversion-invariant energies: application to multiple sclerosis. In Medical Image Computing and Computer-Assisted Intervention-MICCAI 2000, Vol. 1935 of Lecture Notes in Computer Science. Springer, pp. 472–481.

  • Caselles, V., Morel, J., Sapiro, G., and Tannenbaum, A. 1998. Introduction to the special issue on partial differential equations and geometry-driven diffusion in image processing and analysis. IEEE Transactions on Image Processing, 7(3):269–273.

    Article  Google Scholar 

  • Christensen, G.E. 1999. Consistent linear-elastic transformations for image matching. In Proceedings of IPMI, A.K. et al (Eds.), Vol. 1613 of LNCS. Springer-Verlag, Berlin: Heildelberg, pp. 224–237.

  • Cohen, I. 1993. Nonlinear variational method for optical flow computation. In Scandinavian Conference on Image Analysis, Vol. 1, pp. 523–530.

  • Deriche, R., Kornprobst, P., and Aubert, G. 1995. Optical flow estimation while preserving its discontinuities: A variational approach. In Proceedings of the 2nd Asian Conference on Computer Vision, Singapore, Vol. 2, pp. 71–80.

  • Enkelmann, W. 1988. Investigation of multigrid algorithms for the estimation of optical flow fields in image sequences. Computer Vision, Graphics, and Image Processing, 43:150–177.

    Article  Google Scholar 

  • Faugeras, O. 1993. On the evolution of simple curves of the real projective plane. Comptes rendus de l’Académie des Sciences de Paris, Tome 317, Série I (6):565–570. also INRIA Technical report number 1998.

  • Ghosal, S. and Vanek, P. 1996. A fast scalable algorithm for discontinuous optical flow estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 18(2):181–194.

    Article  Google Scholar 

  • Guichard, F. and Rudin, L. 1996. Accurate estimation of discontinuous optical flow by minimizing divergence related functionals. In Proceedings of the International Conference on Image Processing, Vol. I, pp. 497–500.

  • Heitz, F. and Bouthemy, P. 1993. Multimodal estimation of discontinuous optical flow using markov random fields. IEEE Transactions on Pattern Analysis and Machine Intelligence, 15(12):1217–1232.

    Article  Google Scholar 

  • Horn, B. and Schunk, B. 1981. Determining optical flow. Artificial Intelligence, 17:185–203.

    Article  Google Scholar 

  • McCane, B., Novins, K., Crannitch, D., and Galvin, B. 2002. On benchmarking optical flow. Computer Vision and Image Understanding, 84(1):126–143.

    Article  Google Scholar 

  • Meer, P., Mintz, D., Rosenfeld, A., and Kim, D. 1991. Robust regression methods for computer vision: a review. The International Journal of Computer Vision, 6(1):59–70.

    Article  Google Scholar 

  • Memin, E. and Perez, P. 1995. Multiresolution Markov random field and multigrid algorithm for discontinuity preserving estimation of the optical flow. In Proceedings of the International Society for Optical Engineering. San Diego: USA, pp. 30–41.

  • Nagel, H. 1983. Constraints for the estimation of displacement vector fields from image sequences. In International Joint Conference on Artificial Intelligence, pp. 156–160.

  • Nagel, H. and Enkelmann, W. 1986. An investigation of smoothness constraint for the estimation of displacement vector fields from image sequences. IEEE Transactions on Pattern Analysis and Machine Intelligence, 8:565–593.

    Article  Google Scholar 

  • Nagel, H.-H. 1987. On the estimation of optical flow: relations between different approaches and some new results. Artificial Intelligence Journal, 33:299–324.

    Article  Google Scholar 

  • Nési, P. 1993. Variational approach to optical flow estimation managing discontinuities. Image and Vision Computing, 11(7):419–439.

    Article  Google Scholar 

  • Nielsen, M., Johansen, P., Jackson, A., and Lautrup, B. 2002. Brownian Warps: A least committed prior for non-rigid registration. In Springer-Verlag (ed), Medical Image Computing and Computer-Assisted Intervention-MICCAI2002, Vol. 2488 of Lecture Notes in Computer Science.

  • Otte, M. and Nagel, H. 1994. Optical flow estimation: advances and comparisons. In Proceedings of the 3rd European Conference on Computer Vision, J.-O. Eklundh (Ed.), Vol. 800 of Lecture Notes in Computer Science. Springer-Verlag, pp. 51–70.

  • Proesmans, M., L. Van Gool, Pauwels, E., and Oosterlinck, A. 1994. Determination of optical flow and its discontinuities using non-linear diffusion. In Proceedings of the 3rd ECCV, II. Springer-Verlag, pp. 295–304.

  • Sapiro, G. 2001. Geometric Partial Differential Equations and Image Analysis. Cambridge University Press.

  • Schnörr, C. 1991. Determining optical flow for irregular domains by minimizing quadratic functionals of a certain class. The International Journal of Computer Vision, 6(1):25–38.

    Article  Google Scholar 

  • Snyder, M. 1995. On the mathematical foundations of smoothness constraints for the determination of optical flow and for surface reconstruction. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(11):1105–1114.

    Article  Google Scholar 

  • Strecha, C., Tuytelaars, T., and Van Gool, L. 2003. Dense matching of multiple wide-baseline views. In International Conference on Computer Vision, Vol. 2, pp. 1194–1201.

  • Szeliski, R. and Scharstein, D. 2002. Symmetric sub-pixel stereo matching. In Proceedings of the 7th European Conference on Computer Vision, A. Heyden, G. Sparr, M. Nielsen, and P. Johansen (Eds.). Springer-Verlag: Copenhagen, Denmark, Vol. 2, pp. 525–540.

  • Trouvé, A. and Younes, L. 2000. Diffeomorphic matching problems in one dimension: Designing and minimizing matching functionals. In Proceedings of the 6th European Conference on Computer Vision. Dublin: Ireland, pp. 573–587.

  • Weickert, J. 1996a. Anisotropic Diffusion in Image Processing. Ph.D. thesis, University of Kaiserslautern, Germany, Laboratory of Technomathematics.

  • Weickert, J. 1996b. Theoretical foundations of anisotropic diffusion in image processing. Computing Supplement, 11:221–236.

    Google Scholar 

  • Weickert, J. 1998. On discontinuity-preserving optic flow. In Proc. Computer Vision and Mobile Robotics Workshop. Santorini, pp. 115–122.

  • Weickert, J. and Schnörr, C. 2001. A theoretical framework for convex regularizers in PDE-based computation of image motion. The International Journal of Computer Vision, 45(3):245–264.

    Article  MATH  Google Scholar 

  • Zhang, Z. 1996. Determining the Epipolar Geometry and its Uncertainty: A Review. Technical Report 2927, INRIA Sophia-Antipolis, France.

Download references

Author information

Authors and Affiliations

  1. Universidad de Las Palmas de Gran Canaria, Las Palmas, Spain

    Luis Alvarez & Javier Sánchez

  2. INRIA Sophia-Antipolis, 2004, route des Lucioles, BP 93, 06902, Sophia-Antipolis, Cedex, France

    Rachid Deriche & Théo Papadopoulo

Authors
  1. Luis Alvarez
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rachid Deriche
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Théo Papadopoulo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Javier Sánchez
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Théo Papadopoulo.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Alvarez, L., Deriche, R., Papadopoulo, T. et al. Symmetrical Dense Optical Flow Estimation with Occlusions Detection. Int J Comput Vis 75, 371–385 (2007). https://doi.org/10.1007/s11263-007-0041-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-007-0041-4

Keywords

Search

Navigation