Skip to main content
SpringerLink
Log in

Over-Parameterized Variational Optical Flow

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

A novel optical flow estimation process based on a spatio-temporal model with varying coefficients multiplying a set of basis functions at each pixel is introduced. Previous optical flow estimation methodologies did not use such an over parameterized representation of the flow field as the problem is ill-posed even without introducing any additional parameters: Neighborhood based methods of the Lucas–Kanade type determine the flow at each pixel by constraining the flow to be described by a few parameters in small neighborhoods. Modern variational methods represent the optic flow directly via the flow field components at each pixel. The benefit of over-parametrization becomes evident in the smoothness term, which instead of directly penalizing for changes in the optic flow, accumulates a cost of deviating from the assumed optic flow model. Our proposed method is very general and the classical variational optical flow techniques are special cases of it, when used in conjunction with constant basis functions. Experimental results with the novel flow estimation process yield significant improvements with respect to the best results published so far.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Adiv, G. (1985). Determining three-dimensional motion and structure from optical flow generated by several moving objects. IEEE Transactions on Pattern Analysis and Machine Intelligence, 7(4), 384–401.

    Article  Google Scholar 

  • Amiaz, T., & Kiryati, N. (2005). Dense discontinuous optical flow via contour-based segmentation. In Proceedings of the international conference on image processing (Vol. 3, pp. 1264–1267).

  • Amiaz, T., Lubetzky, E., & Kiryati, N. (2007). Coarse to over-fine optical flow estimation. Pattern Recognition 40(9), 2496–2503.

    Article  MATH  Google Scholar 

  • Ari, R. B., & Sochen, N. (2006). A general framework and new alignment criterion for dense optical flow. In Proceedings of the IEEE conference on computer vision and pattern recognition (Vol. 1, pp. 529–536).

  • Barron, J. L., Fleet, D. J., & Beauchemin, S. S. (1994). Performance of optical flow techniques. International Journal of Computer Vision, 12(1), 43–77.

    Article  Google Scholar 

  • Black, M., & Anandan, P. (1991). Robust dynamic motion estimation over time. In Proceedings of the computer vision and pattern recognition (pp. 292–302). Los Alamitos: IEEE Computer Society.

    Google Scholar 

  • Black, M. J., & Anandan, P. (1996). The robust estimation of multiple motions: parametric and piecewise smooth flow fields. Computer Vision and Image Understanding, 63(1), 75–104.

    Article  Google Scholar 

  • Borzi, A., Ito, K., & Kunisch, K. (2002). Optimal control formulation for determining optical flow. SIAM Journal on Scientific Computing, 24(3), 818–847.

    Article  MATH  MathSciNet  Google Scholar 

  • Brox, T., Bruhn, A., Papenberg, N., & Weickert, J. (2004). High accuracy optical flow estimation based on a theory for warping. In T. Pajdla & J. Matas (Eds.), Lecture notes in computer science: Vol. 3024. Proceedings of the ECCV 2004 (pp. 25–36), Prague, Czech Republic. Berlin: Springer.

    Google Scholar 

  • Brox, T., Bruhn, A., & Weickert, J. (2006). Variational motion segmentation with level sets. In: Lecture notes in computer science: Vol. 3951. ECCV 2006, Part 1 (pp. 471–483).

  • Bruhn, A., Weickert, J., & Schnörr, C. (2005). Lucas/Kanade meets Horn/Schunck: combining local and global optic flow methods. International Journal of Computer Vision, 61, 211–231.

    Article  Google Scholar 

  • Cremers, D., & Soatto, S. (2005). Motion competition: a variational approach to piecewise parametric motion segmentation. International Journal of Computer Vision, 62(3), 249–265.

    Article  Google Scholar 

  • Deriche, R., Kornprobst, P., & Aubert, G. (1995). Optical flow estimation while preserving its discontinuities: a variational approach. In Proceedings of the second Asian conference on computer vision (Vol. 2, pp.  290–295).

  • Farnebäck, G. (2000). Fast and accurate motion estimation using orientation tensors and parametric motion models. In Proceedings of the 15th international conference on pattern recognition (Vol. 1, pp. 135–139).

  • Farnebäck, G. (2001). Very high accuracy velocity estimation using orientation tensors, parametric motion and simultaneous segmentation of the motion field. In Proceedings of the 8th international conference on computer vision (Vol. 1, pp. 171–177). Los Alamitos: IEEE Computer Society.

    Google Scholar 

  • Govidu, V. M. (2006). Revisiting the brightness constraint: probabilistic formulation and algorithms. In Lecture notes in computer science: Vol. 3953. Proceedings of the ECCV 2006, Part 3 (pp. 177–188). Berlin: Springer.

    Google Scholar 

  • Horn, B. K. P., & Schunck, B. (1981). Determining optical flow. Artificial Intelligence, 17, 185–203.

    Article  Google Scholar 

  • Ju, S., Black, M., & Jepson, A. (1996). Skin and bones: multi-layer, locally affine, optical flow and regularization with transparency. In Proceedings of the computer vision and pattern recognition (pp. 307–314).

  • Irani, M., Rousso, B., & Peleg, S. (1993). Robust recovery of ego-motion. In Computer analysis of images and patterns (pp. 371–378).

  • Irani, M., Rousso, B., & Peleg, S. (1997). Recovery of ego-motion using region alignment. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(3), 268–272.

    Article  Google Scholar 

  • Liu, H., Chellappa, R., & Rosenfeld, A. (2003). Accurate dense optical flow estimation using adaptive structure tensors and a parametric model. IEEE Transactions on Image Processing, 12, 1170–1180.

    Article  Google Scholar 

  • Lucas, B., & Kanade, T. (1981). An iterative image registration technique with an application to stereo vision. In Proceedings of the 7th international joint conference on artificial intelligence (pp. 674–679).

  • Mémin, E., & Pérez, P. (2002). Hierarchical estimation and segmentation of dense motion fields. International Journal of Computer Vision, 46(2), 129–155.

    Article  MATH  Google Scholar 

  • Nagel, H. H. (1990). Extending the ‘oriented smoothness constraint’ into the temporal domain and the estimation of derivatives of optical flow. In Lecture notes in computer science: Vol. 427. Proceedings of the ECCV (pp. 139–148). Berlin: Springer.

    Google Scholar 

  • Nagel, H. H., & Enkelmann, W. (1986). An investigation of smoothness constraints for the estimation of displacement vector fields from image sequences. IEEE Transactions on Pattern Analysis and Machine Intelligence, 8, 565–593.

    Google Scholar 

  • Nir, T., Kimmel, R., & Bruckstein, A. M. (2005). Variational approach for joint optic-flow computation and video restoration. CIS-2005-03 report, Technion.

  • Papenberg, N., Bruhn, A., Brox, T., Didas, S., & Weickert, J. (2006). Highly accurate optic flow computation with theoretically justified warping. International Journal of Computer Vision, 67(2), 141–158.

    Article  Google Scholar 

  • Roth, S., & Black, M. J. (2005). On the spatial statistics of optical flow. In Proceedings of the tenth international conference on computer vision (ICCV’05) (Vol. 1, pp. 42–49), Washington, DC, USA. Los Alamitos: IEEE Computer Society.

    Chapter  Google Scholar 

  • Sekkati, H., & Mitiche, A. (2003). Dense 3d interpretation of image sequences: a variational approach using anisotropic diffusion. In Proceedings of the ICIAP-03.

  • Sekkati, H., & Mitiche, A. (2006). Joint optical flow estimation, segmentation, and 3d interpretation with level sets. Computer Vision and Image Understanding, 103(2), 89–100.

    Article  Google Scholar 

  • Szeliski, R., & Coughlan, J. (1997). Hierarchical spline-based image registration. International Journal of Computer Vision, 22(3), 199–218.

    Article  Google Scholar 

  • Vázquez, C., Mitiche, A., & Laganière, R. (2006). Joint multiregion segmentation and parametric estimation of image motion by basis function representation and level set evolution. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(5), 782–793.

    Article  Google Scholar 

  • Weickert, J., & Schnörr, C. (2001). Variational optic flow computation with a spatio-temporal smoothness constraint. Journal of Mathematical Imaging and Vision, 14(3), 245–255.

    Article  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Technion—Israel Institute of Technology, Technion City, Haifa, 32000, Israel

    Tal Nir, Alfred M. Bruckstein & Ron Kimmel

Authors
  1. Tal Nir
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alfred M. Bruckstein
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ron Kimmel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tal Nir.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Nir, T., Bruckstein, A.M. & Kimmel, R. Over-Parameterized Variational Optical Flow. Int J Comput Vis 76, 205–216 (2008). https://doi.org/10.1007/s11263-007-0051-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-007-0051-2

Keywords

Search

Navigation