Abstract
A novel optical flow estimation process based on a spatio-temporal model with varying coefficients multiplying a set of basis functions at each pixel is introduced. Previous optical flow estimation methodologies did not use such an over parameterized representation of the flow field as the problem is ill-posed even without introducing any additional parameters: Neighborhood based methods of the Lucas–Kanade type determine the flow at each pixel by constraining the flow to be described by a few parameters in small neighborhoods. Modern variational methods represent the optic flow directly via the flow field components at each pixel. The benefit of over-parametrization becomes evident in the smoothness term, which instead of directly penalizing for changes in the optic flow, accumulates a cost of deviating from the assumed optic flow model. Our proposed method is very general and the classical variational optical flow techniques are special cases of it, when used in conjunction with constant basis functions. Experimental results with the novel flow estimation process yield significant improvements with respect to the best results published so far.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Adiv, G. (1985). Determining three-dimensional motion and structure from optical flow generated by several moving objects. IEEE Transactions on Pattern Analysis and Machine Intelligence, 7(4), 384–401.
Amiaz, T., & Kiryati, N. (2005). Dense discontinuous optical flow via contour-based segmentation. In Proceedings of the international conference on image processing (Vol. 3, pp. 1264–1267).
Amiaz, T., Lubetzky, E., & Kiryati, N. (2007). Coarse to over-fine optical flow estimation. Pattern Recognition 40(9), 2496–2503.
Ari, R. B., & Sochen, N. (2006). A general framework and new alignment criterion for dense optical flow. In Proceedings of the IEEE conference on computer vision and pattern recognition (Vol. 1, pp. 529–536).
Barron, J. L., Fleet, D. J., & Beauchemin, S. S. (1994). Performance of optical flow techniques. International Journal of Computer Vision, 12(1), 43–77.
Black, M., & Anandan, P. (1991). Robust dynamic motion estimation over time. In Proceedings of the computer vision and pattern recognition (pp. 292–302). Los Alamitos: IEEE Computer Society.
Black, M. J., & Anandan, P. (1996). The robust estimation of multiple motions: parametric and piecewise smooth flow fields. Computer Vision and Image Understanding, 63(1), 75–104.
Borzi, A., Ito, K., & Kunisch, K. (2002). Optimal control formulation for determining optical flow. SIAM Journal on Scientific Computing, 24(3), 818–847.
Brox, T., Bruhn, A., Papenberg, N., & Weickert, J. (2004). High accuracy optical flow estimation based on a theory for warping. In T. Pajdla & J. Matas (Eds.), Lecture notes in computer science: Vol. 3024. Proceedings of the ECCV 2004 (pp. 25–36), Prague, Czech Republic. Berlin: Springer.
Brox, T., Bruhn, A., & Weickert, J. (2006). Variational motion segmentation with level sets. In: Lecture notes in computer science: Vol. 3951. ECCV 2006, Part 1 (pp. 471–483).
Bruhn, A., Weickert, J., & Schnörr, C. (2005). Lucas/Kanade meets Horn/Schunck: combining local and global optic flow methods. International Journal of Computer Vision, 61, 211–231.
Cremers, D., & Soatto, S. (2005). Motion competition: a variational approach to piecewise parametric motion segmentation. International Journal of Computer Vision, 62(3), 249–265.
Deriche, R., Kornprobst, P., & Aubert, G. (1995). Optical flow estimation while preserving its discontinuities: a variational approach. In Proceedings of the second Asian conference on computer vision (Vol. 2, pp. 290–295).
Farnebäck, G. (2000). Fast and accurate motion estimation using orientation tensors and parametric motion models. In Proceedings of the 15th international conference on pattern recognition (Vol. 1, pp. 135–139).
Farnebäck, G. (2001). Very high accuracy velocity estimation using orientation tensors, parametric motion and simultaneous segmentation of the motion field. In Proceedings of the 8th international conference on computer vision (Vol. 1, pp. 171–177). Los Alamitos: IEEE Computer Society.
Govidu, V. M. (2006). Revisiting the brightness constraint: probabilistic formulation and algorithms. In Lecture notes in computer science: Vol. 3953. Proceedings of the ECCV 2006, Part 3 (pp. 177–188). Berlin: Springer.
Horn, B. K. P., & Schunck, B. (1981). Determining optical flow. Artificial Intelligence, 17, 185–203.
Ju, S., Black, M., & Jepson, A. (1996). Skin and bones: multi-layer, locally affine, optical flow and regularization with transparency. In Proceedings of the computer vision and pattern recognition (pp. 307–314).
Irani, M., Rousso, B., & Peleg, S. (1993). Robust recovery of ego-motion. In Computer analysis of images and patterns (pp. 371–378).
Irani, M., Rousso, B., & Peleg, S. (1997). Recovery of ego-motion using region alignment. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(3), 268–272.
Liu, H., Chellappa, R., & Rosenfeld, A. (2003). Accurate dense optical flow estimation using adaptive structure tensors and a parametric model. IEEE Transactions on Image Processing, 12, 1170–1180.
Lucas, B., & Kanade, T. (1981). An iterative image registration technique with an application to stereo vision. In Proceedings of the 7th international joint conference on artificial intelligence (pp. 674–679).
Mémin, E., & Pérez, P. (2002). Hierarchical estimation and segmentation of dense motion fields. International Journal of Computer Vision, 46(2), 129–155.
Nagel, H. H. (1990). Extending the ‘oriented smoothness constraint’ into the temporal domain and the estimation of derivatives of optical flow. In Lecture notes in computer science: Vol. 427. Proceedings of the ECCV (pp. 139–148). Berlin: Springer.
Nagel, H. H., & Enkelmann, W. (1986). An investigation of smoothness constraints for the estimation of displacement vector fields from image sequences. IEEE Transactions on Pattern Analysis and Machine Intelligence, 8, 565–593.
Nir, T., Kimmel, R., & Bruckstein, A. M. (2005). Variational approach for joint optic-flow computation and video restoration. CIS-2005-03 report, Technion.
Papenberg, N., Bruhn, A., Brox, T., Didas, S., & Weickert, J. (2006). Highly accurate optic flow computation with theoretically justified warping. International Journal of Computer Vision, 67(2), 141–158.
Roth, S., & Black, M. J. (2005). On the spatial statistics of optical flow. In Proceedings of the tenth international conference on computer vision (ICCV’05) (Vol. 1, pp. 42–49), Washington, DC, USA. Los Alamitos: IEEE Computer Society.
Sekkati, H., & Mitiche, A. (2003). Dense 3d interpretation of image sequences: a variational approach using anisotropic diffusion. In Proceedings of the ICIAP-03.
Sekkati, H., & Mitiche, A. (2006). Joint optical flow estimation, segmentation, and 3d interpretation with level sets. Computer Vision and Image Understanding, 103(2), 89–100.
Szeliski, R., & Coughlan, J. (1997). Hierarchical spline-based image registration. International Journal of Computer Vision, 22(3), 199–218.
Vázquez, C., Mitiche, A., & Laganière, R. (2006). Joint multiregion segmentation and parametric estimation of image motion by basis function representation and level set evolution. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(5), 782–793.
Weickert, J., & Schnörr, C. (2001). Variational optic flow computation with a spatio-temporal smoothness constraint. Journal of Mathematical Imaging and Vision, 14(3), 245–255.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Nir, T., Bruckstein, A.M. & Kimmel, R. Over-Parameterized Variational Optical Flow. Int J Comput Vis 76, 205–216 (2008). https://doi.org/10.1007/s11263-007-0051-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11263-007-0051-2