Skip to main content
SpringerLink
Log in

Hierarchical Estimation and Segmentation of Dense Motion Fields

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

In this paper we present a comprehensive energy-based framework for the estimation and the segmentation of the apparent motion in image sequences. The robust cost functions and the associated hierarchical minimization techniques that we propose mix efficiently non-parametric (dense) representations, local interacting parametric representations, and global non-interacting parametric representations related to a partition into regions. Experimental comparisons, both on synthetic and real images, demonstrate the merit of the approach on different types of photometric and kinematic contents ranging from moving rigid objects to moving fluids.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Adiv, G. 1985. Determining three-dimensional motion and structure from optical flow generated by several moving objects. IEEE Trans. Pattern Anal. Machine Intell., 7:384–401.

    Google Scholar 

  • Ayer, S. and Sawhney, H.S. 1995. Layered representation of motion video using robust maximum-likelihood estimation of mixture models and Mdl encoding. In Proc. Int. Conf. Computer Vision, pp. 777–784.

  • Bab-Hadashiar, A. and Sutter, D. 1998. Robust optic flow computation. Int. J. Computer Vision, 29(1):59–77.

    Google Scholar 

  • Barron, J., Fleet, D., and Beauchemin, S. 1994. Performance of optical flow techniques. Int. J. Computer Vision, 12(1):43–77.

    Google Scholar 

  • Bergen, J., Anandan, P., Hanna, K., and Hingorani, R. 1992. Hierarchical model-based motion estimation. In Proc. Europ. Conf. Computer Vision, G. Sandini (Ed.), vol. 558 of LNCS, pp. 237–252, Springer-Verlag: Berlin.

    Google Scholar 

  • Besag, J. 1986. On the statistical analysis of dirty pictures. J. Royal Statist. Soc., 48B(3):259–302.

    Google Scholar 

  • Black, M. 1994. Recursive non-linear estimation of discontinuous flow fields. In Proc. Europ. Conf. Computer Vision, Stockholm, Sweden, pp. 138–145.

  • Black, M. and Anandan, P. 1996. The robust estimation of multiple motions: Parametric and piecewise-smooth flow fields. Computer Vision and Image Understanding, 63(1):75–104.

    Google Scholar 

  • Black, M. and Jepson, P. 1996. Estimating optical flow in segmented images using variable-order parametric models with local deformations. IEEE Trans. Pattern Anal. Machine Intell., 18(10):972–986.

    Google Scholar 

  • Black, M. and Rangarajan, A. 1996. On the unification of line processes, outlier rejection, and robust statistics with applications in early vision. Int. J. Computer Vision, 19(1):75–104.

    Google Scholar 

  • Bouthemy, P. and Francois, E. 1993. Motion segmentation and qualitative dynamic scene analysis from an image sequence. Int. J. Computer Vision, 10(2):157–182.

    Google Scholar 

  • Chang, M.M., Tekalp, A.M., and Sezan, M.I. 1997. Simultaneous motion estimation and segmentation. IEEE Trans. Image Processing, 6(9):1326–1333.

    Google Scholar 

  • Charbonnier, P., Blanc-Féraud, L., Aubert, G., and Barlaud, M. 1997. Deterministic edge-preserving regularization in computed imaging. IEEE Trans. Image Processing, 6(2):298–311.

    Google Scholar 

  • Cohen, I. and Herlin, I. 1999. Non uniform multiresolution method for optical flow and phase portrait models: Environmental applications. Int. J. Computer Vision, 33(1):29–49.

    Google Scholar 

  • Corpetti, T., Mémin, E., and Pérez, P. 2000. Dense fluid flow estimation. IRISA, Technical Report No. 1352.

  • Delanay, A. and Bresler, Y. 1998. Globally convergent edge-preserving regularized reconstruction: An application to limited-angle tomography. IEEE Trans. Image Processing, 7(2):204–221.

    Google Scholar 

  • Deriche, R., Kornprobst, P., and Aubert, G. 1995. Optical flow estimation while preserving its discontinuities: A variational approach. In Proc. Asian Conf. Computer Vision, Singapore, vol. 1, pp. 290–295.

    Google Scholar 

  • Enkelmann, W. 1988. Investigation of multigrid algorithms for the estimation of optical flow fields in image sequences. Comp. Vision Graph. and Image Proces., 43:150–177.

    Google Scholar 

  • Geman, D. and Reynolds, G. 1992. Constrained restoration and the recovery of discontinuities. IEEE Trans. Pattern Anal. Machine Intell., 14(3):367–383.

    Google Scholar 

  • Hackbusch, W. 1985. Multi-Grid Methods and Applications. Springer-Verlag: Berlin.

    Google Scholar 

  • Hellier, P., Barillot, C., Mémin, E., and Pérez, P. (2001). Hierarchical estimation of a dense deformation field for 3D robust registration. IEEE Transaction on Medical Imaging, 20(5):388–402.

    Google Scholar 

  • Horn, B. and Schunck, B. 1981. Determining optical flow. Artificial Intelligence, 17:185–203.

    Google Scholar 

  • Huang, Y., Palaniappan, K., Zhuand, X., and Cavanaugh, J. 1995. Optic flow field segmentation and motion estimation using a robust genetic partitioning algorithm. IEEE Trans. Pattern Anal. Machine Intell., 17(12):1177–1190.

    Google Scholar 

  • Ju, X., Black, M.J., and Jepson, A.D. 1996. Skin and bones: Multi-layer, locally affine, optical flow and regularization with transparency. In Proc. Conf. Comp. Vision Pattern Rec., pp. 307–314.

  • Kornprobst, P., Deriche, R., and Aubert, G. 1999. Image sequence analysis via partial differential equations. Journal of Mathematical Imaging and Vision, 11(1):5–26.

    Google Scholar 

  • Lai, S. and Vemuri, B. 1998. Reliable and efficient computation of optical flow. Int. J. Computer Vision, 29(2):87–105.

    Google Scholar 

  • Leclerc, Y. 1989. Constructing simple stable descriptions for image partitioning. Int. J. Computer Vision, 3:73–102.

    Google Scholar 

  • Mémin, E. and Pérez, P. 1998a. Dense estimation and object-based segmentation of the optical flow with robust techniques. IEEE Trans. Image Processing, 7(5):703–719.

    Google Scholar 

  • Mémin, E. and Pérez, P. 1998b. Amultigrid approach for hierarchical motion estimation. In Proc. Int. Conf. Computer Vision, Bombay, India, pp. 933–938.

  • Mémin, E. and Pérez, P. 1999. Fluid motion recovery by coupling dense and parametric motion fields. In Int. Conf. on Computer, ICCV'99, pp. 620–625.

  • Mitiche, A. and Bouthemy, P. 1996. Computation of image motion: A synopsis of current problems and methods. Int. Journ. of Comp. Vis., 19(1):29–55.

    Google Scholar 

  • Mumford, D. and Shah, J. 1989. Optimal approximation by piecewise smooth functions and associated variational problems. Comm. Pure and Appl. Math., 42:577–685.

    Google Scholar 

  • Murray, D.W. and Buxton, H. 1987. Scene segmentation from visual motion using global optimization. IEEE Trans. Pattern Anal. Machine Intell., 9(2):220–228.

    Google Scholar 

  • Musse, O., Heitz, F., and Armspach, J.P. 1999. 3D deformable image matching using multiscale minimization of global energy functions. In Proc. Conf. Comp. Vision Pattern Rec., Fort Collins, Colorado, vol. 2, pp. 478–485

    Google Scholar 

  • Nesi, P. 1993. Variational approach to optical flow estimation managing discontinuities. Image and Vision Computing, 11(7):419–439.

    Google Scholar 

  • Oisel, L., Mémin, E., and Morin, L. 2000. Geometric driven optical flow estimation and segmentation for 3d reconstruction. In Proc. Europ. Conf. Computer Vision, Dublin, Irland, and D. Vernon, (Ed.), vol. II, pp. 847–863. Springer Notes in Computer Science vol. 1843.

  • Samson, C., Blanc-Ferraud, L., Aubert, G., and Zerubia, J. 1999. Simultaneous image classification and restoration using a variational approach. In Proc. Conf. Comp. Vision Pattern Rec., Fort Collins, Colorado, vol. 2, pp. 618–623.

    Google Scholar 

  • Schnorr, C., Sprengel, R., and Neumann, B. 1996. A variational approach to the design of early vision algorithms. Computing Suppl., vol. 11, pp. 149–165.

    Google Scholar 

  • Schunck, B.G. 1986. The image flow constraint equation. Comput. Vision, Graphics, Image Proc., 35:20–46.

    Google Scholar 

  • Stiller, C. 1997. Object-based estimation of dense motion fields. IEEE Trans. Image Processing, 6(2):234–250.

    Google Scholar 

  • Szeliski, R. and Coughlan, J. 1994 Hierarchical spline-based image registration. In Proc. Conf. Comp. Vision Pattern Rec., Seattle, Washington, pp. 194–201.

  • Szeliski, R. and Shum, H.-Y. 1996. Motion estimation with quadtree splines. IEEE Trans. Pattern Anal. Machine Intell., 18(12):1199–1210.

    Google Scholar 

  • Thisted, R.A. 1988. Elements of Statistical Computing. Chapman and Hall: London.

    Google Scholar 

  • Viéville, T. and Faugeras, O. 1992. Robust and fast computation of unbiased intensity derivatives in images. In Proc. Europ. Conf. Computer Vision, S. Margherita Ligure, pp. 203–212.

  • Yezzi, A., Tsai, A., and Willsky, A. 1999. A statistical approach to image segmentation for bimodal and trimodal imagery. In Proc. Int. Conf. Computer Vision, Corfu, Greece.

  • Zhu, S. and Yuille, A. 1996. Region competition: Unifying snakes, region growing, and Bayes/MDL for multiband image segmentation. IEEE Trans. Pattern Anal. Machine Intell., 18:884–900.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. IRISA/Univ. Rennes I, Campus de Beaulieu, 35042, Rennes Cedex, France

    Etienne Mémin

  2. Microsoft Research, St George House 1, Guildhall Street, Cambridge, CB2 3NH, UK

    Patrick Pérez

Authors
  1. Etienne Mémin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Patrick Pérez
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Mémin, E., Pérez, P. Hierarchical Estimation and Segmentation of Dense Motion Fields. International Journal of Computer Vision 46, 129–155 (2002). https://doi.org/10.1023/A:1013539930159

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1013539930159

Search

Navigation