Skip to main content
SpringerLink
Log in

Sparse Occlusion Detection with Optical Flow

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

We tackle the problem of detecting occluded regions in a video stream. Under assumptions of Lambertian reflection and static illumination, the task can be posed as a variational optimization problem, and its solution approximated using convex minimization. We describe efficient numerical schemes that reach the global optimum of the relaxed cost functional, for any number of independently moving objects, and any number of occlusion layers. We test the proposed algorithm on benchmark datasets, expanded to enable evaluation of occlusion detection performance, in addition to optical flow.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Alvarez, L., Deriche, R., Papadopoulo, T., & Sánchez, J. (2007). Symmetrical dense optical flow estimation with occlusions detection. International Journal of Computer Vision, 75(3), 371–385.

    Article  Google Scholar 

  • Ayvaci, A., & Soatto, S. (2011). Efficient model selection for detachable object detection. In Proc. of energy minimization methods in computer vision and pattern recognition, July 2011.

    Google Scholar 

  • Ayvaci, A., Raptis, M., & Soatto, S. (2010). Occlusion detection and motion estimation with convex optimization. In Advances in neural information processing systems.

    Google Scholar 

  • Baker, S., Scharstein, D., Lewis, J., Roth, S., Black, M., & Szeliski, R. (2007). A database and evaluation methodology for optical flow. In Proc. of the international conference on computer vision (pp. 1–8).

    Google Scholar 

  • Becker, S., Bobin, J., & Candes, E. (2009). Nesta: a fast and accurate first-order method for sparse recovery. Arxiv preprint arXiv, 904.

  • Ben-Ari, R., & Sochen, N. (2007). Variational stereo vision with sharp discontinuities and occlusion handling. In Proc. of international conference on computer vision (pp. 1–7).

    Google Scholar 

  • Black, M., & Anandan, P. (1996). The robust estimation of multiple motions: Parametric and piecewise-smooth flow fields. Computer Vision and Image Understanding, 63(1), 75–104.

    Article  Google Scholar 

  • Brox, T., Bruhn, A., Papenberg, N., & Weickert, J. (2004). High accuracy optical flow estimation based on a theory for warping. In Proc. of European conference on computer vision (pp. 25–36).

    Google Scholar 

  • Bruhn, A., Weickert, J., & Schnörr, C. (2005). Lucas/Kanade meets Horn/Schunck: Combining local and global optic flow methods. International Journal of Computer Vision, 61(3), 211–231.

    Article  Google Scholar 

  • Candes, E., Wakin, M., & Boyd, S. (2008). Enhancing sparsity by reweighted L1 minimization. The Journal of Fourier Analysis and Applications, 14(5), 877–905.

    Article  MathSciNet  MATH  Google Scholar 

  • Caselles, V., Coll, B., & Morel, J.-M. (1999). Topographic maps and local contrast changes in natural images. International Journal of Computer Vision, 33(1), 5–27.

    Article  MathSciNet  Google Scholar 

  • Chan, T., Esedoglu, S., & Nikolova, M. (2006). Algorithms for finding global minimizers of denoising and segmentation models. SIAM Journal on Applied Mathematics, 66(1), 1632–1648

    MathSciNet  MATH  Google Scholar 

  • Dahl, J., Hansen, P., Jensen, S., & Jensen, T. (2009). Algorithms and software for total variation image reconstruction via first-order methods. Numerical Algorithms, 67–92.

  • Gibson, J. J. (1984). The ecological approach to visual perception. LEA.

  • Goldstein, T., & Osher, S. (2009). The split Bregman method for L1 regularized problems. SIAM Journal on Imaging Sciences, 2(2), 323–343.

    Article  MathSciNet  MATH  Google Scholar 

  • He, X., & Yuille, A. (2010). Occlusion boundary detection using pseudo-depth. In Proc. of the European conference on computer vision.

    Google Scholar 

  • Horn, B., & Schunck, B. (1981). Determining optical flow. Computer Vision, 17, 185–203.

    Google Scholar 

  • Humayun, A., Mac Aodha, O., & Brostow, G. J. (2011). Learning to find occlusion regions. In Proc. of conference on computer vision and pattern recognition.

    Google Scholar 

  • Ince, S., & Konrad, J. (2008). Occlusion-aware optical flow estimation. IEEE Transactions on Image Processing, 17(8), 1443–1451.

    Article  MathSciNet  Google Scholar 

  • Jackson, J. D., Yezzi, A. J., & Soatto, S. (2005). Dynamic shape and appearance modeling via moving and deforming layers. In Proc. of workshop on energy minimization in computer vision and pattern recognition (EMMCVPR) (pp. 427–438).

    Chapter  Google Scholar 

  • Jackson, J., Yezzi, A. J., & Soatto, S. (2008). Dynamic shape and appearance modeling via moving and deforming layers. International Journal of Computer Vision.

  • Kim, Y., Martínez, A., & Kak, A. (2005). Robust motion estimation under varying illumination. Image and Vision Computing, 23(4), 365–375.

    Article  Google Scholar 

  • Kolmogorov, V., & Zabih, R. (2001). Computing visual correspondence with occlusions via graph cuts. In Proc. of international conference on computer vision (pp. 508–515).

    Google Scholar 

  • Lim, K., Das, A., & Chong, M. (2002). Estimation of occlusion and dense motion fields in a bidirectional Bayesian framework. IEEE Transactions on Pattern Analysis and Machine Intelligence, 712–718.

  • Negahdaripour, S. (1998). Revised definition of optical flow: integration of radiometric and geometric cues for dynamic scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 961–979.

  • Nesterov, Y. (1983). A method for unconstrained convex minimization problem with the rate of convergence O (1/k 2). Doklady Akademii Nauk SSSR, 269, 543–547.

    MathSciNet  Google Scholar 

  • Nesterov, Y. (2005). Smooth minimization of non-smooth functions. Mathematical Programming, 103(1), 127–152.

    Article  MathSciNet  MATH  Google Scholar 

  • Proesmans, M., Van Gool, L., & Oosterlinck, A. (1994). Determination of optical flow and its discontinuities using a non-linear diffusion. In Proc. of European conference of computer vision.

    Google Scholar 

  • Robert, C. P. (2001). The Bayesian choice. New York: Springer.

    MATH  Google Scholar 

  • Rudin, L., Osher, S., & Fatemi, E. (1992). Nonlinear total variation based noise removal algorithms. Physica. D, 60, 259–268.

    Article  MATH  Google Scholar 

  • Shulman, D., & Herve, J. (1989). Regularization of discontinuous flow fields. In Proc. of workshop on visual motion (pp. 81–86).

    Chapter  Google Scholar 

  • Soatto, S. (2011). Actionable information in vision. In Machine learning for computer vision. Berlin: Springer.

    Google Scholar 

  • Soatto, S., & Yezzi, A. (2002). Deformotion: deforming motion, shape average and the joint segmentation and registration of images. In Proc. of the European conference on computer vision (Vol. 3, pp. 32–47).

    Google Scholar 

  • Soatto, S., Yezzi, A. J., & Jin, H. (2003). Tales of shape and radiance in multiview stereo. In Proc. of international conference on computer vision (pp. 974–981). October 2003.

    Chapter  Google Scholar 

  • Stein, A., & Hebert, M. (2009). Occlusion boundaries from motion: low-level detection and mid-level reasoning. International Journal of Computer Vision, 82(3), 325–357.

    Article  Google Scholar 

  • Strecha, C., Fransens, R., & Van Gool, L. (2004). A probabilistic approach to large displacement optical flow and occlusion detection. In ECCV workshop SMVP (pp. 71–82). Berlin: Springer.

    Google Scholar 

  • Sun, J., Li, Y., Kang, S., & Shum, H. (2005). Symmetric stereo matching for occlusion handling. In Proc. of conference on computer vision and pattern recognition (Vol. 2, p. 399).

    Google Scholar 

  • Sun, D., Roth, S., & Black, M. (2010). Secrets of optical flow estimation and their principles. In Proc. of conference on computer vision and pattern recognition (pp. 2432–2439).

    Google Scholar 

  • Sundaramoorthi, G., Petersen, P., Varadarajan, V. S., & Soatto, S. (2009). On the set of images modulo viewpoint and contrast changes. In Proc. of conference on computer vision and pattern recognition.

    Google Scholar 

  • Teng, C., Lai, S., Chen, Y., & Hsu, W. (2005). Accurate optical flow computation under non-uniform brightness variations. Computer Vision and Image Understanding, 97(3), 315–346.

    Article  Google Scholar 

  • Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society. Series B. Methodological, 58(1), 267–288.

    MathSciNet  MATH  Google Scholar 

  • Verri, A., & Poggio, T. (1989). Motion field and optical flow: Qualitative properties. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11(5), 490–498.

    Article  Google Scholar 

  • Wang, J., & Adelson, E. (1994). Representing moving images with layers. IEEE Transactions on Image Processing, 3(5), 625–638.

    Article  Google Scholar 

  • Wang, Y., Yin, W., & Zhang, Y. (2007). A fast algorithm for image deblurring with total variation regularization (CAAM Technical Reports). Rice University.

  • Wedel, A., Pock, T., Zach, C., Bischof, H., & Cremers, D. (2008). An improved algorithm for TV-L1 optical flow. In Proc. of statistical and geometrical approaches to visual motion analysis: International Dagstuhl seminar.

    Google Scholar 

  • Wedel, A., Cremers, D., Pock, T., & Bischof, H. (2009). Structure- and motion-adaptive regularization for high accuracy optic flow. In Proc. of international conference on computer vision.

    Google Scholar 

  • Weiss, P., Blanc-Féraud, L., & Aubert, G. (2009). Efficient schemes for total variation minimization under constraints in image processing. SIAM Journal on Scientific Computing, 31, 2047.

    Article  MathSciNet  MATH  Google Scholar 

  • Werlberger, M., Trobin, W., Pock, T., Wedel, A., Cremers, D., & Bischof, H. (2009). Anisotropic Huber-L1 optical flow. In Proc. of British machine vision conference.

    Google Scholar 

  • Xiao, J., Cheng, H., Sawhney, H., Rao, C., Isnardi, M., et al. (2006). Bilateral filtering-based optical flow estimation with occlusion detection. In Proc. of European conference of computer vision (Vol. 3951, p. 211).

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of California, Los Angeles, 405 Hilgard Ave, Boelter Hall #3811, Los Angeles, 90095, CA, USA

    Alper Ayvaci, Michalis Raptis & Stefano Soatto

Authors
  1. Alper Ayvaci
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michalis Raptis
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Stefano Soatto
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alper Ayvaci.

Additional information

A. Ayvaci and M. Raptis contributed equally to this work.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ayvaci, A., Raptis, M. & Soatto, S. Sparse Occlusion Detection with Optical Flow. Int J Comput Vis 97, 322–338 (2012). https://doi.org/10.1007/s11263-011-0490-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-011-0490-7

Keywords

Search

Navigation