Skip to main content
SpringerLink
Log in

Inertial Constrained Hierarchical Belief Propagation for Optical Flow

  • Conference paper
  • First Online:
PRICAI 2018: Trends in Artificial Intelligence (PRICAI 2018)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 11012))

Included in the following conference series:

  • 3248 Accesses

Abstract

In computer vision, optical flow estimation has attracted the researchers’ interests for decades. Loopy belief propagation (LBP) is widely used for obtaining accurate optical flow in recent years. But its time-consumption and unfitness for large displacement scenes remains a challenging problem. In order to improve the performance of belief propagation in optical flow estimation, we propose an Inertial Constrained Hierarchical Belief Propagation (IHBPFlow) to estimate accurate optical flow. We treat input images as Markov random fields (MRF) and use possible displacements as labels and perform BP on hierarchical MRFs, i.e. superpixel MRF and pixel MRF. First we perform BP on the superpixel MRF, where the step of candidate displacements is enlarged so that the label space can be reduced. Then the basic displacements obtained from the superpixel MRF are used as initial values of the pixel MRF, which effectively compresses the space of labels, thus the process on the pixel MRF can be accelerated. Furthermore, we integrate multi-frame images and previous displacements as inertial constrained information into the proposed hierarchical BP model to enhance its ability to get reliable displacements in scenes where not enough texture information can be provided. Our method performs well on accuracy and speed and obtains competitive results on MPI Sintel dataset.

This work was supported by National Nature Science Foundation of China (No. 61773166), Natural Science Foundation of Shanghai (No. 17ZR1408200) and the Science and Technology Commission of Shanghai Municipality under research grant (No. 14DZ2260800).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    http://sintel.is.tue.mpg.de/results.

References

  1. Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., Süsstrunk, S.: SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans. Pattern Anal. Mach. Intell. 34(11), 2274–2282 (2012)

    Article  Google Scholar 

  2. Bao, L., Yang, Q., Jin, H.: Fast edge-preserving patchmatch for large displacement optical flow. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3534–3541 (2014)

    Google Scholar 

  3. Barnes, C., Shechtman, E., Goldman, D.B., Finkelstein, A.: The generalized patchmatch correspondence algorithm. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6313, pp. 29–43. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15558-1_3

    Chapter  Google Scholar 

  4. Brox, T., Bregler, C., Malik, J.: Large displacement optical flow. In: Computer Vision and Pattern Recognition, pp. 41–48. IEEE (2009)

    Google Scholar 

  5. Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: Pajdla, T., Matas, J. (eds.) ECCV 2004. LNCS, vol. 3024, pp. 25–36. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-24673-2_3

    Chapter  Google Scholar 

  6. Brox, T., Malik, J.: Large displacement optical flow: descriptor matching in variational motion estimation. IEEE Trans. Pattern Anal. Mach. Intell. 33(3), 500–513 (2011)

    Article  Google Scholar 

  7. Butler, D.J., Wulff, J., Stanley, G.B., Black, M.J.: A naturalistic open source movie for optical flow evaluation. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7577, pp. 611–625. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33783-3_44

    Chapter  Google Scholar 

  8. Chen, Q., Koltun, V.: Full flow: optical flow estimation by global optimization over regular grids. In: IEEE Conference on Computer Vision and Pattern Recognition (2016)

    Google Scholar 

  9. Felzenszwalb, P.F., Huttenlocher, D.P.: Efficient belief propagation for early vision. Int. J. Comput. Vis. 70(1), 41–54 (2006)

    Article  Google Scholar 

  10. Horn, B.K., Schunck, B.G.: Determining optical flow. Artif. Intell. 17(1–3), 185–203 (1981)

    Article  Google Scholar 

  11. Hu, Y., Song, R., Li, Y.: Efficient coarse-to-fine patchmatch for large displacement optical flow. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5704–5712 (2016)

    Google Scholar 

  12. Kennedy, R., Taylor, C.J.: Hierarchically-constrained optical flow. In: IEEE Conference on Computer Vision and Pattern Recognition (2015)

    Google Scholar 

  13. Li, Y., Min, D., Brown, M.S., Do, M.N., Lu, J.: SPM-BP: Sped-up patchmatch belief propagation for continuous MRFS. In: IEEE International Conference on Computer Vision (2015)

    Google Scholar 

  14. Liu, C., Yuen, J., Torralba, A.: SIFT flow: dense correspondence across scenes and its applications. IEEE Trans. Pattern Anal. Mach. Intell. 33(5), 978–994 (2011)

    Article  Google Scholar 

  15. Lowe, D.G.: Object recognition from local scale-invariant features. In: International Conference on Computer Vision, vol. 2, pp. 1150–1157 (1999)

    Google Scholar 

  16. Revaud, J., Weinzaepfel, P., Harchaoui, Z., Schmid, C.: EpicFlow: edge-preserving interpolation of correspondences for optical flow. In: IEEE Conference on Computer Vision and Pattern Recognition (2015)

    Google Scholar 

  17. Sun, D., Roth, S., Black, M.J.: Secrets of optical flow estimation and their principles. In: Computer Vision and Pattern Recognition, pp. 2432–2439 (2010)

    Google Scholar 

  18. Sun, D., Roth, S., Black, M.J.: A quantitative analysis of current practices in optical flow estimation and the principles behind them. Int. J. Comput. Vis. 106(2), 115–137 (2014)

    Article  Google Scholar 

  19. Weinzaepfel, P., Revaud, J., Harchaoui, Z., Schmid, C.: Deepflow: large displacement optical flow with deep matching. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1385–1392 (2013)

    Google Scholar 

  20. Xu, J., Ranftl, R., Koltun, V.: Accurate optical flow via direct cost volume processing. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017)

    Google Scholar 

  21. Zabih, R., Woodfill, J.: Non-parametric local transforms for computing visual correspondence. In: Eklundh, J.-O. (ed.) ECCV 1994. LNCS, vol. 801, pp. 151–158. Springer, Heidelberg (1994). https://doi.org/10.1007/BFb0028345

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Shanghai Key Laboratory of Multidimensional Information Processing, Department of Computer Science and Technology, East China Normal University, Shanghai, China

    Zixing Zhang & Ying Wen

Authors
  1. Zixing Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ying Wen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ying Wen .

Editor information

Editors and Affiliations

  1. Southeast University, Nanjing, China

    Xin Geng

  2. University of Tasmania, Hobart, Tasmania, Australia

    Byeong-Ho Kang

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhang, Z., Wen, Y. (2018). Inertial Constrained Hierarchical Belief Propagation for Optical Flow. In: Geng, X., Kang, BH. (eds) PRICAI 2018: Trends in Artificial Intelligence. PRICAI 2018. Lecture Notes in Computer Science(), vol 11012. Springer, Cham. https://doi.org/10.1007/978-3-319-97304-3_44

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-97304-3_44

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-97303-6

  • Online ISBN: 978-3-319-97304-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation