Skip to main content
SpringerLink
Log in

Optical flow estimation via weighted guided filtering with non-local steering kernel

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

The weighted median filter and the guided image filter are considered important methods for the recently popular variational and non-local total variational optical flow estimation. Their attractive advantages are that outlier reduction is attained, while motion boundaries are preserved. However, these methods still suffer from halo artifacts near edges caused by motion occlusion and illumination changes in adverse outdoor conditions. To overcome these drawbacks, we propose weighted guided filtering with a non-local steering kernel during the coarse-to-fine optical flow estimation. The weighted guided filtering can preserve the motion edges more efficiently by incorporating edge-aware weighting into the filtering process, and the non-local steering kernel can leverage the edge direction more sufficiently. First, we formulate weighted guided filtering with a non-local steering kernel to preserve the edges and improve the robustness of optical flow estimation. Second, we present a combination of median filtering and weighted guided filtering with a non-local steering kernel to optimize the optical flow estimation under the coarse-to-fine process. We compare the proposed method with several state-of-the-art methods using the Middlebury and MPI Sintel test datasets. The results indicate that the proposed method is robust for optical flow estimation and able to preserve motion boundaries.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Mahfouf, Z., Merouani, H.F., Bouchrika, I., Harrati, N.: Investigating the use of motion-based features from optical flow for gait recognition. Neurocomputing 283, 140–149 (2018)

    Article  Google Scholar 

  2. Berthin, S.T., Helio, P.: Detection of complex video events through visual rhythm. Vis. Comput. 34, 145–165 (2018)

    Article  Google Scholar 

  3. Sevilla-Lara, L., Sun, D., Jampani, V., Black, M.J.: Optical flow with sementic segmentation and localized layers. In: Proceedings of IEEE conference computer vision and pattern recognition, pp 3889–3898 (2016)

  4. Guancheng, C., Huabiao Q.: Class-discriminative focal loss for extreme imbalanced multiclass object detection towards autonomous driving. Vis. Comput. https://doi.org/10.1007/s00371-021-02067-9 (2021)

  5. Bengtsson, T., McKelvey, T., Lindstrom, K.: On robust optical flow estimation on image sequences with differently exposed frames using primal–dual optimization. Image Vis. Comput. 57, 78–88 (2017)

    Article  Google Scholar 

  6. Kim, Y.-H., Martinez, A.M., Kak, A.C.: Robust motion estimation under varying illumination. Image Vis. Comput. 23, 365–375 (2005)

    Article  Google Scholar 

  7. Sun, D., Roth, S., Black, M.J.: A quantitative analysis of current practices in optical flow estimation and the principles behind them. Int. J. Comput. Vis. 106, 115–137 (2014)

    Article  Google Scholar 

  8. Horn, B.K.P., Schunck, B.G.: Determining optical flow. Artif. Intell. 17, 185–203 (1981)

    Article  MATH  Google Scholar 

  9. Sanchez, J., Llopis, E.M., Facciolo, G.: TV-\(L^{1}\) optical flow estimation. J. Image Process On Line (IPOL) 3, 137–150 (2013)

    Article  Google Scholar 

  10. Belhachmi, Z., Hecht, F.: An adaptive approach for the segmentation and the TV-filtering in the optical flow estimation. J. Math Imaging Vis. 54, 358–377 (2016)

    Article  MATH  Google Scholar 

  11. Zhang, C., Chen, Z., Wang, M., Li, M., Jiang, S.: Robust non-local optical flow estimation with occlusion detection. IEEE Trans. Image Process. 26, 4055–4067 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  12. Zhang, C., Ge, L., Chen, Z., Li, M., Liu, W., Chen, H.: Refined TV-L\(^{1}\) optical flow estimation using joint filtering. IEEE Trans. Multimed. 22, 349–364 (2020)

    Article  Google Scholar 

  13. Dong, C., Wang, Z., Han, J., Xing, C., Tang, S.: A non-local propagation filtering scheme for edge-preserving in variational optical flow computation. Signal Process. Image Commun. 93, 116143 (2021)

    Article  Google Scholar 

  14. Zhai, M., Xiang, X., Zhang, R., Lv, N., Saddik, A.E.: Optical flow estimation using channel attention mechanism and dilated convolutional neural networks. Neurocomputing 368, 124–132 (2019)

    Article  Google Scholar 

  15. Hur, J., Roth, S.: Optical Flow Estimation in the Deep Learning Age. Modelling Human Motion. Springer, Cham, pp. 119–140 (2020)

  16. Zhang, Y., Fan, H., Zheng, J., Zhang, C.:Improving the optical flow accuracy based on the total variation of local–global method. In: IEEE International Conference on Big Data (Big Data), pp. 4658–4664 (2018)

  17. Wedel, A., Pock, T., Zach, C., Bischof, H., Cremers, D.: An improved algorithm for TV-\(L^{1}\) optical flow. In: Statistical and Geometrical Approaches to Visual Analysis, pp. 23–45 (2009)

  18. Sun, D., Roth, S., Black, M.J.:Secrets of optical flow estimation and their principles. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 2432–2439 (2010)

  19. Zhang, C., Ge, L., Chen, Z., Qin, R., Li, M., Liu, W.: Guided filtering: toward edge-preserving for optical flow. IEEE Access. 6, 26958–26970 (2018)

    Article  Google Scholar 

  20. He, K., Sun, J., Tang, X.: Guided image filtering. IEEE Trans. Pattern Anal. Mach. Intell. 35, 1397–1409 (2013)

    Article  Google Scholar 

  21. He, K., Sun, J., Tang, X.: High accuracy optical flow estimation based on a theory for warping. In: Proceedings of European Conference on Computer Vision, pp. 25–36 (2004)

  22. Papenberg, N., Bruhn, A., Brox, T., Didas, S., Weickert, J.: Highly accurate optic flow computation with theoretically justified warping. Intell. J. Comput. Vis. 67, 141–158 (2006)

    Article  Google Scholar 

  23. Brox, T., Malik, J.: Large displacement optical flow: descriptor matching in variational motion estimation. IEEE Trans. Pattern Anal. Mach. Intell. 33, 500–513 (2010)

    Article  Google Scholar 

  24. Ince, S., Konrad, J.: Occlusion-aware optical flow estimation. IEEE Trans. Image Process. 17, 1443–1451 (2008)

    Article  MathSciNet  Google Scholar 

  25. Nagel, H.-H., Enkelmann, W.: An investigation of smoothness constraints for the estimation of displacement vector fields from image sequences. IEEE Trans. Pattern Anal. Mach. Intell. 8, 565–593 (1986)

    Article  Google Scholar 

  26. Alvarez, L., Sanchez, J., Lefebure, M., Monreal, J.E: A PDE model for computing the optical flow. In: Proceedings of 16th Congress Ecuaciones Diferencialesy Aplicaciones, pp. 1349–1356 (1999)

  27. Wang, H.Y., Ma, K.K.: Accurate optical flow estimation in noisy sequences by robust tensor-driven anisotropic diffusion. In: IEEE International Conference on Image Processing, pp. 111–1292 (2005)

  28. Wulff, J., Black, M.J.: Modeling blurred video with layers. In: Proceedings of IEEE International Conference on Computer Vision, pp. 236–252 (2014)

  29. Ren, X.: Local grouping for optical flow. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2008)

  30. Gavaskar, R.G., Chaudhury, K.N.: Fast adaptive bilateral filtering. IEEE Trans. Image Process. 28, 779–790 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  31. Tomasi, C., Manduchi, R.: Bilateral filtering for gray and color images. In: Proceedings of IEEE International Conference on Computer Vision, pp. 836–846 (1992)

  32. Shen, X., Zhou, C., Xu, L., Jia, J.: Mutual-structure for joint filtering. In: Proceedings of IEEE International Conference on Computer Vision, pp. 3406–3414 (2015)

  33. Li, Z., Zheng, J., Zhu, Z., Yao, W., Wu, S.: Weighted guided image filtering. IEEE Trans. Image Process. 35, 1397–1409 (2013)

    MATH  Google Scholar 

  34. Sun, Z., Han, B., Li, J., Zhang, J.: Weighted guided image filtering with steering kernel. IEEE Trans. Image Process. 29, 500–508 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  35. Scharstein, D., Hirschmuller, H., Kitajima, Y., Krathwohl, G., Nesic, N., Wang, X., Westling, P.: High-resolution stereo datasets with subpixel-accurate ground truth. In: German Conference on Pattern Recognition (GCPR), Munster, Germany, vol. 8753, pp. 31–42 (2014)

  36. Lai, R., Mo, Y., Liu, Z., Guan, J.: Local and nonlocal steering kernel weighted total variation model for image denoising. Symmetry 11, 329 (2019)

    Article  Google Scholar 

  37. Butler, D.J., Wulff, J., Stanely, G.B., Black, M.J.: A naturalistic open source movie for optical flow evaluation. In: Proceedings of European Conference on Computer Vision, pp. 611–625 (2012)

  38. Hu, Y., Song, R., Li, Y.: Efficient coarse-to-fine patchmatch for large displacement optical flow. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 5704–5712 (2016)

  39. Tu, Z., Xie, W., Cao, J., Gemeren, C.V., Poppe, R., Veltkamp, R.C.: Variational method for joint optical flow estimation and edge-aware image restoration. Pattern Recogn. 65, 11–25 (2017)

Download references

Acknowledgements

We acknowledge the support of the CSC and Fujian key Laboratory of Sensing and Computing for Smart City of Xiamen University, China.

Author information

Authors and Affiliations

  1. Fujian Key Laboratory of Sensing and Computing for Smart City, School of Informatics, Xiamen University, Xiamen, 361005, China

    Sana Rao & Hanzi Wang

Authors
  1. Sana Rao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hanzi Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sana Rao.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rao, S., Wang, H. Optical flow estimation via weighted guided filtering with non-local steering kernel. Vis Comput 39, 835–845 (2023). https://doi.org/10.1007/s00371-021-02349-2

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-021-02349-2

Keywords

Search

Navigation