Skip to main content

Advertisement

Log in

Adaptive cascaded and parallel feature fusion for visual object tracking

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Due to its quick tracking, simple deployment, and straightforward principle, correlation filter-based tracking methods continue to have significant research implications. In order to make full use of different features while balancing the tracking speed and performance, the adaptive cascaded and parallel feature fusion-based tracker (ACPF), which could estimate the position, rotation and scale, respectively, is proposed. Comparing with other correlation filter-based trackers, the ACPF could fuse deep and handcrafted features in both Log-Polar and Cartesian branch and update templates according to the weights of response maps adaptively. Adaptive linear weights (ALW) are proposed to fuse feature response maps adaptively in the Log-Polar coordinates branch to improve the estimation of scale and rotation of object by solving constrained optimization problems. Response maps of shallow and deep features are merged adaptively by cascading numerous ALW modules in the Cartesian branch to make better utilize shallow and deep feature, and increase tracking accuracy. The final results are computed simultaneously by Cartesian and Log-Polar branch in parallel. Additionally, the learning rates are automatically changed in accordance with the weights of the ALW module to execute the adaptive template update. Extensive experiments on benchmarks show that the proposed tracker achieves the comparable results, especially in dealing with the challenges of deformation, rotation and scale variation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data availability

The data that support the findings of this study are available from the corresponding author upon reasonable request.

References

  1. Z. Li, F. Liu, W. Yang, S. Peng, J. Zhou, A survey of convolutional neural networks: analysis, applications, and prospects. In: IEEE Transactions on Neural Networks and Learning Systems (2021)

  2. Li, B., Wu, W., Wang, Q., Zhang, F., Xing, J., Yan, J.: Siamrpn++: evolution of siamese visual tracking with very deep networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4282–4291 (2019)

  3. Guo, D., Wang, J., Cui, Y., Wang, Z., Chen, S.: Siamcar: siamese fully convolutional classification and regression for visual tracking. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6269–6277 (2020)

  4. Yan, B., Peng, H., Wu, K., Wang, D., Fu, J., Lu, H.: Lighttrack: finding lightweight neural networks for object tracking via one-shot architecture search. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15180–15189 (2021)

  5. Danelljan, M., Bhat, G., Shahbaz Khan, F., Felsberg, M.: Eco: efficient convolution operators for tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6638–6646 (2017)

  6. Lukezic, A., Vojir, T., Cehovin Zajc, L., Matas, J., Kristan, M.: Discriminative correlation filter with channel and spatial reliability. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6309–6318 (2017)

  7. Bertinetto, L., Valmadre, J., Golodetz, S., Miksik, O., Torr, P. H.: Staple: complementary learners for real-time tracking In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1401–1409 (2016)

  8. Zhang, J., Ma, S., Sclaroff, S.: MEEM: robust tracking via multiple experts using entropy minimization. In: European Conference on Computer Vision, Springer, pp. 188–203 (2014)

  9. Wu, Y., Lim, J., Yang, M.-H.: Object tracking benchmark. IEEE Trans. Pattern Anal. Mach. Intell. 37(9), 1834–1848 (2015). https://doi.org/10.1109/TPAMI.2014.2388226

    Article  PubMed  Google Scholar 

  10. Kristan, M., Matas, J., Leonardis, A., Vojir, T., Pflugfelder, R., Fernandez, G., Nebehay, G., Porikli, F., Čehovin, L.: A novel performance evaluation methodology for single-target trackers. IEEE Trans. Pattern Anal. Mach. Intell. 38(11), 2137–2155 (2016). https://doi.org/10.1109/TPAMI.2016.2516982

    Article  PubMed  Google Scholar 

  11. Li, Y., Zhu, J., Hoi, S.C., Song, W., Wang, Z., Liu, H.: Robust estimation of similarity transformation for visual object tracking. Proc. AAAI Conf. Artif. Intell. 33, 8666–8673 (2019)

    Google Scholar 

  12. Bhat, G., Johnander, J., Danelljan, M., Khan, F. S. , Felsberg, M.: Unveiling the power of deep tracking. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 483–498 (2018)

  13. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), Vol. 1. IEEE, pp. 886–893 (2005)

  14. Van De Weijer, J., Schmid, C., Verbeek, J., Larlus, D.: Learning color names for real-world applications. IEEE Trans. Image Process. 18(7), 1512–1523 (2009)

    Article  MathSciNet  PubMed  ADS  Google Scholar 

  15. Wu, Y., Lim, J., Yang, M.-H.: Online object tracking: a benchmark. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2411–2418 (2013)

  16. Wu, Y., Lim, J., Yang, M.-H.: Object tracking benchmark. IEEE Trans. Pattern Anal. Mach. Intell. 37(9), 1834–1848 (2015)

    Article  PubMed  Google Scholar 

  17. Kristan, M., Matas, J., Leonardis, A., Felsberg, M., Cehovin, L., Fernandez, G., Vojir, T., Hager, G., Nebehay, G., Pflugfelder,R.: The visual object tracking vot2015 challenge results, in Proceedings of the IEEE international conference on computer vision workshops, pp. 1–23 (2015)

  18. M. Kristan, A. Leonardis, J. Matas, M. Felsberg, R. Pflugfelder, L. Cehovin Zajc, T. Vojir, G. Bhat, A. Lukezic, A. Eldesokey, et al.: The sixth visual object tracking vot2018 challenge results. In: Proceedings of the European Conference on Computer Vision (ECCV) Workshops (2018)

  19. Kristan, M., Leonardis, A., Matas, J., et al. The tenth visual object tracking vot2022 challenge results[C]//Computer Vision–ECCV 2022 Workshops: Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part VIII. Cham: Springer Nature Switzerland, 431–460 (2023)

  20. Mueller, M., Smith, N., Ghanem, B.: A benchmark and simulator for UAV tracking. In: European Conference on Computer Vision, Springer, pp. 445–461 (2016)

  21. Abbass, M.Y., Kwon, K.-C., Kim, N., Abdelwahab, S.A., El-Samie, F.E.A., Khalaf, A.A.: A survey on online learning for visual tracking. Vis. Comput. 37(5), 993–1014 (2021)

    Article  Google Scholar 

  22. Bolme, D.S., Beveridge, J.R., Draper, B.A., Lui, Y.M.: Visual object tracking using adaptive correlation filters. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE, pp. 2544–2550 (2010)

  23. Henriques, J. F., Caseiro, R., Martins, P., Batista, J.: Exploiting the circulant structure of tracking-by-detection with kernels. In: European Conference on Computer Vision, Springer, pp. 702–715 (2012)

  24. Henriques, J.F., Caseiro, R., Martins, P., Batista, J.: High-speed tracking with kernelized correlation filters. IEEE Trans. Pattern Anal. Mach. Intell. 37(3), 583–596 (2014)

    Article  Google Scholar 

  25. Qian, Q., Wu, X.-J., Kittler, J., Xu, T.-Y.: Correlation tracking with implicitly extending search region. Vis. Comput. 37(5), 1029–1043 (2021)

    Article  Google Scholar 

  26. Danelljan, M., Hager, G., Khan, F. S., Felsberg, M.: Convolutional features for correlation filter based visual tracking. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, , pp. 58–66 (2015)

  27. Danelljan, M., Hager, G., Khan, F. S., Felsberg, M.: Learning spatially regularized correlation filters for visual tracking. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4310–4318 (2015)

  28. Gundogdu, E., Alatan, A.A.: Good features to correlate for visual tracking. IEEE Trans. Image Process. 27(5), 2526–2540 (2018)

    Article  MathSciNet  ADS  Google Scholar 

  29. Zhang, W., Du, Y., Chen, Z., Deng, J., Liu, P.: Robust adaptive learning with Siamese network architecture for visual tracking. Vis. Comput. 37(5), 881–894 (2021)

    Article  Google Scholar 

  30. Huang, Y., Lu, R., Li, X., Qi, N., Yang, X.: Discriminative correlation tracking based on spatial attention mechanism for low-resolution imaging systems. Vis. Comput. 38(4), 1495–1508 (2022)

    Article  Google Scholar 

  31. Fan, C., Zhang, R., Ming, Y.: MP-LN: motion state prediction and localization network for visual object tracking. Vis. Comput. 38(12), 4291–4306 (2022)

    Article  Google Scholar 

  32. Zhang, J., Zhao, K., Dong, B., Fu, Y., Wang, Y., Yang, X., Yin, B.: Multi-domain collaborative feature representation for robust visual object tracking. Vis. Comput. 37(9), 2671–2683 (2021)

    Article  Google Scholar 

  33. Danelljan, M., Häger, G., Khan, F., Felsberg, M.: Accurate scale estimation for robust visual tracking. In: British Machine Vision Conference, Nottingham, September 1–5, Bmva Press, (2014)

  34. Li, Y., Zhu, J.: A scale adaptive kernel correlation filter tracker with feature integration. In: European Conference on Computer Vision, Springer, pp. 254–265 ( 2014)

  35. Bouraffa, T., Feng, Z., Yan, L., Xia, Y., Xiao, B.: Multi-feature fusion tracking algorithm based on peak-context learning. Image Vis. Comput. 123, 104468 (2022)

    Article  Google Scholar 

  36. Dunnhofer, M., Simonato, K., Micheloni, C.: Combining complementary trackers for enhanced long-term visual object tracking. Image Vis. Comput. 122, 104448 (2022)

    Article  Google Scholar 

  37. Ma, C., Huang, J.-B., Yang, X., Yang, M.-H.: Hierarchical convolutional features for visual tracking. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3074–3082 (2015)

  38. Valmadre, J., Bertinetto, L., Henriques, J., Vedaldi, A., Torr, P. H.: End-to-end representation learning for correlation filter based tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2805–2813 (2017)

  39. Wang, Q., Gao, J., Xing, J., Zhang, M., Hu, W.: Dcfnet: Discriminant correlation filters network for visual tracking, arXiv preprint arXiv:1704.04057 (2017)

  40. Yang, S., Chen, H., Xu, F., Li, Y., Yuan, J.: High-performance UAVs visual tracking based on Siamese network. Vis. Comput. 38(6), 2107–2123 (2022)

    Article  Google Scholar 

  41. Zhang, J., Sun, J., Wang, J., Yue, X.-G.: Visual object tracking based on residual network and cascaded correlation filters. J. Ambient Intell. Human. Comput. 12(8), 8427–8440 (2021)

    Article  Google Scholar 

  42. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition, arXiv preprint arXiv:1409.1556 (2014)

  43. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

  44. Bertinetto, L., Valmadre, J., Henriques, J. F., Vedaldi, A., Torr, P. H.: Fully-convolutional siamese networks for object tracking. In: European Conference on Computer Vision, Springer, pp. 850–865 (2016)

  45. Li, B., Yan, J., Wu, W., Zhu, Z., Hu, X.: High performance visual tracking with siamese region proposal network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8971–8980 (2018)

  46. Danelljan, M., Häger, G., Khan, F.S., Felsberg, M.: Discriminative scale space tracking. IEEE Trans. Pattern Anal. Mach. Intell. 39(8), 1561–1575 (2016)

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

This research is partially supported by Natural Science Foundation of Hebei Province (F2022201013) and Startup Foundation for Advanced Talents of Hebei University (No. 521100221003).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Kunlun Li.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This document is the results of the research project funded by Natural Science Foundation of Hebei Province (F2022201013) and Startup Foundation for Advanced Talents of Hebei University (No. 521100221003).

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, J., Li, S., Li, K. et al. Adaptive cascaded and parallel feature fusion for visual object tracking. Vis Comput 40, 2119–2138 (2024). https://doi.org/10.1007/s00371-023-02908-9

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-023-02908-9

Keywords

Navigation