Skip to main content
SpringerLink
Log in

Visual tracking via confidence template updating spatial-temporal regularized correlation filters

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

As an accurate and robust method for visual object tracking task, discriminative correlation filter (DCF) framework has received significant popularity from various researchers. For a given tracking sequence, the DCF formulates the target appearance model and tracking model according to the target information of the first frame, and then predicts the location and the scale of tracking target via specific tracking strategy while the tracking model and appearance model are updated via model learning strategy. However, the tracking performance of DCF tracker is limited by the undesirable impact of boundary effect caused by algorithm deficiency and response aberrance arisen from complex tracking environment. Aiming at tackling the above difficulties, a visual tracking method based on spatial-temporal regularized correlation filter with confidence template updating is developed, in which spatial-temporal regulariziers are formulated for tracking model learning process to tackle the spawning noises in the model learning process and an adaptive model template updating strategy for tracking strategy process is adopted to repress the response aberrance effect. The main findings and scientific contributions of our method (Ours) includes: 1) a novel spatial regularization method is introduced to restrain the boundary effect and to improve the overall tracking performance by penalizing the edge coefficient of correlation filter; 2) aiming at addressing the appearance variation of tracking target, a novel temporal regularizer is suggested to formulate a more stable learning process for the tracking model and further surmount the model noise caused by deficient model learning; 3) a novel adaptive updating strategy of model template is provided to alleviate the aberrances of response representations and obtain more accurate target prediction results. Extensive experimental results with 351 challenging videos on various datasets OTB2013, OTB2015, Temple-Color and UAV123 have proven that Ours can achieve favorable performances against other state-of-the-art trackers and efficiently adapt to a variety of complex scenarios in the tracking task.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Algorithm 1
Algorithm 2
Algorithm 3
Algorithm 4
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Nai K, Li Z, Gan Y et al (2023) Robust visual tracking via multitask sparse correlation filters learning[J]. IEEE Trans Neural Netw Learn Syst 34(1):502–515

    Article  MathSciNet  Google Scholar 

  2. An Z, Wang X, Li B et al (2023) Robust visual tracking for UAVs with dynamic feature weight selection[J]. Appl Intell 53(4):3836–3849

    Article  Google Scholar 

  3. Wei B, Chen H, Cao S et al (2023) An IoU-aware Siamese network for real-time visual tracking[J]. Neurocomputing 527(3):13–26

    Article  Google Scholar 

  4. Hosseiny MS, Alruwaili F, Clancy MP et al (2023) Automatic alignment of fractured femur: Integration of robot and optical tracking system[J]. IEEE Robot Auto Lett 8(5):2438–2445

    Article  Google Scholar 

  5. Javed S, Danelljan M, Khan FS et al (2023) Visual object tracking with discriminative filters and siamese networks: A survey and outlook[J]. IEEE Trans Pattern Anal Mach Intell 45(5):6552–6574

    Google Scholar 

  6. Zadeh SM, Cheng L, Yakhdan HG et al (2022) Deep learning for visual tracking: A comprehensive survey[J]. IEEE Trans Intell Transp Syst 23(5):3943–3968

    Article  Google Scholar 

  7. Wu Y, Lim J, Yang MH (2013) Online object tracking: A benchmark[C]. Proc IEEE Conf Comput Vis Pattern Recognit:2411–2418

  8. Wu Y, Lim J, Yang MH (2015) Object tracking benchmark[J]. IEEE Trans Pattern Anal Mach Intell 37(9):1834–1848

    Article  Google Scholar 

  9. Kristan M, Leonardis A, Matas J et al (2018) The sixth visual object tracking vot2018 challenge results[C]. Proc Europ Conf Comput Vis Worksh:8–53

  10. Yang YJ, Gu XD (2023) Joint correlation and attention based feature fusion network for accurate visual tracking[J]. IEEE Trans Image Process 32:1705–1715

    Article  Google Scholar 

  11. Meng FY, Gong XM, Zhang Y (2023) SiamRank: A siamese based visual tracking network with ranking strategy[J]. Pattern Recogn 141:109630

    Article  Google Scholar 

  12. Bao H, Shu P, Zhang HC et al (2023) Siamese-based twin attention network for visual tracking[J]. IEEE Trans Circuits Syst Video Technol 33(2):847–860

    Article  Google Scholar 

  13. Elayaperumal D, Joo YH (2023) Learning spatial variance-key surrounding-aware tracking via multi-expert deep feature fusion[J]. Inf Sci 629(6):502–519

    Article  Google Scholar 

  14. Lin FL, Fu CH, He YJ (2022) ReCF: Exploiting response reasoning for correlation filters in real-time uav tracking[J]. IEEE Trans Intell Transp Syst 23(8):10469–10480

    Article  Google Scholar 

  15. Bolme DS, Beveridge JR, Draper BA et al (2010) Visual object tracking using adaptive correlation filters[C]. Proc IEEE Conf Comput Vis Pattern Recognit:2544–2550

  16. Henriques JF, Caseiro R, Martins P et al (2012) Exploiting the circulant structure of tracking-by-detection with kernels[C]. Proc Eur Conf Comput Vis:702–715

  17. Henriques JF, Caseiro R, Martins P et al (2015) High-speed tracking with kernelized correlation filters[J]. IEEE Trans Pattern Anal Mach Intell 37(3):583–596

    Article  Google Scholar 

  18. N Aslam and V Sharma (2017) Foreground detection of moving object using Gaussian mixture model[C]. Proceedings of the 2017 IEEE International Conference on Communication and Signal Processing, pp 1071–1074

  19. Feng Z, Wang P (2023) A model adaptive updating kernel correlation filter tracker with deep CNN features[J]. Eng Appl Artif Intell 123:106250

    Article  Google Scholar 

  20. N Aslam and M H Kolekar (2022) A probabilistic approach for detecting human motion in video sequence using gaussian mixture model[C]. Proceedings of 2022 2nd International Conference on Emerging Frontiers in Electrical and Electronic Technologies, 20223712704578

  21. Danelljan M, Hager G, Khan FS et al (2015) Learning spatially regularized correlation filters for visual tracking[C]. IEEE Inte Conf Comput Vis:4310–4318

  22. Yuan X, Liu J, Cheng D (2023) Motion-regularized background-aware correlation filter for marine radar target tracking[J]. IEEE Geosci Remote Sens Lett 20:3504705

    Article  Google Scholar 

  23. Chen L, Liu Y (2023) A robust spatial-temporal correlation filter tracker for efficient UAV visual tracking[J]. Appl Intell 53(4):4415–4430

    Article  Google Scholar 

  24. Wang M, Liu Y, Huang Z (2017) Large margin object tracking with circulant feature maps[C]. IEEE Conf Comput Vis Patt Recog:4021–4029

  25. Choi J, Chang HJ, Yun S et al (2017) Attentional correlation filter network for adaptive visual tracking[J]. IEEE Conf Comput Vis Patt Recog:6931–6939

  26. Huang Z, Fu C, Li Y et al (2019) Learning aberrance repressed correlation filters for real-time UAV tracking[C]. IEEE/CVF Int Conf Comput Vis:2891–2900

  27. Wei B, Chen H, Ding Q et al (2023) SiamSTC: Updatable Siamese tracking network via spatio-temporal context[J]. Knowl-Based Syst 263(3):110286

    Article  Google Scholar 

  28. Aslam N, Kolekar MH (2023) DeMAAE: Deep multiplicative attention-based autoencoder for identification of peculiarities in video sequences[J]. Vis Comput, 20232214155578

  29. Fan N, Liu Q, Li X et al (2023) Siamese residual network for efficient visual tracking[J]. Inf Sci 624(5):606–623

    Article  Google Scholar 

  30. Zhang J, He Y, Wang S (2023) Learning adaptive sparse spatially-regularized correlation filters for visual tracking[J]. IEEE Signal Proc Lett 30:11–15

    Article  Google Scholar 

  31. Zhou L, Li J, Lei B et al (2023) Correlation filter tracker with sample-reliability awareness and self-guided update[J]. IEEE Trans Circuits Syst Video Technol 33(1):118–131

    Article  Google Scholar 

  32. Liang P, Blasch E, Ling H (2015) Encoding color information for visual tracking: Algorithms and benchmark[J]. IEEE Trans Image Proc 24(12):5630–5644

    Article  MathSciNet  Google Scholar 

  33. Mueller M, Smith N, Ghanem B (2016) A benchmark and simulator for uav tracking[J]. Far East J Math ences 2(2):445–461

    Google Scholar 

  34. Danelljan M, Häger G, Khan F et al (2014) Accurate scale estimation for robust visual tracking[C]. British Machine Vision Conference, pp 1–11

  35. Bertinetto L, Valmadre J, Golodetz S et al (2016) Staple: Complementary learners for real-time tracking[C]. IEEE Conference on Computer Vision and Pattern Recognition, pp 1401–1409

  36. Galoogahi HK, Fagg A, Lucey S (2017) Learning background-aware correlation filters for visual tracking[C]. International Conference on Computer Vision, pp 1135–1143

  37. Danelljan M, Bhat G, Khan FS et al ECO: Efficient convolution operators for tracking[C]. IEEE Conference on Computer Vision and Pattern Recognition, pp 6931–693

  38. Mueller M, Smith N, Ghanem B (2017) Context-aware correlation filter tracking[C] Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 138–139

  39. Xu T, Feng ZH, Wu XJ, Kittler J (2019) Learning adaptive discriminative correlation filters via temporal consistency preserving spatial feature selection for robust visual object Tracking[J]. IEEE Trans Image Process 28(11):5596–5609

    Article  MathSciNet  Google Scholar 

  40. Li Y, Fu C, Ding F et al (2020) AutoTrack: Towards high-performance visual tracking for UAV with automatic spatio-temporal regularization[C]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 11923–11932

  41. Li B, Fu C, Ding F et al (2021) ADTrack: Target-aware dual filter learning for real-time anti-dark UAV tracking[C]. Proc - IEEE Int Con Robot Auto 5:496–502

    Google Scholar 

  42. Zhu W, Wang Z, Xu L et al (2022) Exploiting temporal coherence for self-supervised visual tracking by using vision transformer[J]. Knowl-Based Syst 251:109318

    Article  Google Scholar 

  43. Xu T, Feng Z, Wu X et al (2023) Toward robust visual object tracking with independent target-agnostic detection and effective Siamese cross-task interaction[J]. IEEE Trans Image Process 32:1541–1554

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China (No. 61671222, No. 61903162), the Postgraduate Research & Practice Innovation Program of Jiangsu Province (KYCX22_3822) and the Postgraduate Research & Practice Innovation Program of Jiangsu Province (KYCX21_3484).

Author information

Authors and Affiliations

  1. School of Electronics and Information, Jiangsu University of Science and Technology, Zhenjiang, 212100, China

    Mengquan Liang, Xuedong Wu, Siming Tang, Zhiyu Zhu, Qiang Zhang & Baiheng Cao

  2. College of Electrical and Information Engineering, Hunan University, Changsha, 410082, China

    Yaonan Wang

Authors
  1. Mengquan Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xuedong Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Siming Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhiyu Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yaonan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Qiang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Baiheng Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xuedong Wu.

Ethics declarations

Conflict of interest

On behalf of all authors, the corresponding author states that there is no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liang, M., Wu, X., Tang, S. et al. Visual tracking via confidence template updating spatial-temporal regularized correlation filters. Multimed Tools Appl 83, 37053–37072 (2024). https://doi.org/10.1007/s11042-023-16707-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-16707-w

Keywords

Search

Navigation