Skip to main content
SpringerLink
Log in

A robust spatial-temporal correlation filter tracker for efficient UAV visual tracking

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Correlation filters (CFs) have exhibited remarkable performance in visual target tracking, especially in aerial tracking of unmanned aerial vehicles (UAVs). Most existing CF-based trackers focus on how to effectively settle the unwanted boundary effect problem, while ignoring the different contributions of the discriminative features, which would lead to suboptimal performance in tracking. In this work, a robust spatial-temporal correlation filter, i.e., the temporal regularized background-aware correlation filter (TRBCF), is proposed. In detail, by extracting real background patches as negative samples and introducing a temporal regularization term, TRBCF improves the discriminability between the target and background in the spatial domain, and achieves continuous tracking in temporal sequences. Moreover, in order to selectively highlight the informative features and effectively represent the target, a novel multi-feature fusion mechanism based on the channel-wise response maps is proposed. Extensive experiments are conducted to evaluate the effectiveness of the proposed TRBCF on three classical UAV datasets (UAV123@10fps, DTB70, and UAVDT), and TRBCF performs favorably compared with the state-of-the-art trackers, with a real-time speed (41.38 fps) on a single CPU.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Tahir A, Böling J, Haghbayan MH, Toivonen HT, Plosila J (2019) Swarms of unmanned aerial vehicles: a survey. Journal of Industrial Information Integration 16:100106

    Article  Google Scholar 

  2. Yuan C, Liu ZX, Zhang YM (2017) Aerial images-based forest fire detection for firefighting using optical remote sensing techniques and unmanned aerial vehicles. Journal of Intelligent & Robotic Systems 88:635–654

    Article  Google Scholar 

  3. Kanellakis C, Nikolakopoulos G (2017) Survey on computer vision for UAVs: Current developments and trends. Journal of Intelligent & Robotic Systems 87:141–168

    Article  Google Scholar 

  4. Jiao LC, Wang D, Bai YD, Chen PH, Liu F (2021) Deep learning in visual tracking: A review. IEEE Transactions on Neural Networks and Learning Systems. https://doi.org/10.1109/TNNLS.2021.3136907https://doi.org/10.1109/TNNLS.2021.3136907

  5. Bolme DS, Beveridge JR, Draper BA, Lui YM (2010) Visual object tracking using adaptive correlation filters. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 2544–2550

  6. Henriques JF, Caseiro R, Martins P, Batista J (2015) High-speed tracking with kernelized correlation filters. IEEE Trans Pattern Anal Mach Intell 37(3):583–596

    Article  Google Scholar 

  7. Zhang JM, Liu Y, Liu HH, Wang J, Zhang YD (2021) Distractor-aware visual tracking using hierarchical correlation filters adaptive selection. Applied Intelligence. https://doi.org/10.1007/s10489-021-02694-8https://doi.org/10.1007/s10489-021-02694-8

  8. Danelljan M, Bhat G, Khan FS, Felsberg M (2017) ECO: Efficient convolution operators for tracking. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 6931–6939

  9. Danelljan M, Häger G, Khan FS, Felsberg M (2016) Discriminative scale space tracking. IEEE Trans Pattern Anal Mach Intell 39(8):1561–1575

    Article  Google Scholar 

  10. Li Y, Zhu JK (2014) A scale adaptive kernel correlation filter tracker with feature integration. In: Proceedings of European Conference on Computer Vision, pp 254–265

  11. Danelljan M, Häger G, Khan FS, Felsberg M (2015) Learning spatially regularized correlation filters for visual tracking. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 4310–4318

  12. Gao LN, Liu B, Fu P, Xu MZ, Li JB (2021) Visual tracking via dynamic saliency discriminative correlation filter. Applied Intelligence. https://doi.org/10.1007/s10489-021-02260-2

  13. Galoogahi HK, Fagg A, Lucey S (2017) Learning background-aware correlation filters for visual tracking. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 1135–1143

  14. Wang MM, Liu Y, Huang ZY (2017) Large margin object tracking with circulant feature maps. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 4800–4808

  15. Li F, Tian C, Zuo WM, Zhang L, Yang MH (2018) Learning spatial-temporal regularized correlation filters for visual tracking. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 4904–4913

  16. Huang ZY, Fu CH, Li YM, Lin FL, Lu P (2019) Learning aberrance repressed correlation filters for real-time UAV tracking. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 2891–2900

  17. Touil DE, Terki N, Medouakh S (2018) Learning spatially correlation filters based on convolutional features via PSO algorithm and two combined color spaces for visual tracking. Appl Intell 48:2837–2846

    Article  Google Scholar 

  18. Bertinetto L, Valmadre J, Henriques JF, Vedaldi A, Torr PHS (2016) Fully-convolutional Siamese networks for object tracking. In: Proceedings of European Conference on Computer Vision, pp 850–865

  19. Jung I, Son J, Baek M, Han B (2018) Real-time MDNet. In: Proceedings of European Conference on Computer Vision, pp 89–104

  20. Song YB, Ma C, Wu XH, Gong LJ, Bao LC (2018) VITAL: Visual tracking via adversarial learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 8990–8999

  21. Valmadre J, Bertinetto L, Henriques JF, Vedaldi A, Torr PHS (2017) End-to-end representation learning for correlation filter based tracking. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 2805–2813

  22. Guo Q, Feng W, Zhou C, Huang R, Wan L, Wang S (2017) Learning dynamic Siamese network for visual object tracking. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 1763–1771

  23. Li B, Gan ZG, Chen DQ, Aleksandrovich DS (2020) UAV Maneuvering target tracking in uncertain environments based on deep reinforcement learning and meta-learning. Remote Sens 12(22):3789

    Article  Google Scholar 

  24. Li YM, Fu CH, Ding FQ, Huang ZY, Lu G (2020) AutoTrack: Towards high-performance visual tracking for UAV with automatic spatio-temporal regularization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 11923–11932

  25. Fu CH, Li BW, Ding FQ, Lin FL, Lu G (2021) Correlation filters for unmanned aerial vehicle-based aerial tracking: A review and experimental evaluation. IEEE Geoscience and Remote Sensing Magazine. https://doi.org/10.1109/MGRS.2021.3072992

  26. Lin FL, Fu CH, He YJ, Guo FY, Tang Q (2020) BiCF: Learning bidirectional incongruity-aware correlation filter for efficient UAV object tracking. In: IEEE International Conference on Robotics and Automation, pp 2365–2371

  27. Marvasti-Zadeh SM, Cheng L, Ghanei-Yakhdan H, Kasaei S (2021) Deep learning for visual tracking: A comprehensive survey. IEEE Transactions on Intelligent Transportation Systems. https://doi.org/10.1109/TITS.2020.3046478

  28. Danelljan M, Robinson A, Khan FS, Felsberg M (2016) Beyond correlation filters: Learning continuous convolution operators for visual tracking. In: Proceedings of European Conference on Computer Vision, pp 472–488

  29. Li B, Wu W, Wang Q, Zhang FY, Xing JL, Yan JJ (2019) SiamRPN++: Evolution of Siamese visual tracking with very deep networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 4282–4291

  30. Li B, Yan JJ, Wu W, Zhu Z, Hu XL (2018) High performance visual tracking with Siamese region proposal network. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 8971–8980

  31. Zhu Z, Wang Q, Li B, Wu W, Yan JJ, Hu WM (2018) Distractor-aware Siamese networks for visual object tracking. In: Proceedings of European Conference on Computer Vision, pp 101–117

  32. Mueller M, Smith N, Ghanem B (2016) A benchmark and simulator for UAV tracking. In: Proceedings of European Conference on Computer Vision, pp 445–461

  33. Li SY, Yeung DY (2017) Visual object tracking for unmanned aerial vehicles: A benchmark and new motion models. In: Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, pp 4140–4146

  34. Yu HY, Li GR, Zhang WG, Huang QM, Du DW, Tian Q, Sebe N (2020) The unmanned aerial vehicle benchmark: Object detection, tracking and baseline. Int J Comput Vis 128:1141–1159

    Article  Google Scholar 

  35. Vaswani A, Shazeer N, Parmar N, Uszkoreit J (2017) Attention is all you need. In: Advances in neural information processing systems, pp 5998–6008

  36. Tao R, Gavves E, Smeulders AWM (2016) Siamese instance search for tracking. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 1420–1429

  37. Voigtlaender P, Luiten J, Torr PHS, Leibe B (2020) Siam R-CNN: Visual tracking by re-detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 6578–6588

  38. Chen ZD, Zhong BN, Li GR, Zhang SP, Ji RR (2020) Siamese box adaptive network for visual tracking. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 6668–6677

  39. Yan B, Zhang XY, Wang D, Lu HC, Yang XY (2021) Alpha-refine: Boosting tracking performance by precise bounding box estimation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 5289–5298

  40. Chen X, Yan B, Zhu JW, Wang D, Yang XY, Lu HC (2021) Transformer tracking. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 8126–8135

  41. Guo DY, Shao YY, Cui Y, Wang ZH, Zhang LY, Shen CH (2021) Graph attention tracking. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 9543–9552

  42. Liu Z, Lin YT, Cao Y, Hu H, Wei YX, Zhang Z, Lin S, Guo BN (2021) Swin transformer: Hierarchical vision transformer using shifted windows. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 10012–10022

  43. Carion N, Massa F, Synnaeve G, Usunier N, Kirillov A, Zagoruyko S (2020) End-to-end object detection with transformers. In: Proceedings of European Conference on Computer Vision, pp 213–229

  44. Wang N, Zhou WG, Wang J, Li HQ (2021) Transformer meets tracker: Exploiting temporal context for robust visual tracking. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 1571–1580

  45. Mayer C, Danelljan M, Bhat G, Paul M, Paudel DP, Yu F, Van Gool L (2022) Transforming model prediction for tracking. arXiv:2203.11192

  46. Cao ZA, Huang ZY, Pan L, Zhang SW, Liu ZW, Fu CH (2022) TCTrack:, Temporal contexts for aerial tracking. arXiv:2203.01885

  47. Zhou XY, Yin TW, Koltun V, Krähenbühl P (2022) Global tracking transformers. arXiv:2203.13250

  48. Shen QH, Qiao L, Guo JY, Li PX, Li X, Li B, Feng WT, Gan WH, Wu W, Ouyang W (2022) Unsupervised learning of accurate Siamese tracking. arXiv:2204.01475

  49. Xie F, Wang CY, Wang GT, Cao Y, Yang WK, Zeng WJ (2022) Correlation-aware deep tracking. arXiv:2203.01666

  50. Zhou ZK, Chen JQ, Pei WJ, Mao KG, Wang HP, He ZY (2022) Global tracking via ensemble of local trackers. arXiv:2203.16092

  51. Ye JJ, Fu CH, Zheng GZ, Paudel DP, Chen G (2022) Unsupervised domain adaptation for nighttime aerial tracking. arXiv:2203.10541

  52. Ma F, Shou MZ, Zhu LC, Fan HQ, Xu YL, Yang Y, Yan ZC (2022) Unified transformer tracker for object tracking. arXiv:2203.15175

Download references

Acknowledgments

This work was supported in part by the National Natural Science Foundation of China under Grant 62033007, Grant 61873146, and Grant 61821004.

Author information

Authors and Affiliations

  1. School of Control Science and Engineering, Shandong University, Jinan, 250061, People’s Republic of China

    Lin Chen & Yungang Liu

Authors
  1. Lin Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yungang Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yungang Liu.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, L., Liu, Y. A robust spatial-temporal correlation filter tracker for efficient UAV visual tracking. Appl Intell 53, 4415–4430 (2023). https://doi.org/10.1007/s10489-022-03727-6

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-03727-6

Keywords

Search

Navigation