Skip to main content

Advertisement

Log in

Visual tracking for UAV using adaptive spatio-temporal regularized correlation filters

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

The advance of visual tracking has provided unmanned aerial vehicle (UAV) with the intriguing capability for various practical applications. With promising performance and efficiency, discriminative correlation filter (DCF)-based trackers have drawn significant attention and undergone remarkable progress. However, the boundary effect and filter degradation remain two intractable problems. In this work, we propose a novel Adaptive Spatio-Temporal Regularized Correlation Filters (ASTR-CF) model to address the two problems. The ASTR-CF model simultaneously optimizes the spatial and temporal regularization weights adaptively, and it is optimized by the alternating direction method of multipliers (ADMM) effectively. Extensive experiments on 4 UAV tracking benchmarks have proven the superiority of the proposed ASTR-CF compared with more than 30 state-of-the-art trackers in terms of accuracy and speed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Bertinetto L, Valmadre J, Golodetz S, Miksik O, Torr PHS (2016) Staple: Complementary learners for real-time tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1401–1409, https://doi.org/10.1109/CVPR.2016.156

  2. Bertinetto L, Valmadre J, Henriques JF, Vedaldi A, Torr PHS (2016) Fully-convolutional siamese networks for object tracking. In: Proceedings of the European Conference on Computer Vision Workshops, pp 850–865, https://doi.org/10.1007/978-3-319-48881-3_56

  3. Bibi A, Mueller M, Ghanem B (2016) Target response adaptation for correlation filter tracking. In: Proceedings of the European Conference on Computer Vision, pp 419–433, https://doi.org/10.1007/978-3-319-46466-4_25

  4. Bolme DS, Beveridge JR, Draper BA, Lui YM (2010) Visual object tracking using adaptive correlation filters. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 2544–2550, https://doi.org/10.1109/CVPR.2010.5539960

  5. Boyd S, Parikh N, Chu E (2011) Distributed optimization and statistical learning via the alternating direction method of multipliers. Now Publishers Inc, https://doi.org/10.1561/9781601984616

  6. Dai K, Wang D, Lu H, Sun C, Li J (2019) Visual tracking via adaptive spatially-regularized correlation filters. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 466i5–4674, https://doi.org/10.1109/CVPR.2019.00480

  7. Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, vol 1, pp 886–893, https://doi.org/10.1109/CVPR.2005.177

  8. Danelljan M, Hager G, Khan FS, Felsberg M (2015) Learning spatially regularized correlation filters for visual tracking. In: Proceedings of the IEEE International Conference on Computer Vision, pp 4310–4318, https://doi.org/10.1109/ICCV.2015.490

  9. Danelljan M, Hager G, Shahbaz Khan F, Felsberg M (2016) Adaptive decontamination of the training set: A unified formulation for discriminative visual tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1430–1438, https://doi.org/10.1109/CVPR.2016.159

  10. Danelljan M, Robinson A, Shahbaz Khan F, Felsberg M (2016) Beyond correlation filters: Learning continuous convolution operators for visual tracking. In: Proceedings of the European Conference on Computer Vision, pp 472–488, https://doi.org/10.1007/978-3-319-46454-1_29

  11. Danelljan M, Bhat G, Khan FS, Felsberg M (2017) Eco: Efficient convolution operators for tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 9631–6939, https://doi.org/10.1109/CVPR.2017.733

  12. Danelljan M, Hager̈ G, Khan FS, Felsberg M (2017) Discriminative scale space tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence 39(8):1561–1575, https://doi.org/10.1109/TPAMI.2016.2609928

  13. De Moraes RS, de Freitas EP (2020) Multi-uav based crowd monitoring system. IEEE Transactions on Aerospace and Electronic Systems 56(2):1332–1345, https://doi.org/10.1109/TAES.2019.2952420

  14. Du D, Qi Y, Yu H, Yang Y, Duan K, Li G, Zhang W, Huang Q, Tian Q (2018) The unmanned aerial vehicle benchmark: Object detection and tracking. In: Proceedings of the European Conference on Computer Vision, pp 375–391, https://doi.org/10.1007/978-3-030-01249-6_23

  15. Fan H, Ling H (2017) Parallel tracking and verifying: A framework for real-time and high accuracy visual tracking. In: Proceedings of the IEEE International Conference on Computer Vision, pp 5487–5495, https://doi.org/10.1109/ICCV.2017.585

  16. Fan H, Lin L, Yang F, Chu P, Deng G, Yu S, Bai H, Xu Y, Liao C, Ling H (2019) Lasot: A high-quality benchmark for large-scale single object tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 5369–5378, https://doi.org/10.1109/CVPR.2019.00552

  17. Fu C, Zhang Y, Huang Z, Duan R, Xie Z (2019) Part-based background-aware tracking for uav with convolutional features. IEEE Access 7:79997–80010, https://doi.org/10.1109/ACCESS.2019.2922703

  18. Fu C, Xu J, Lin F, Guo F, Liu T, Zhang Z (2020) Object saliency-aware dual regularized correlation filter for real-time aerial tracking. IEEE Transactions on Geoscience and Remote Sensing 58(12):8940–8951, https://doi.org/10.1109/TGRS.2020.2992301

  19. Galoogahi HK, Fagg A, Lucey S (2017) Learning background-aware correlation filters for visual tracking. In: Proceedings of the IEEE International Conference on Computer Vision, pp 1145–1152, https://doi.org/10.1109/ICCV.2017.129

  20. Gao P, Yuan R, Wang F, Xiao L, Fujita H, Zhang Y (2020) Siamese attentional keypoint network for high performance visual tracking. Knowledge-Based Systems 193:105448, https://doi.org/10.1016/j.knosys.2019.105448

  21. Gao P, Zhang Q, Wang F, Xiao L, Fujita H, Zhang Y (2020) Learning reinforced attentional representation for end-to-end visual tracking. Information Sciences 517:52–67, https://doi.org/10.1016/j.ins.2019.12.084

  22. Han Z, Wang P, Ye Q (2020) Adaptive discriminative deep correlation filter for visual object tracking. IEEE Transactions on Circuits and Systems for Video Technology 30(1):155–166, https://doi.org/10.1109/TCSVT.2018.2888492

  23. He Z, Fan Y, Zhuang J, Dong Y, Bai H (2017) Correlation filters with weighted convolution responses. In: Proceedings of the International Conference on Computer Vision Workshops, pp 1992–2000, https://doi.org/10.1109/ICCVW.2017.233

  24. Held D, Thrun S, Savarese S (2016) Learning to track at 100 fps with deep regression networks. In: Proceedings of the European Conference on Computer Vision, pp 749–765, https://doi.org/10.1007/978-3-319-46448-0_45

  25. Henriques JF, Caseiro R, Martins P, Batista J (2012) Exploiting the circulant structure of tracking-by-detection with kernels. In: Proceedings of the European Conference on Computer Vision, pp 702–715, https://doi.org/10.1007/978-3-642-33765-9_50

  26. Henriques JF, Caseiro R, Martins P, Batista J (2015) High-speed tracking with kernelized correlation filters. IEEE Transactions on Pattern Analysis and Machine Intelligence 37(3):583–596, https://doi.org/10.1109/TPAMI.2014.2345390

  27. Huang Z, Fu C, Li Y, Lin F, Lu P (2019) Learning aberrance repressed correlation filters for real-time uav tracking. In: Proceedings of the International Conference on Computer Vision, pp 2891–2900, https://doi.org/10.1109/ICCV.2019.00298

  28. Kristan M, Matas J, Leonardis A, Vojir T, Pflugfelder R, Fernandez G, Nebehay G, Porikli F, ÚCCehovin L (2016) A novel performance evaluation methodology for single-target trackers. IEEE Transactions on Pattern Analysis and Machine Intelligence 38(11):2137–2155, https://doi.org/10.1109/TPAMI.2016.2516982

  29. Li F, Yao Y, Li P, Zhang D, Zuo W, Yang M (2017) Integrating boundary and center correlation filters for visual tracking with aspect ratio variation. In: Proceedings of the IEEE International Conference on Computer Vision Workshop, pp 2001–2009, https://doi.org/10.1109/ICCVW.2017.234

  30. Li F, Tian C, Zuo W, Zhang L, Yang M (2018) Learning spatial-temporal regularized correlation filters for visual tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 4904–4913, https://doi.org/10.1109/CVPR.2018.00515

  31. Li S, Yeung DY (2017) Visual object tracking for unmanned aerial vehicles: A benchmark and new motion models. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp 4140–4146

  32. Li X, Liu Q, He Z, Wang H, Zhang C, Chen WS (2016) A multi-view model for visual tracking via correlation filters. Knowledge-Based Systems 113:88–99, https://doi.org/10.1016/j.knosys.2016.09.014

  33. Li Y, Fu C, Ding F, Huang Z, Lu G (2020) Autotrack: Towards high-performance visual tracking for uav with automatic spatio-temporal regularization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 11923–11932, https://doi.org/10.1109/CVPR42600.2020.01194

  34. Li Y, Fu C, Huang Z, Zhang Y, Pan J (2021) Intermittent contextual learning for keyfilter-aware uav object tracking using deep convolutional feature. IEEE Transactions on Multimedia 23:810–822, https://doi.org/10.1109/TMM.2020.2990064

  35. Liu T, Wang G, Yang Q (2015) Real-time part-based visual tracking via adaptive correlation filters. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 4902–4912, https://doi.org/10.1109/CVPR.2015.7299124

  36. Lukezic A, Vojir T, CehovinZajc L, Matas J, Kristan M (2018) Discriminative correlation filter tracker with channel and spatial reliability. International Journal of Computer Vision 126(7):671–688, https://doi.org/10.1007/s11263-017-1061-3

  37. Ma C, Huang JB, Yang X, Yang MH (2019) Robust visual tracking via hierarchical convolutional features. IEEE Transactions on Pattern Analysis and Machine Intelligence 41(11):2709–2723, https://doi.org/10.1109/TPAMI.2018.2865311

  38. Mueller M, Smith N, Ghanem B (2016) A benchmark and simulator for uav tracking. In: Proceedings of the European Conference on Computer Vision, pp 445–461, https://doi.org/10.1007/978-3-319-46448-0_27

  39. Mueller M, Smith N, Ghanem B (2017) Context-aware correlation filter tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1387–1395, https://doi.org/10.1109/CVPR.2017.152

  40. Nam H, Han B (2016) Learning multi-domain convolutional neural networks for visual tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 4293–4302, https://doi.org/10.1109/CVPR.2016.465

  41. Padhy RP, Xia F, Choudhury SK, Sa PK, Bakshi S (2019) Monocular vision aided autonomous uav navigation in indoor corridor environments. IEEE Transactions on Sustainable Computing 4(1):96–108, https://doi.org/10.1109/TSUSC.2018.2810952

  42. Qi Y, Zhang S, Qin L, Yao H, Huang Q, Lim J, Yang M (2016) Hedged deep tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 4303–4311, https://doi.org/10.1109/CVPR.2016.466

  43. Rey N, Volpi M, Joost S, Tuia D (2017) Detecting animals in african savanna with uavs and the crowds. Remote Sensing of Environment 200:341–351, https://doi.org/10.1016/j.rse.2017.08.026

  44. Song Y, Ma C, Gong L, Zhang J, Lau RWH, Yang M (2017) Crest: Convolutional residual learning for visual tracking. In: Proceedings of the IEEE International Conference on Computer Vision, pp 2574–2583, https://doi.org/10.1109/ICCV.2017.279

  45. Sun Y, Sun C, Wang D, He Y, Lu H (2019) Roi pooled correlation filters for visual tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 5776–5784, https://doi.org/10.1109/CVPR.2019.00593

  46. Tang Y, Hu Y, Cui J, Liao F, Lao M, Lin F, Teo RSH (2019) Vision-aided multi-uav autonomous flocking in gps-denied environment. IEEE Transactions on Industrial Electronics 66(1):616–626, https://doi.org/10.1109/TIE.2018.2824766

  47. Valmadre J, Bertinetto L, Henriques J, Vedaldi A, Torr PHS (2017) End-to-end representation learning for correlation filter based tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 5000–5008, https://doi.org/10.1109/CVPR.2017.531

  48. van de Weijer J, Schmid C, Verbeek J, Larlus D (2009) Learning color names for real-world applications. IEEE Transactions on Image Processing 18(7):1512–1523, https://doi.org/10.1109/TIP.2009.2019809

  49. Voigtlaender P, Luiten J, Torr PHS, Leibe B (2020) Siam r-cnn: Visual tracking by re-detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, https://doi.org/10.1109/CVPR42600.2020.00661

  50. Wang C, Wang J, Shen Y, Zhang X (2019) Autonomous navigation of uavs in large-scale complex environments: A deep reinforcement learning approach. IEEE Transactions on Vehicular Technology 68(3):2124–2136, https://doi.org/10.1109/TVT.2018.2890773

  51. Wang L, Ouyang W, Wang X, Lu H (2016) Stct: Sequentially training convolutional networks for visual tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1373–1381, https://doi.org/10.1109/CVPR.2016.153

  52. Wang M, Liu Y, Huang Z (2017) Large margin object tracking with circulant feature maps. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 4800–4808, https://doi.org/10.1109/CVPR.2017.510

  53. Wang N, Zhou W, Tian Q, Hong R, Wang M, Li H (2018) Multi-cue correlation filters for robust visual tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 4844–4853, https://doi.org/10.1109/CVPR.2018.00509

  54. Wang N, Song Y, Ma C, Zhou W, Liu W, Li H (2019) Unsupervised deep tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1308–1317, https://doi.org/10.1109/CVPR.2019.00140

  55. Wang W, Zhang K, Lv M, Wang J (2020) Hierarchical spatiotemporal context-aware correlation filters for visual tracking. IEEE Transactions on Cybernetics pp 1–14, https://doi.org/10.1109/TCYB.2020.2964757

  56. Wen L, Zhu P, Du D, et al (2019) Visdrone-sot2018: The vision meets drone single-object tracking challenge results. In: Proceedings of the European Conference on Computer Vision, pp 469–495, https://doi.org/10.1007/978-3-030-11021-5_28

  57. Wu Y, Lim J, Yang MH (2015) Object tracking benchmark. IEEE Transactions on Pattern Analysis and Machine Intelligence 37(9):1834–1848, https://doi.org/10.1109/TPAMI.2014.2388226

  58. Xu T, Feng ZH, Wu XJ, Kittler J (2019) Learning adaptive discriminative correlation filters via temporal consistency preserving spatial feature selection for robust visual object tracking. IEEE Transactions on Image Processing 28(11):5596–5609, https://doi.org/10.1109/TIP.2019.2919201

  59. Xu Y, Wang Z, Li Z, Yuan Y, Yu G (2020) Siamfc++: Towards robust and accurate visual tracking with target estimation guidelines. In: Proceedings of the Association for the Advance of Artificial Intelligence, https://doi.org/10.1609/aaai.v34i07.6944

  60. Yun S, Choi J, Yoo Y, Yun K, Choi JY (2017) Action-decision networks for visual tracking with deep reinforcement learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1349–1358, https://doi.org/10.1109/CVPR.2017.148

  61. Zahran S, Moussa AM, Sesay AB, El-Sheimy N (2019) A new velocity meter based on hall effect sensors for uav indoor navigation. IEEE Sensors Journal 19(8):3067–3076, https://doi.org/10.1109/JSEN.2018.2890094

  62. Zhang L, Varadarajan J, Suganthan PN, Ahuja N, Moulin P (2017) Robust visual tracking using oblique random forests. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 5825–5834, https://doi.org/10.1109/CVPR.2017.617

  63. Zhang T, Xu C, Yang M (2017) Multi-task correlation particle filter for robust object tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 4819–4827, https://doi.org/10.1109/CVPR.2017.512

  64. Zhang T, Xu C, Yang M (2019) Robust structural sparse tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence 41(2):473–486, https://doi.org/10.1109/TPAMI.2018.2797082

  65. Zhang W, Song K, Rong X, Li Y (2019) Coarse-to-fine uav target tracking with deep reinforcement learning. IEEE Transactions on Automation Science and Engineering 16(4):1522–1530, https://doi.org/10.1109/TASE.2018.2877499

Download references

Acknowledgements

The authors sincerely thank Prof. Zheng Liu from the University of British Columbia for his helps to check and revise the organization and language. Many thanks to the anonymous reviewers and editors for the valuable comments and suggestions. This work is supported by the National Natural Science Foundation of China (No. 61801272 and 61601266).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Mingliang Gao.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xu, L., Gao, M., Li, Q. et al. Visual tracking for UAV using adaptive spatio-temporal regularized correlation filters. Appl Intell 52, 7566–7581 (2022). https://doi.org/10.1007/s10489-021-02825-1

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-02825-1

Keywords

Navigation