Abstract
The advance of visual tracking has provided unmanned aerial vehicle (UAV) with the intriguing capability for various practical applications. With promising performance and efficiency, discriminative correlation filter (DCF)-based trackers have drawn significant attention and undergone remarkable progress. However, the boundary effect and filter degradation remain two intractable problems. In this work, we propose a novel Adaptive Spatio-Temporal Regularized Correlation Filters (ASTR-CF) model to address the two problems. The ASTR-CF model simultaneously optimizes the spatial and temporal regularization weights adaptively, and it is optimized by the alternating direction method of multipliers (ADMM) effectively. Extensive experiments on 4 UAV tracking benchmarks have proven the superiority of the proposed ASTR-CF compared with more than 30 state-of-the-art trackers in terms of accuracy and speed.
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10489-021-02825-1/MediaObjects/10489_2021_2825_Fig1_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10489-021-02825-1/MediaObjects/10489_2021_2825_Fig2_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10489-021-02825-1/MediaObjects/10489_2021_2825_Fig3_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10489-021-02825-1/MediaObjects/10489_2021_2825_Fig4_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10489-021-02825-1/MediaObjects/10489_2021_2825_Fig5_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10489-021-02825-1/MediaObjects/10489_2021_2825_Fig6_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10489-021-02825-1/MediaObjects/10489_2021_2825_Fig7_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10489-021-02825-1/MediaObjects/10489_2021_2825_Fig8_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10489-021-02825-1/MediaObjects/10489_2021_2825_Fig9_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10489-021-02825-1/MediaObjects/10489_2021_2825_Fig10_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10489-021-02825-1/MediaObjects/10489_2021_2825_Fig11_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10489-021-02825-1/MediaObjects/10489_2021_2825_Fig12_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10489-021-02825-1/MediaObjects/10489_2021_2825_Fig13_HTML.png)
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Bertinetto L, Valmadre J, Golodetz S, Miksik O, Torr PHS (2016) Staple: Complementary learners for real-time tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1401–1409, https://doi.org/10.1109/CVPR.2016.156
Bertinetto L, Valmadre J, Henriques JF, Vedaldi A, Torr PHS (2016) Fully-convolutional siamese networks for object tracking. In: Proceedings of the European Conference on Computer Vision Workshops, pp 850–865, https://doi.org/10.1007/978-3-319-48881-3_56
Bibi A, Mueller M, Ghanem B (2016) Target response adaptation for correlation filter tracking. In: Proceedings of the European Conference on Computer Vision, pp 419–433, https://doi.org/10.1007/978-3-319-46466-4_25
Bolme DS, Beveridge JR, Draper BA, Lui YM (2010) Visual object tracking using adaptive correlation filters. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 2544–2550, https://doi.org/10.1109/CVPR.2010.5539960
Boyd S, Parikh N, Chu E (2011) Distributed optimization and statistical learning via the alternating direction method of multipliers. Now Publishers Inc, https://doi.org/10.1561/9781601984616
Dai K, Wang D, Lu H, Sun C, Li J (2019) Visual tracking via adaptive spatially-regularized correlation filters. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 466i5–4674, https://doi.org/10.1109/CVPR.2019.00480
Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, vol 1, pp 886–893, https://doi.org/10.1109/CVPR.2005.177
Danelljan M, Hager G, Khan FS, Felsberg M (2015) Learning spatially regularized correlation filters for visual tracking. In: Proceedings of the IEEE International Conference on Computer Vision, pp 4310–4318, https://doi.org/10.1109/ICCV.2015.490
Danelljan M, Hager G, Shahbaz Khan F, Felsberg M (2016) Adaptive decontamination of the training set: A unified formulation for discriminative visual tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1430–1438, https://doi.org/10.1109/CVPR.2016.159
Danelljan M, Robinson A, Shahbaz Khan F, Felsberg M (2016) Beyond correlation filters: Learning continuous convolution operators for visual tracking. In: Proceedings of the European Conference on Computer Vision, pp 472–488, https://doi.org/10.1007/978-3-319-46454-1_29
Danelljan M, Bhat G, Khan FS, Felsberg M (2017) Eco: Efficient convolution operators for tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 9631–6939, https://doi.org/10.1109/CVPR.2017.733
Danelljan M, Hager̈ G, Khan FS, Felsberg M (2017) Discriminative scale space tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence 39(8):1561–1575, https://doi.org/10.1109/TPAMI.2016.2609928
De Moraes RS, de Freitas EP (2020) Multi-uav based crowd monitoring system. IEEE Transactions on Aerospace and Electronic Systems 56(2):1332–1345, https://doi.org/10.1109/TAES.2019.2952420
Du D, Qi Y, Yu H, Yang Y, Duan K, Li G, Zhang W, Huang Q, Tian Q (2018) The unmanned aerial vehicle benchmark: Object detection and tracking. In: Proceedings of the European Conference on Computer Vision, pp 375–391, https://doi.org/10.1007/978-3-030-01249-6_23
Fan H, Ling H (2017) Parallel tracking and verifying: A framework for real-time and high accuracy visual tracking. In: Proceedings of the IEEE International Conference on Computer Vision, pp 5487–5495, https://doi.org/10.1109/ICCV.2017.585
Fan H, Lin L, Yang F, Chu P, Deng G, Yu S, Bai H, Xu Y, Liao C, Ling H (2019) Lasot: A high-quality benchmark for large-scale single object tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 5369–5378, https://doi.org/10.1109/CVPR.2019.00552
Fu C, Zhang Y, Huang Z, Duan R, Xie Z (2019) Part-based background-aware tracking for uav with convolutional features. IEEE Access 7:79997–80010, https://doi.org/10.1109/ACCESS.2019.2922703
Fu C, Xu J, Lin F, Guo F, Liu T, Zhang Z (2020) Object saliency-aware dual regularized correlation filter for real-time aerial tracking. IEEE Transactions on Geoscience and Remote Sensing 58(12):8940–8951, https://doi.org/10.1109/TGRS.2020.2992301
Galoogahi HK, Fagg A, Lucey S (2017) Learning background-aware correlation filters for visual tracking. In: Proceedings of the IEEE International Conference on Computer Vision, pp 1145–1152, https://doi.org/10.1109/ICCV.2017.129
Gao P, Yuan R, Wang F, Xiao L, Fujita H, Zhang Y (2020) Siamese attentional keypoint network for high performance visual tracking. Knowledge-Based Systems 193:105448, https://doi.org/10.1016/j.knosys.2019.105448
Gao P, Zhang Q, Wang F, Xiao L, Fujita H, Zhang Y (2020) Learning reinforced attentional representation for end-to-end visual tracking. Information Sciences 517:52–67, https://doi.org/10.1016/j.ins.2019.12.084
Han Z, Wang P, Ye Q (2020) Adaptive discriminative deep correlation filter for visual object tracking. IEEE Transactions on Circuits and Systems for Video Technology 30(1):155–166, https://doi.org/10.1109/TCSVT.2018.2888492
He Z, Fan Y, Zhuang J, Dong Y, Bai H (2017) Correlation filters with weighted convolution responses. In: Proceedings of the International Conference on Computer Vision Workshops, pp 1992–2000, https://doi.org/10.1109/ICCVW.2017.233
Held D, Thrun S, Savarese S (2016) Learning to track at 100 fps with deep regression networks. In: Proceedings of the European Conference on Computer Vision, pp 749–765, https://doi.org/10.1007/978-3-319-46448-0_45
Henriques JF, Caseiro R, Martins P, Batista J (2012) Exploiting the circulant structure of tracking-by-detection with kernels. In: Proceedings of the European Conference on Computer Vision, pp 702–715, https://doi.org/10.1007/978-3-642-33765-9_50
Henriques JF, Caseiro R, Martins P, Batista J (2015) High-speed tracking with kernelized correlation filters. IEEE Transactions on Pattern Analysis and Machine Intelligence 37(3):583–596, https://doi.org/10.1109/TPAMI.2014.2345390
Huang Z, Fu C, Li Y, Lin F, Lu P (2019) Learning aberrance repressed correlation filters for real-time uav tracking. In: Proceedings of the International Conference on Computer Vision, pp 2891–2900, https://doi.org/10.1109/ICCV.2019.00298
Kristan M, Matas J, Leonardis A, Vojir T, Pflugfelder R, Fernandez G, Nebehay G, Porikli F, ÚCCehovin L (2016) A novel performance evaluation methodology for single-target trackers. IEEE Transactions on Pattern Analysis and Machine Intelligence 38(11):2137–2155, https://doi.org/10.1109/TPAMI.2016.2516982
Li F, Yao Y, Li P, Zhang D, Zuo W, Yang M (2017) Integrating boundary and center correlation filters for visual tracking with aspect ratio variation. In: Proceedings of the IEEE International Conference on Computer Vision Workshop, pp 2001–2009, https://doi.org/10.1109/ICCVW.2017.234
Li F, Tian C, Zuo W, Zhang L, Yang M (2018) Learning spatial-temporal regularized correlation filters for visual tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 4904–4913, https://doi.org/10.1109/CVPR.2018.00515
Li S, Yeung DY (2017) Visual object tracking for unmanned aerial vehicles: A benchmark and new motion models. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp 4140–4146
Li X, Liu Q, He Z, Wang H, Zhang C, Chen WS (2016) A multi-view model for visual tracking via correlation filters. Knowledge-Based Systems 113:88–99, https://doi.org/10.1016/j.knosys.2016.09.014
Li Y, Fu C, Ding F, Huang Z, Lu G (2020) Autotrack: Towards high-performance visual tracking for uav with automatic spatio-temporal regularization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 11923–11932, https://doi.org/10.1109/CVPR42600.2020.01194
Li Y, Fu C, Huang Z, Zhang Y, Pan J (2021) Intermittent contextual learning for keyfilter-aware uav object tracking using deep convolutional feature. IEEE Transactions on Multimedia 23:810–822, https://doi.org/10.1109/TMM.2020.2990064
Liu T, Wang G, Yang Q (2015) Real-time part-based visual tracking via adaptive correlation filters. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 4902–4912, https://doi.org/10.1109/CVPR.2015.7299124
Lukezic A, Vojir T, CehovinZajc L, Matas J, Kristan M (2018) Discriminative correlation filter tracker with channel and spatial reliability. International Journal of Computer Vision 126(7):671–688, https://doi.org/10.1007/s11263-017-1061-3
Ma C, Huang JB, Yang X, Yang MH (2019) Robust visual tracking via hierarchical convolutional features. IEEE Transactions on Pattern Analysis and Machine Intelligence 41(11):2709–2723, https://doi.org/10.1109/TPAMI.2018.2865311
Mueller M, Smith N, Ghanem B (2016) A benchmark and simulator for uav tracking. In: Proceedings of the European Conference on Computer Vision, pp 445–461, https://doi.org/10.1007/978-3-319-46448-0_27
Mueller M, Smith N, Ghanem B (2017) Context-aware correlation filter tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1387–1395, https://doi.org/10.1109/CVPR.2017.152
Nam H, Han B (2016) Learning multi-domain convolutional neural networks for visual tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 4293–4302, https://doi.org/10.1109/CVPR.2016.465
Padhy RP, Xia F, Choudhury SK, Sa PK, Bakshi S (2019) Monocular vision aided autonomous uav navigation in indoor corridor environments. IEEE Transactions on Sustainable Computing 4(1):96–108, https://doi.org/10.1109/TSUSC.2018.2810952
Qi Y, Zhang S, Qin L, Yao H, Huang Q, Lim J, Yang M (2016) Hedged deep tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 4303–4311, https://doi.org/10.1109/CVPR.2016.466
Rey N, Volpi M, Joost S, Tuia D (2017) Detecting animals in african savanna with uavs and the crowds. Remote Sensing of Environment 200:341–351, https://doi.org/10.1016/j.rse.2017.08.026
Song Y, Ma C, Gong L, Zhang J, Lau RWH, Yang M (2017) Crest: Convolutional residual learning for visual tracking. In: Proceedings of the IEEE International Conference on Computer Vision, pp 2574–2583, https://doi.org/10.1109/ICCV.2017.279
Sun Y, Sun C, Wang D, He Y, Lu H (2019) Roi pooled correlation filters for visual tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 5776–5784, https://doi.org/10.1109/CVPR.2019.00593
Tang Y, Hu Y, Cui J, Liao F, Lao M, Lin F, Teo RSH (2019) Vision-aided multi-uav autonomous flocking in gps-denied environment. IEEE Transactions on Industrial Electronics 66(1):616–626, https://doi.org/10.1109/TIE.2018.2824766
Valmadre J, Bertinetto L, Henriques J, Vedaldi A, Torr PHS (2017) End-to-end representation learning for correlation filter based tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 5000–5008, https://doi.org/10.1109/CVPR.2017.531
van de Weijer J, Schmid C, Verbeek J, Larlus D (2009) Learning color names for real-world applications. IEEE Transactions on Image Processing 18(7):1512–1523, https://doi.org/10.1109/TIP.2009.2019809
Voigtlaender P, Luiten J, Torr PHS, Leibe B (2020) Siam r-cnn: Visual tracking by re-detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, https://doi.org/10.1109/CVPR42600.2020.00661
Wang C, Wang J, Shen Y, Zhang X (2019) Autonomous navigation of uavs in large-scale complex environments: A deep reinforcement learning approach. IEEE Transactions on Vehicular Technology 68(3):2124–2136, https://doi.org/10.1109/TVT.2018.2890773
Wang L, Ouyang W, Wang X, Lu H (2016) Stct: Sequentially training convolutional networks for visual tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1373–1381, https://doi.org/10.1109/CVPR.2016.153
Wang M, Liu Y, Huang Z (2017) Large margin object tracking with circulant feature maps. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 4800–4808, https://doi.org/10.1109/CVPR.2017.510
Wang N, Zhou W, Tian Q, Hong R, Wang M, Li H (2018) Multi-cue correlation filters for robust visual tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 4844–4853, https://doi.org/10.1109/CVPR.2018.00509
Wang N, Song Y, Ma C, Zhou W, Liu W, Li H (2019) Unsupervised deep tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1308–1317, https://doi.org/10.1109/CVPR.2019.00140
Wang W, Zhang K, Lv M, Wang J (2020) Hierarchical spatiotemporal context-aware correlation filters for visual tracking. IEEE Transactions on Cybernetics pp 1–14, https://doi.org/10.1109/TCYB.2020.2964757
Wen L, Zhu P, Du D, et al (2019) Visdrone-sot2018: The vision meets drone single-object tracking challenge results. In: Proceedings of the European Conference on Computer Vision, pp 469–495, https://doi.org/10.1007/978-3-030-11021-5_28
Wu Y, Lim J, Yang MH (2015) Object tracking benchmark. IEEE Transactions on Pattern Analysis and Machine Intelligence 37(9):1834–1848, https://doi.org/10.1109/TPAMI.2014.2388226
Xu T, Feng ZH, Wu XJ, Kittler J (2019) Learning adaptive discriminative correlation filters via temporal consistency preserving spatial feature selection for robust visual object tracking. IEEE Transactions on Image Processing 28(11):5596–5609, https://doi.org/10.1109/TIP.2019.2919201
Xu Y, Wang Z, Li Z, Yuan Y, Yu G (2020) Siamfc++: Towards robust and accurate visual tracking with target estimation guidelines. In: Proceedings of the Association for the Advance of Artificial Intelligence, https://doi.org/10.1609/aaai.v34i07.6944
Yun S, Choi J, Yoo Y, Yun K, Choi JY (2017) Action-decision networks for visual tracking with deep reinforcement learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1349–1358, https://doi.org/10.1109/CVPR.2017.148
Zahran S, Moussa AM, Sesay AB, El-Sheimy N (2019) A new velocity meter based on hall effect sensors for uav indoor navigation. IEEE Sensors Journal 19(8):3067–3076, https://doi.org/10.1109/JSEN.2018.2890094
Zhang L, Varadarajan J, Suganthan PN, Ahuja N, Moulin P (2017) Robust visual tracking using oblique random forests. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 5825–5834, https://doi.org/10.1109/CVPR.2017.617
Zhang T, Xu C, Yang M (2017) Multi-task correlation particle filter for robust object tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 4819–4827, https://doi.org/10.1109/CVPR.2017.512
Zhang T, Xu C, Yang M (2019) Robust structural sparse tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence 41(2):473–486, https://doi.org/10.1109/TPAMI.2018.2797082
Zhang W, Song K, Rong X, Li Y (2019) Coarse-to-fine uav target tracking with deep reinforcement learning. IEEE Transactions on Automation Science and Engineering 16(4):1522–1530, https://doi.org/10.1109/TASE.2018.2877499
Acknowledgements
The authors sincerely thank Prof. Zheng Liu from the University of British Columbia for his helps to check and revise the organization and language. Many thanks to the anonymous reviewers and editors for the valuable comments and suggestions. This work is supported by the National Natural Science Foundation of China (No. 61801272 and 61601266).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Xu, L., Gao, M., Li, Q. et al. Visual tracking for UAV using adaptive spatio-temporal regularized correlation filters. Appl Intell 52, 7566–7581 (2022). https://doi.org/10.1007/s10489-021-02825-1
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-021-02825-1