Skip to main content
SpringerLink
Log in

UAV tracking based on saliency detection

  • Methodologies and Application
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

This paper presents a novel unmanned aerial vehicle tracking framework. First, hierarchical convolutional neural network features are used to track the object independently in a correlation filter tracking framework. Second, a stability criterion is proposed, which is based on the variance of tracking results of each layer. Next, tracking result is adaptively fused via the variance. Meanwhile, the criterion can be used to measure the quality of tracking results. A saliency detection method is utilized to generate candidate regions when tracking failure occurs. By virtue of this method, our tracking algorithm can robustly cope with appearance changes and prevent drifting issues. Experimental results show that our proposed tracking algorithm performs favorably against state-of-the-art methods on two benchmark datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  • Babenko B, Yang M-H, Belongie S (2011) Robust object tracking with online multiple instance learning. IEEE Trans Pattern Anal Mach Intell 33(8):1619–1632

    Article  Google Scholar 

  • Bao C, Wu Y, Ling H, Ji H (2012) Real time robust l1 tracker using accelerated proximal gradient approach. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1830–1837

  • Bolme DS, Beveridge JR, Draper BA, Lui YM (2010) Visual object tracking using adaptive correlation filters. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2544–2550

  • Collins R, Zhou X, Teh SK (2005) An open source tracking test bed and evaluation web site. In: IEEE international workshop on performance evaluation of tracking and surveillance (PETS 2005), vol 2(6)

  • Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: international conference on computer vision pattern recognition (CVPR’05), vol. 1. IEEE Computer Society, pp 886–893

  • Danelljan M, Hger G, Shahbaz Khan F, Felsberg M (2014a) Accurate scale estimation for robust visual tracking. In: BMVC

  • Danelljan M, Khan FS, Felsberg M, Weijer J (2014b) Adaptive color attributes for real-time visual tracking. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1090–1097

  • Danelljan M, Hager G, Khan FS, Felsberg M (2015) Learning spatially regularized correlation filters for visual tracking. In: Proceedings of the IEEE international conference on computer vision, pp 4310–4318

  • Danelljan M, Hager G, Khan FS, Felsberg M (2016a) Discriminative scale space tracking. IEEE Trans Pattern Anal Mach Intell 39(8):1561–1575

    Article  Google Scholar 

  • Danelljan M, Robinson A, Shahbaz Khan F, Felsberg M (2016b) Beyond correlation filters: learning continuous convolution operators for visual tracking, In: The European conference on computer vision, pp 472–488

  • Danelljan M, Bhat G, Khan FS, Felsberg M (2017) ECO: efficient convolution operators for tracking. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 6638–6646

  • de Jesus RJ (2018) Discrete time control based in neural networks for pendulums. Appl Soft Comput 68:821–832

    Article  Google Scholar 

  • de Jesus RJ, Garcia E, Aquino G, Aguilar-Ibanez C, Pacheco J, Zacarias A (2018) Learning of operator hand movements via least angle regression to be teached in a manipulator. Evol Syst. https://doi.org/10.1007/s12530-018-9224-1

    Article  Google Scholar 

  • Fu C, Carrio A, Olivares-Mendez M, Suarez-Fernandez R, Campoy P (2014) Robust real-time vision-based aircraft tracking from unmanned aerial vehicles. In: 2014 IEEE international conference on robotics and automation (ICRA), pp 5441–5446

  • Garcia Sanchez J, Tavera-Mosqueda S, Silva-Ortigoza R, Hernandez-Guzman V, Sandoval-Gutierrez J, Marcelino-Aranda M, Taud H, Marciano-Melchor M (2018) Robust switched tracking control for wheeled mobile robots considering the actuators and drivers. Sensors 18(12):4316

    Article  Google Scholar 

  • Grabner H, Grabner M, Bischof H (2006) Real-time tracking via on-line boosting. BMVC 1:1–6

    Google Scholar 

  • Hare S, Saffari A, Torr PHS (2015) Struck: structured output tracking with kernels. IEEE Trans Pattern Anal Mach Intell 38(10):2096–2109

    Article  Google Scholar 

  • He W, Dong Y (2017) Adaptive fuzzy neural network control for a constrained robot using impedance learning. IEEE Trans Neur Netw Learn Syst 29(4):1174–1186

    Article  Google Scholar 

  • Henriques JF, Caseiro R, Martins P, Batista J (2012) Exploiting the circulant structure of tracking-by-detection with kernels. In: The European conference on computer vision, pp 702–715

  • Henriques JF, Caseiro R, Martins P, Batista J (2015) Highspeed tracking with kernelized correlation filters. IEEE Trans Pattern Anal Mach Intell 37(3):583–596

    Article  Google Scholar 

  • Hong Z, Chen Z, Wang C, Mei X, Prokhorov D, Tao D (2015a) Multi-store tracker (muster): a cognitive psychology inspired approach to object tracking. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp 749–758

  • Hong S, You T, Kwak S, Han B (2015b) Online tracking by learning discriminative saliency map with convolutional neural network. In: International conference on machine learning, pp 597–606

  • Hua Y, Alahari K, Schmid C (2014) Occlusion and motion reasoning for long-term tracking. In: the European conference on computer vision, pp 172–187

  • Jia X, Lu H, Yang MH (2012) Visual tracking via adaptive structural local sparse appearance model. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1822–1829

  • Kalal Z, Mikolajczyk K, Matas J (2012) Tracking-learning detection. IEEE Trans Pattern Anal Mach Intell 34(7):1409–1422

    Article  Google Scholar 

  • Kendall A, Salvapantula N, Stol K (2014) On-board object tracking control of a quadcopter with monocular vision. In: 2014 international conference on unmanned aircraft systems (ICUAS), pp 404–411

  • Kristan M, Pflugfelder R, Leonardis A, Matas J, Cehovin L, Nebehay G, Vojir T, Fernandez G, Lukezic A, Dimitriev A et al (2014) The visual object tracking VOT2014 challenge results. In: The European conference on computer vision workshops. Springer, pp 191–217

  • Kristan M, Matas J, Leonardis A, Felsberg M, Cehovin L, Fernandez G, Vojir T, Hager G, Nebehay G, Pflugfelder R (2015) The visual object tracking VOT2015 challenge results. In: Proceedings of the IEEE international conference on computer vision workshop, pp 1–23

  • Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Neural information processing systems (NeurIPS), pp 1097–1175

  • Li S, Yeung D-Y (2017) Visual object tracking for unmanned aerial vehicles: a benchmark and new motion models. In: Association for the advancement of artificial intelligence, pp 4140–4146

  • Li Y, Zhu J (2014) A scale adaptive kernel correlation filter tracker with feature integration. In: the European conference on computer vision workshop, pp 254–265

  • Li A, Lin M, Wu Y, Yang M, Yan S (2016) NUS-PRO: a new visual tracking challenge. IEEE Trans Pattern Anal Mach Intell 38(2):335–349

    Article  Google Scholar 

  • Liang P, Blasch E, Ling H (2015) Encoding color information for visual tracking: algorithms and benchmark. IEEE Trans Image Process 24(12):5630–5644

    Article  MathSciNet  Google Scholar 

  • Lim H, Sinha SN (2015) Monocular localization of a moving person onboard a quadrotor MAV. In: 2015 IEEE international conference on robotics and automation (ICRA), pp 2182–2189

  • Ma C, Huang J-B, Yang X, Yang M-H (2015) Hierarchical convolutional features for visual tracking. In: Proceedings of the IEEE international conference on computer vision, pp 3074–3082

  • Mei X, Ling H (2009) Robust visual tracking using l1 minimization. In: Proceedings of the IEEE international conference on computer vision, pp 1436–1443

  • Mueller M, Smith N, Ghanem B (2016a) A benchmark and simulator for UAV tracking. In: the European conference on computer vision, pp 445–461

  • Mueller M, Sharma G, Smith N, Ghanem B (2016b) Persistent aerial tracking system for UAVs. In: 2016 IEEE/RSJ international conference on intelligent robots and systems (IROS)

  • Nam H, Han B (2016) Learning multi-domain convolutional neural networks for visual tracking. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4293–4302

  • Nussberger A, Grabner H, Van Gool L (2014) Aerial object tracking from an airborne platform. In: 2014 International conference on unmanned aircraft systems (ICUAS), pp 1284–1293

  • Pestana J, Sanchez-Lopez J, Campoy P, Saripalli S (Oct 2013) Vision based GPS-denied object tracking and following for unmanned aerial vehicles. In: 2013 IEEE international symposium on safety, security, and rescue robotics (SSRR), pp 1–6

  • Possegger H, Mauthner T, Bischof H (2015) In defense of color-based model-free tracking. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2113–2120

  • Qadir A, Neubert J, Semke W, Schultz R (2011) On-board visual tracking with unmanned aircraft system (UAS). In: Aerospace conferences, American Institute of Aeronautics and Astronautics, p 1503

  • Qi Y, Zhang S, Qin L, Yao H, Huang Q, Lim J, Yang M-H (2016) Hedged deep tracking. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4303–4311

  • Ross DA, Lim J, Lin R-S, Yang M-H (2008) Incremental learning for robust visual tracking. Int J Comput Vis (IJCV) 77(1–3):125–141

    Article  Google Scholar 

  • Rubio JJ (2012) Modified optimal control with a back propagation network for robotic arms. IET Control Theory Appl 6(14):2216–2225

    Article  MathSciNet  Google Scholar 

  • Seo H, Milanfar P (2010) Visual saliency for automatic target detection, boundary detection, and image quality assessment. In: The international conference on acoustics, speech, and signal processing, pp 5578–5581

  • Seo HJ, Milanfar P (2011) Face verification using the lark representation. IEEE Trans Inf Forensics Secur 6(4):1275–1286

    Article  Google Scholar 

  • Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. In: The international conference on learning representations (ICLR)

  • Smeulders AW, Chu DM, Cucchiara R, Calderara S, Dehghan A, Shah M (2014) Visual tracking: an experimental survey. IEEE Trans Pattern Anal Mach Intell 36(7):1442–1468

    Article  Google Scholar 

  • Song S, Xiao J (2013) Tracking revisited using RGBD camera: unified benchmark and baselines. In: Proceedings of the IEEE international conference on computer vision. IEEE, pp 233–240

  • Sun Z, Wang Y, Laganiere R (2018) Online model adaptation for UAV tracking with convolutional neural network. In: Computer and robot vision, pp 329–336

  • Sun Z, Wang Y, Laganiere R (2019) Hard negative mining for correlation filters in visual tracking. Mach Vis Appl 30(3):487–506

    Article  Google Scholar 

  • Vedaldi A, Lenc K (2015) Matconvnet: convolutional neural networks for matlab. In: Proceedings of the 23rd ACM international conference on multimedia, pp 689–692

  • Wang N, Yeung D-Y (2013) Learning a deep compact image representation for visual tracking. In: Neural information processing systems (NeurIPS), pp 809–817

  • Wang Y, Hu S, Wu S (2014) Visual tracking based on group sparsity learning. Mach Vis Appl 26:127–139

    Article  Google Scholar 

  • Wang N, Li S, Gupta A, Yeung D (2015a) Transferring rich feature hierarchies for robust visual tracking. arXiv preprint: arXiv:1501.04587

  • Wang L, Ouyang W, Wang X, Lu H (2015b) Visual tracking with fully convolutional networks. In: Proceedings of the IEEE international conference on computer vision, pp 3119–3127

  • Wang N, Shi J, Yeung D-Y, Jia J (2015c) Understanding and diagnosing visual tracking systems. In: Proceedings of the IEEE international conference on computer vision, pp 3101–3109

  • Wu Y, Lim J, Yang M-H (2013) Online object tracking: a benchmark. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2411–2418

  • Wu Y, Lim J, Yang M-H (2015) Object tracking benchmark. IEEE Trans Pattern Anal Mach Intell 37(9):1834–1848

    Article  Google Scholar 

  • Yen VT, Nan WY, Van Cuong P (2018) Recurrent fuzzy wavelet neural networks based on robust adaptive sliding mode control for industrial robot manipulators. Neural Comput Appl. https://doi.org/10.1007/s00521-018-3520-3

    Article  Google Scholar 

  • Zhang T, Ghanem B, Liu S, Ahuja N (2012a) Robust visual tracking via multi-task sparse learning. In: IEEE conference on computer vision and pattern recognition, pp 1–8

  • Zhang K, Zhang L, Yang M-H (2012b) Real-time compressive tracking. In the European conference on computer vision, pp 864–871

  • Zhang J, Ma S, Sclaroff S (2014) MEEM: robust tracking via multiple experts using entropy minimization. In: the European conference on computer vision, pp 188–203

  • Zhu G, Wang J, Wu Y, Zhang X, Lu H (2016) MC-HOG correlation tracking with saliency proposal. In: Association for the advancement of artificial intelligence, pp 3690–3696

Download references

Acknowledgements

This work was partially supported by the Chinese Academy of Science Pioneer Hundred Talents Program (Type C) under Grant No. 2017-122, National Science Fund for Young Scholars under Grant No. 61806186 and National Natural Science Foundation of China (No. 61873246). We thank the anonymous editor and reviewers for their careful reading and many insightful comments and suggestions.

Author information

Authors and Affiliations

  1. School of Aeronautics and Astronautics, Shanghai Jiao Tong University, Shanghai, China

    Yong Wang & Lingkun Luo

  2. School of Electrical Engineering and Computer Science, University of Ottawa, Ottawa, Canada

    Yong Wang

  3. School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University, Shanghai, China

    Xinbin Luo

  4. College of Electric and Information Engineering, Zhengzhou University of Light Industry, Zhengzhou, China

    Huanlong Zhang

  5. Fujian Institute of Research on the Structure of Matter, Chinese Academy of Sciences, Fuzhou, China

    Xian Wei

Authors
  1. Yong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xinbin Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lingkun Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Huanlong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xian Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xinbin Luo.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest regarding the publication of this paper.

Human and animal rights

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Communicated by V. Loia.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, Y., Luo, X., Luo, L. et al. UAV tracking based on saliency detection. Soft Comput 24, 12149–12162 (2020). https://doi.org/10.1007/s00500-019-04652-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-019-04652-6

Keywords

Search

Navigation