Skip to main content
SpringerLink
Log in

Study of UAV tracking based on CNN in noisy environment

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Recently, there are lots of tracking methods proposed to improve the performance of visual tracking in videos with challenging situations, such as background clutter, severe occlusion, rotation, and so on. In real unmanned aerial vehicle (UAV) based tracking systems, there are various noises occurring during video capturing, transmission, and processing. However, most existing studies pay attention to improve the robustness and accuracy of visual tracking while ignoring the performance of tracking methods on videos with noise. In this paper, we investigate the performance evaluation of existing tracking methods on videos with noise. A group of noisy UAV based tracking video datasets are constructed and used to the benchmark datasets for analysis of tracking methods. Furthermore, we propose an algorithm for robustness tracking in noisy videos. The performance of 9 tracking methods is evaluated on the proposed dataset. We provide the detailed analysis and discussion on the robustness analysis of different tracking methods on videos with different variance of noises. Our investigation shows that it is still challenging for effective tracking for existing methods on videos with noise. And our proposed method shows promising results in noisy videos.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

References

  1. Bertinetto L, Valmadre J, Golodetz S, Miksik O, Torr PH (2016) Staple: Complementary learners for real-time tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition

  2. Bertinetto L, Valmadre J, Golodetz S, Miksik O, Torr PHS (2016) Staple: Complementary learners for real-time tracking. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp 1401–1409

  3. Bolme D, Beveridge J, Draper B, Lui Y (2010) Visual object tracking using adaptive correlation filters. In: Proc. of IEEE conference on computer vision and pattern recognition, pp. 2544–2550

  4. Chen Z, Hong Z, Tao D (2015) An experimental survey on correlation filter-based tracking. arXiv preprint arXiv:1509.05520

  5. Danelljan M, Bhat G, Shahbaz Khan F, Felsberg M (2017) Eco: Efficient convolution operators for tracking. In: IEEE CVPR

  6. Danelljan M, Hager G, Khan F, Felsberg M (2014) Accurate scale estimation for robust visual tracking. In: Proceedings of British Machine Vision Conference

  7. Danelljan M, Hager G, Khan F S, Felsberg M (2016) Adaptive decontamination of the training set: a unified formulation for discriminative visual tracking. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp 1430–1438

  8. Danelljan M, Hager G, Khan FS, Felsberg M (2017) Discriminative scale space tracking. IEEE transactions on pattern analysis and machine intelligence

  9. Danelljan M, Hager G, Shahbaz Khan F, Felsberg M (2015) Learning spatially regularized correlation filters for visual tracking. In: Proceedings of IEEE International Conference on Computer Vision, pp 4310–4318

  10. Danelljan M, Hager G, Shahbaz Khan F, Felsberg M (2015) Convolutional features for correlation filter based visual tracking. In: Proceedings of IEEE International Conference on Computer Vision Workshops, pp 58–66

  11. Du D, Qi Y, Yu H, Yang Y, Duan K, Li G, Zhang W, Huang Q, Tian Q (2018) The Unmanned Aerial Vehicle Benchmark: Object Detection and Tracking. arXiv preprint arXiv:1804.00518

  12. Fan H, Lin L, Yang F, Chu P, Deng G, Yu S, Bai H, Xu Y, Liao C, Ling H (2018) LaSOT: A High-quality Benchmark for Large-scale Single Object Tracking, . arXiv preprint arXiv:1809.07845

  13. Fang Y, Yuan Y, Li L, Jinjian W, Lin W, Li Z (2017) Performance evaluation of visual tracking algorithms on video sequences with quality degradation. IEEE Access 5:2430–2441

    Article  Google Scholar 

  14. Fiaz M, Mahmood A, Jung SK (2018) Tracking Noisy Targets: A Review of Recent Object Tracking Approaches. arXiv preprint arXiv:1802.03098

  15. Fu C, Carrio A, Olivares-Mendez M, Suarez-Fernandez R, Campoy P (2014) Robust real-time vision-based aircraft tracking from unmanned aerial vehicles. In: Robotics and automation (ICRA), IEEE international conference on, pp. 5441-5446

  16. Galoogahi HK, Fagg A, Huang C, Ramanan D, Lucey S (2017) Need for speed: A benchmark for higher frame rate object tracking. In: ICCV

  17. Gaszczak A, Breckon TP, Han J (2011) Real-time people and vehicle detection from uav imagery. In: Roning J., Casasent D. P., Hall E. L. (eds) IST/SPIE Electronic imaging. vol. 7878, pp. 78780b-13, International Society for Optics and Photonics

  18. Hendrycks D, Dietterich T (2019) Benchmarking neural network robustness to common corruptions and perturbations ICLR

  19. Henriques JF, Caseiro R, Martins P, Batista J (2012) Exploiting the circulant structure of tracking-by-detection with kernels. In: proceedings of European Conference on Computer Vision, pp 702–715

  20. Henriques JF, Caseiro R, Martins P, Batista J (2015) High-speed tracking with kernelized correlation filters. IEEE Transactions on Pattern Analysis and Machine Intelligence 37(3):583–596

    Article  Google Scholar 

  21. Kendall A, Salvapantula N, Stol K (2014) On-board object tracking control of a quadcopter with monocular vision. In: Unmanned aircraft systems (ICUAS), international conference on, pp. 404-411

  22. Kiani Galoogahi H, Fagg A, Lucey S (2017) Learning background aware correlation filters for visual tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1135–1143

  23. Kristan M, Matas J, Leonardis A, Vojr T, Pflugfelder R, Fernandez G, Nebehay G, Porikli F, Cehovin L (2016) A novel performance evaluation methodology for single-target trackers. TPAMI 38(11):2137–2155

    Article  Google Scholar 

  24. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Proceedings of Advances in Neural Information Processing Systems, pp 1097–1105

  25. Li A, Lin M, Wu Y, Yang M-H, Yan S (2016) Nus-pro: a new visual tracking challenge. TPAMI 38(2):335–349

    Article  Google Scholar 

  26. Li F, Tian C, Zuo W, Zhang L, Yang M-H (2018) Learning Spatial-Temporal Regularized Correlation Filters for Visual Tracking. arXiv preprint arXiv:1803.08679

  27. Li X, Weiming H, Shen C, Zhang Z, Dick A, Van Den Hengel A (2013) A survey of appearance models in visual object tracking. ACM transactions on Intelligent Systems and Technology (TIST) 4(4):58

    Google Scholar 

  28. Li S, Yeung D-Y (2017) Visual object tracking for unmanned aerial vehicles: a benchmark and new motion models. In: AAAI, pp 4140–4146

  29. Liang P, Blasch E, Ling H (2015) Encoding color information for visual tracking: Algorithms and benchmark. TIP 24(12):5630–5644

    MathSciNet  MATH  Google Scholar 

  30. Lim H, Sinha SN (2015) Monocular localization of a moving person onboard a quadrotor mav. In: Robotics and automation (ICRA), IEEE international conference on, pp. 2182–2189

  31. Ma C, Huang J-B, Yang X, Yang M-H (2015) Hierarchical convolutional features for visual tracking. In: Proceedings of IEEE International Conference on Computer Vision, pp 3074–3082

  32. Madry A, et al. (2018) Towards deep learning models resistant to adversarial attacks ICLR

  33. Mueller M, Sharma G, Smith N, Ghanem B (2016) Persistent aerial tracking system for uavs. In: Intelligent robots and systems (IROS), 2016 IEEE/RSJ international conference

  34. Mueller M, Smith N, Ghanem B (2016) A benchmark and simulator for uav tracking. In: ECCV

  35. Muller M, Bibi A, Giancola S, Al-Subaihi S, Ghanem B (2018) Trackingnet: A large-scale dataset and benchmark for object tracking in the wild. In: ECCV

  36. Naseer T, Sturm J, Cremers D (Nov 2013) Follow me: Person following and gesture recognition with a quadrocopter. In: Intelligent robots and systems (IROS), 2013 IEEE/RSJ international conference on, pp. 624–630

  37. Nussberger A, Grabner H, Van Gool L (2014) Aerial object tracking from an airborne platform. In: Unmanned aircraft systems (ICUAS), international conference on, pp. 1284-1293

  38. Pestana J, Sanchez-Lopez J, Campoy P, Saripalli S (2013) Vision based gps-denied object tracking and following for unmanned aerial vehicles. In: Safety, security, and rescue robotics (SSRR), IEEE international symposium on, pp. 1–6

  39. Pollard T, Antone M (2012) Detecting and tracking all moving objects in wide-area aerial video. In: Computer vision and pattern recognition workshops (CVPRW), 2012 IEEE computer society conference on, pp. 15–22

  40. Portmann J, Lynen S, Chli M, Siegwart R (2014) People detection and tracking from aerial thermal views. In: Robotics and automation (ICRA), 2014 IEEE international conference on, pp. 1794–1800

  41. Prokaj J, Medioni G (2014) Persistent tracking for wide area aerial surveillance. In: Computer vision and pattern recognition (CVPR), 2014 IEEE conference on, pp. 1186–1193

  42. Qadir A, Neubert J, Semke W, Schultz R (2011) Chap, On-Board Visual Tracking With Unmanned Aircraft System (UAS), Infotech@Aerospace Conferences, American Institute of Aeronautics and Astronautics

  43. Qi Y, Zhang S, Qin L, Yao H, Huang Q, Lim J, Yang M-H (2016) Hedged deep tracking in. In: IEEE CVPR

  44. Real E, Shlens J, Mazzocchi S, Pan X, Vanhoucke V (2017) Youtube-bounding boxes: A large high-precision human-annotated data set for object detection in video. In: CVPR

  45. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition International Conference on Learning Representations

  46. Smeulders AW, Chu DM, Cucchiara R, Calderara S, Dehghan A, Shah M (2014) Visual tracking: an experimental survey. TPAMI 36 (7):1442–1468

    Article  Google Scholar 

  47. Solis Montero A, Lang J, Laganiere R (2015) Scalable Kernel Correlation Filter With Sparse Feature Integration. In: IEEE International Conference on Computer Vision Workshop on Visual Object Tracking (VOT2015) pp 24–31, Santiago, Chile

  48. Sun Z, Wang Y, Laganiere R (2018) Online Model Adaptation for UAV Tracking With Convolutional Neural Network 15th Conference on Computer and Robot Vision (CRV)

  49. Tramer F, Kurakin A, Papernot N, Goodfellow I, Boneh D, McDaniel P (2018) Ensemble adversarial training: Attacks and defenses ICLR

  50. Valmadre J, Bertinetto L, Henriques JF, Tao R, Vedaldi A, Smeulders A, Torr P, Gavves E (2018) Longterm tracking in the wild: A benchmark. In: ECCV

  51. Wang F, Chen L, Li C, Huang S, Chen Y, Qian C, Loy CC (2018) The devil of face recognition is in the noise ECCV

  52. Wang N, Shi J, Yeung D-Y, Jia J (2015) Understanding and diagnosing visual tracking systems. In: Proceedings of the IEEE International Conference on Computer Vision, pp 3101–3109

  53. Wang X, Zheng Z, He Y, Yan F, Zeng Z, Yang Y (2020) Progressive Local Filter Pruning for Image Retrieval Acceleration arxiv: Computer Vision and Pattern Recognition

  54. Wang N, Zhou W, Li H (2018) Reliable Re-detection for Long-term Tracking. IEEE Transactions on Circuits and Systems for Video Technology

  55. Wang N, Zhou W, Qi T, Hong R, Wang M, Li H (2018) Multi-Cue Correlation filters for robust visual tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 4844–4853

  56. Wu Y, Lim J, Yang M-H (2013) Online object tracking: A benchmark. In: CVPR

  57. Wu Y, Lim J, Yang M-H (2015) Object tracking benchmark. TPAMI 37(9):1834–1848

    Article  Google Scholar 

  58. Yilmaz A, Javed O, Shah M (2006) Object tracking: A survey, ACM CSUR, vol. 38, no. 4, Article 13

  59. Zhang T, Xu C, Yang M-H (2017) Multi-task correlation particle filter for robust object tracking. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

  60. Zhang K, Zhang L, Yang M-H, Zhang D (2013) Fast visual tracking via dense spatio-temporal context learning. In: Proceedings of European Conference on Computer Vision, pp 127–141

  61. Zhou Y, Liu D, Huang T (2018) Survey of Face Detection on Low-quality Images. In: Automatic Face Gesture Recognition (FG 2018), 13th IEEE International Conference on, pp. 769–773. IEEE

Download references

Acknowledgment

This research is partially supported by NSF of China (No: 61602246), NSF of Jiangsu Province (No: BK20171430), the Fundamental Research Funds for the Central Universities (No: 30918011319), and the “Summit of the Six Top Talents” Program (No: DZXX-027).

Author information

Authors and Affiliations

  1. School of Aeronautics and Astronautics, Shanghai Jiao Tong University, Shanghai, China

    Zhuojin Sun

  2. School of Aeronautics and Astronautics, Sun Yat-Sen University, Guangzhou, China

    Yong Wang

  3. The PCA Lab, the Key Laboratory of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education, School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, 210094, P.R. China

    Chen Gong

  4. School of Electrical Engineering and Computer Science, University of Ottawa, Ottawa, Canada

    Robert Laganiére

Authors
  1. Zhuojin Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chen Gong
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Robert Laganiére
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yong Wang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This research is partially supported by NSF of China (No: 61602246, 61973162), NSF of Jiangsu Province (No: BK20171430), the Fundamental Research Funds for the Central Universities (No: 30918011319), and the “Summit of the Six Top Talents” Program (No: DZXX-027), the Young Elite Scientists Sponsorship Program by Jiangsu Province, and the Young Elite Scientists Sponsorship Program by CAST (No: 2018QNRC001)

Electronic supplementary material

Below is the link to the electronic supplementary material.

(PDF 341 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sun, Z., Wang, Y., Gong, C. et al. Study of UAV tracking based on CNN in noisy environment. Multimed Tools Appl 80, 5351–5372 (2021). https://doi.org/10.1007/s11042-020-09713-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09713-9

Keywords

Search

Navigation