Skip to main content

Advertisement

Springer Nature Link
Log in

A robust tracking architecture using tracking failure detection in Siamese trackers

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

In recent years, due to the impressive performance on the speed and accuracy, the Siamese network has gained a lot of popularity in the visual tracking community. However, both the spatiotemporal correlation of adjacent frames and confidence assessment of the results of the classification branch are missing in the offline-trained Siamese tracker. In this paper, a robust tracking architecture is proposed to implement the tracking failure detection and make better tracking decisions for the Siamese tracker. It consists of two stages including tracking failure detection and proposal re-selection. Firstly, a Siamese tracker is adopted as the baseline, and a tracking failure detection mechanism is proposed based on motion estimation of object via optical flow. It can timely supervise the reliability of the tracking system. Secondly, when the tracking failure occurs, the proposal selection strategy is optimized with spatiotemporal information to re-select more reasonable results. The overall mechanism can guide the tracker to handle target drift problem by tracking failure detection and proposal re-selection. Several representative Siamese trackers are utilized to validate the effectiveness of our approach. Furthermore, the performance of our approach is demonstrated based on extensive experiments on popular benchmarks, which can improve the robustness of the model.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Tao R, Gavves E, Smeulders AWM (2016) Siamese instance search for tracking. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 1420–1429

  2. Bertinetto L, Valmadre J, Henriques JF, Vedaldi A, Torr PHS (2016) Fully-convolutional siamese networks for object tracking. In: computer science - computer vision and pattern recognition (CVPR)

  3. Bo L, Yan J, Wei W, Zheng Z, Hu X (2018) High performance visual tracking with siamese region proposal network. In: 2018 IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp. 8971–8980

  4. Ren S, He K, Girshick R, Sun J (2017) Faster r-cnn: Towards real-time object detection with region proposal networks. 39(6) 1137–1149

  5. Li B, Wu W, Wang Q, F Z. J (2018) Siamrpn++: Evolution of siamese visual tracking with very deep networks. In: computer vision and pattern recognition (CVPR)

  6. Wang Q, Zhang L, Bertinetto L, Hu W, Torr PHS (2018) Fast online object tracking and segmentation: a unifying approach. In: computer science - computer vision and pattern recognition (CVPR)

  7. Liao B, Wang C, Wang Y, Wang Y, Yin J (2020) Pg-net: Pixel to global matching network for visual tracking. In: computer vision – ECCV 2020. Lecture notes in computer science. Springer, pp 429–444

  8. Guo Q, Wei F, Zhou C, Rui H, Song W (2017) Learning dynamic siamese network for visual object tracking. In: 2017 IEEE international conference on computer vision (ICCV)

  9. Li P, Chen B, Ouyang W, Wang D, Yang X, Lu H (2020) Gradnet: Gradient-guided network for visual object tracking. In: 2019 IEEE/CVF international conference on computer vision (ICCV)

  10. Fan B, Tian J, Peng Y, Tang Y (2021) Discriminative siamese complementary tracker with flexible update. IEEE Transactions on Multimedia 1–1

  11. Li S, Zhao S, Cheng B, Chen J (2020) Noise-aware framework for robust visual tracking. IEEE Transactions on Cybernetics PP(99):1–14

    Google Scholar 

  12. Chen S, Qiu C, Zhang Z (2021) An efficient method for tracking failure detection using parallel correlation filtering and siamese network. Appl Intell (Dordrecht Netherlands)

  13. Nam H, Han B (2016) Learning multi-domain convolutional neural networks for visual tracking. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 4293–4302

  14. Bhat G, Danelljan M, Van Gool L, Timofte R (2019) Learning discriminative model prediction for tracking. In: 2019 IEEE/CVF international conference on computer vision (ICCV), pp 6181–6190

  15. Wang M, Liu Y, Huang Z (2017) Large margin object tracking with circulant feature maps. IEEE Computer Society

  16. Lukežič A (2017) Zajc: Fucolot – a fully-correlational long-term tracker. In: computer science - computer vision and pattern recognition (CVPR)

  17. Wang W, Chen Z, Douadji L, Shi M (2019) Robust correlation filter tracking with deep semantic supervision. IET Image Proc 13(5):754–760

    Article  Google Scholar 

  18. Dai K, Zhang Y, Wang D, Li J, Lu H, Yang X (2020) High-performance long-term tracking with meta-updater. In: computer science - computer vision and pattern recognition (CVPR)

  19. Valmadre J, Bertinetto L, Henriques JF, Vedaldi A, Torr P (2017) End-to-end representation learning for correlation filter based tracking. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR)

  20. Zhu Z, Wang Q, Li B, Wu W, Yan J, Hu W (2018) Distractor-aware siamese networks for visual object tracking. In: computer vision – ECCV 2018, pp 103–119

  21. Zhang Z, Peng H (2020) Deeper and wider siamese networks for real-time visual tracking. In: 2019 IEEE/CVF conference on computer vision and pattern recognition (CVPR)

  22. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: IEEE conference on computer vision and pattern recognition (CPVR)

  23. Guo D, Wang J, Cui Y, Wang Z, Chen S (2019) Siamcar: Siamese fully convolutional classification and regression for visual tracking. In: computer science - computer vision and pattern recognition (CVPR)

  24. Voigtlaender P, Luiten J, Torr P, Leibe B (2019) Siam r-cnn: Visual tracking by re-detection. In: computer science - computer vision and pattern recognition (CVPR)

  25. Guo D, Wang J, Cui Y, Wang Z, Chen S (2020) Siamcar: Siamese fully convolutional classification and regression for visual tracking. In: IEEE conference on computer vision and pattern recognition

  26. Yang K, H Z, W P (2021) Siamcorners: Siamese corner networks for visual tracking. IEEE Transactions on Multimedia

  27. Chan S, T J, X Z (2022) Siamese implicit region proposal network with compound attention for visual tracking. IEEE Trans Image Process 31:1882–1894

    Article  Google Scholar 

  28. Alan L, TVJM, Luka Ehovin Z, Kristan M (2018) Fclt - a fully-correlational long-term tracker. In: ACCV

  29. Jack V, JFHRTAVAWMSPHST, Luca B, Gavves E (2018) Long-term tracking in the wild: a benchmark. In: ECCV

  30. Chang HJ, Park MS, Jeong H, Jin YC (2011) Tracking failure detection by imitating human visual perception. In: IEEE international conference on image processing, pp 3293–3296

  31. Dai K, Zhang Y, Wang D, Li J, Lu H, Yang X (2020) High-performance long-term tracking with meta-updater. In: 2020 IEEE/CVF conference on computer vision and pattern recognition (CVPR) 6297–6306

  32. Ilchae Jung MB, Jeany S, Han B (2018) Real-time mdnet. In: ECCV

  33. Wang W, Chen Z, Douadji L, Shi M (2019) Robust correlation filter tracking with deep semantic supervision. IET Image Process 13:754–760

    Article  Google Scholar 

  34. Beyer L, Breuers S, Kurin V, Leibe B (2017) Towards a principled integration of multi-camera re-identification and tracking through optimal bayes filters. In: 2017 IEEE conference on computer vision and pattern recognition workshops (CVPRW), pp 1444–1453

  35. Kalal Z, Mikolajczyk K, Matas J (2012) Tracking-learning-detection. IEEE Trans Pattern Anal Mach Intell 34(7):1409–1422. 10.1109/TPAMI.2011.239

    Article  Google Scholar 

  36. Chao M, Yang X, Zhang C, Yang MH (2015) Long-term correlation tracking. In: 2015 IEEE conference on computer vision and pattern recognition (CVPR)

  37. Wang M, Liu Y, Huang Z (2017) Large margin object tracking with circulant feature maps. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR), pp. 4800– 4808

  38. Luke A, Zajc LC, Tom Matas J, Kristan M (2018) Fucolot - a fully-correlational long-term tracker. In: ACCV

  39. Ma C, Huang J-B, Yang X, Yang M-H (2018) Adaptive correlation filters with long-term and short-term memory for object tracking. Int J Comput Vis 126:771–796

    Article  Google Scholar 

  40. Hong Z, Chen Z, Wang C, Mei X, Prokhorov DV, Tao D (2015) Multi-store tracker (muster): A cognitive psychology inspired approach to object tracking. In: 2015 IEEE conference on computer vision and pattern recognition (CVPR), pp 749–758

  41. Yuan D, Li D, He Z, Zhang X (2018) Particle filter re-detection for visual tracking via correlation filters. Multimed Tools Appl 78:14277–14301

    Article  Google Scholar 

  42. Yan B, Zhao H, Wang D, Lu H, Yang X (2019) ‘Skimming-perusal’ tracking: A framework for real-time and robust long-term tracking. In: 2019 IEEE/CVF international conference on computer vision (ICCV), pp 2385–2393

  43. Voigtlaender P, Luiten J, Torr PHS, Leibe B (2020) Siam r-cnn: Visual tracking by re-detection. Computer Vision and Pattern Recognition (CVPR) 6577–6587

  44. Zhang Y, Wang D, Wang L, Qi J, Lu H (2018) Learning regression and verification networks for long-term visual tracking. arXiv:1809.04320

  45. Lucas B (1981) An iterative image registration technique with an application to stereo vision (darpa). Proc Ijcai 81(3):674–679

    Google Scholar 

  46. Horn B, Schunck BG (1981) Determining optical flow. Artif Intell 17(1-3):185–203

    Article  MATH  Google Scholar 

  47. Sun D, Roth S, Black MJ (2010) Secrets of optical flow estimation and their principles. In: the twenty-third IEEE conference on computer vision and pattern recognition, CVPR 2010, san francisco, CA, USA, 13-18 June 2010

  48. Brox T, Malik J (2011) Fellow, IEEE: Large displacement optical flow: Descriptor matching in variational motion estimation. IEEE Trans Pattern Anal Mach Intell 33(3):500–513

    Article  Google Scholar 

  49. Black MJ, Anandan P (1996) The robust estimation of multiple motions: Parametric and piecewise-smooth flow fields. Comput Vis Image Underst 63(1):75–104

    Article  Google Scholar 

  50. Liu PD, MIoT C (2009) Beyond pixels : exploring new representations and applications for motion analysis Massachusetts Institute of Technology

  51. Galoogahi HK, Fagg A, Lucey S (2017) Learning background-aware correlation filters for visual tracking. 2017 IEEE international conference on computer vision (ICCV)

  52. Danelljan M, Bhat G, Khan FS, Felsberg M (2017) Eco: Efficient convolution operators for tracking. In: computer vision and pattern recognition (CVPR)

  53. Li F, Tian C, Zuo W, Zhang L, Yang MH (2018) Learning spatial-temporal regularized correlation filters for visual tracking. In: 2018 IEEE/CVF conference on computer vision and pattern recognition (CVPR)

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China (No. 62075028)

Author information

Authors and Affiliations

  1. School of Automation Engineering, University of Electronic Science and Technology of China, Xiyuan Avenue, Chengdu, 611730, Sichuan, China

    Xin Lu, Fusheng Li, Yanchun Zhao & Wanqi Yang

  2. Yangtze Delta Region Institute, University of Electronic Science and Technology of China, Xisaishan Avenue, Huzhou, 313001, Zhejiang, China

    Xin Lu, Fusheng Li, Yanchun Zhao & Wanqi Yang

Authors
  1. Xin Lu
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Fusheng Li
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Yanchun Zhao
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Wanqi Yang
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Fusheng Li.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lu, X., Li, F., Zhao, Y. et al. A robust tracking architecture using tracking failure detection in Siamese trackers. Appl Intell 53, 12564–12579 (2023). https://doi.org/10.1007/s10489-022-04154-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-04154-3

Keywords