Skip to main content
SpringerLink
Log in

An efficient method for tracking failure detection using parallel correlation filtering and Siamese network

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

In recent years, object tracking based on Siamese network has attracted the attention by taking advantage of both speed and accuracy. However, when similar object interference, severe occlusion and other factors cause temporary tracking failure, a restricted search window makes Siamese network difficult to retrieve the object again, and results in unrecoverable tracking failures. In this paper, a general and efficient two-way verification tracking failure detection method using parallel correlation filtering and Siamese network is proposed, which can detect the tracking failure and retrieve the object again. Firstly, we construct parallel Siamese network and correlation filter tracking network, and get tracking results respectively. Secondly, we make a preliminary judgment on the reliability of the tracking results based on the overlap ratio of the tracking results. Finally, based on two-way verification, which tracker failed to track is finally determined, and the search window of Siamese network is optimized to retrieve the object again. We comparisons with state-of-the art trackers on benchmark datasets: OTB100, VOT2016, VOT2018, VOT2019 and NFS. The results show that the method we proposed can detect the tracking failure and retrieve the object again, thus improving the accuracy of object tracking.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Fiaz M, Mahmood A, Javed S, Jung SK (2019) Handcrafted and deep trackers: Recent visual object tracking approaches and trends. ACM Comput Surv 52(2): Article Number 43

  2. Bolme DS, Beveridge JR, Draper BA, Lui YM (2010) Visual object tracking using adaptive correlation filters. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 2544–2550

  3. Henriques JF, Caseiro R, Martins P, Batista J (2012) Exploiting the circulant structure of tracking-by-detection with kernels. In: Proceedings of the european conference on computer vision (ECCV), pp 702–715

  4. Danelljan M, Khan FS, Felsberg M, Van de Weijer J (2014) Adaptive color attributes for real-time visual tracking. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 1090–1097

  5. Danelljan M, Häger G, Khan FS, Felsberg M (2014) Accurate scale estimation for robust visual tracking. In: Proceedings of the British machine vision conference, pp 0–0

  6. Henriques JF, Caseiro R, Martins P, Batista J (2015) High-speed tracking with kernelized correlation filters. IEEE Trans Pattern Ana Mach Intell 37(3):583–596

    Article  Google Scholar 

  7. Danelljan M, Hager G, Khan FS, Felsberg M (2016) Discriminative scale space tracking. IEEE Trans Pattern Ana Mach Intell 39(8):1561–1575

    Article  Google Scholar 

  8. Wang M, Liu Y, Huang Z (2017) Large margin object tracking with circulant feature maps. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 4800–4808

  9. Zhu GB, Wang JQ, Wu Y, Zhang XY, Lu HQ (2016) MC HOG correlation tracking with saliency proposal. In: Proceedings of the association-for-the-advancement-of-artificial-intelligence conference on artificial intelligence, pp 3690–3696

  10. Bertinetto L, Valmadre J, Golodetz S, Miksik O, Torr PH (2016) Staple: Complementary learners for real-time tracking. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 1401–1409

  11. He R, Wu X, Sun ZN, Tan TN (2018) Wasserstein CNN: Learning invariant features for NIR-VIS face recognition. IEEE Transa Pattern Anal Mach Intell 41(7):1761–1773

    Article  Google Scholar 

  12. Fu K, Zhao Q, Gu IYH (2019) Refinet: A deep segmentation assisted refinement network for salient object detection. IEEE Trans Multimed 21(2):457–469

    Article  Google Scholar 

  13. Ma C, Huang JB, Yang X, Yang MH (2016) Hierarchical convolutional features for visual tracking. In: Proceedings of the IEEE international conference on computer vision (ICCV), pp 3074–3082

  14. Danelljan M, Hager G, Khan FS, Felsberg M (2016) Convolutional features for correlation filter based visual tracking. In: Proceedings of the IEEE international conference on computer vision (ICCV), pp 621–629

  15. Danelljan M, Robinson A, Khan FS, Felsberg M (2016) Beyond correlation filters: Learning continuous convolution operators for visual tracking. In: Proceedings of the european conference on computer vision (ECCV), pp 472–488

  16. Bertinetto L, Valmadre J, Henriques JF, Vedaldi A, Torr PHS (2016) Fully-convolutional Siamese networks for object tracking. In: Proceedings of the european conference on computer vision (ECCV), pp 850–865

  17. Tao R, Gavves E, Smeulders AWM (2016) Siamese instance search for tracking. In: Proceedings of the IEEE international conference on computer vision (ICCV), pp 1420–1429

  18. Li B, Yan J, Wu W, Zhu Z, Hu X (2018) High performance visual tracking with Siamese region proposal network. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 8971–8980

  19. Li B, Wu W, Wang Q, Zhang F, Xing J, Yan J (2019) Siamrpn++: Evolution of Siamese visual tracking with very deep networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 4277–4286

  20. Wang Q, Zhang L, Bertinetto L, Hu WM, Torr PHS (2019) Fast online object tracking and segmentation: A unifying approach. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 1328–1338

  21. Li T, Wu PH, Ding FF, Yang WY (2020) Parallel dual networks for visual object tracking. Appl Intell 50(12):4631–4646

    Article  Google Scholar 

  22. Chen ZD, Zhong BN, Li GR, Zhang SP, Ji RR (2020) Siamese box adaptive network for visual tracking. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 6668–6677

  23. Guo DY, Wang J, Cui Y, Wang YC, Chen SY (2020) SiamCAR: Siamese fully convolutional classification and regression for visual tracking. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 6269–6277

  24. Danelljan M, Bhat G, Khan FS, Felsberg M (2019) ATOM: Accurate tracking by overlap maximization. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 4655–4664

  25. Bhat G, Danelljan M, Van Gool L, Timofte R (2019) Learning discriminative model prediction for tracking. In: Proceedings of the IEEE international conference on computer vision (ICCV), pp 6181–6190

  26. Dai KN, Zhang YH, Wang D, Li JH, Lu HC, Yang XY (2020) High-performance long-term tracking with meta-updater. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 6297–6306

  27. Guo DG, Shao YY, Cui Y, Wang ZH, Zhang LY, Shen CH (2021) Graph attention tracking. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 0–0

  28. Wu Y, Lim J, Yang MH (2015) Object tracking benchmark. IEEE Trans Pattern Anal Mach Intell 37(9):1834–1848

    Article  Google Scholar 

  29. Kristan M, Leonardis A, Matas J, Felsberg M, Pflugfelder R, Cehovin Zajc L, et al (2016) The visual object tracking VOT2016 challenge results. In: Proceedings of the european conference on computer vision (ECCV), pp 777–823

  30. Kristan M, Matas J, Leonardis A, Felsberg M, Pflugfelder R, Kamarainen JK, et al (2019) The seventh visual object tracking VOT2019 challenge results. In: Proceedings of the IEEE/CVF international conference on computer vision (ICCV), pp 2206–2241

  31. Galoogahi HK, Fagg A, Huang C, Ramanan D, Lucey S (2017) Need for speed: A benchmark for higher frame rate object tracking. In: Proceedings of the IEEE international conference on computer vision (ICCV), pp 1134–1143

  32. Nam H, Han B (2016) Learning multi-domain convolutional neural networks for visual tracking. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 4293–4302

  33. Wang MM, Liu Y, Huang ZY (2017) Large margin object tracking with circulant feature maps. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 4800–4808

  34. Lukezic A, Zajc LC, Vojir T, Matas J, Kristan M (2018) FuCoLoT-A fully-correlational long-term tracker. In: Proceedings of the asian conference on computer vision (ACCV), pp 595–611

  35. Dong XP, Shen JB (2018) Triplet loss in Siamese network for object tracking. In: Proceedings of the european conference on computer vision (ECCV), pp 472–488

  36. Guo Q, Feng W, Zhou C, Huang R, Wan L, Wang S (2017) Learning dynamic Siamese network for visual object tracking. In: Proceedings of the IEEE international conference on computer vision (ICCV), pp 1763–1771

  37. He AF, Luo C, Tian XM, Zeng W (2018) A twofold Siamese network for real-time object tracking. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 4834–4843

  38. Valmadre J, Bertinetto L, Henriques J, Vedaldi A, Torr PH (2017) End-to-end representation learning for correlation filter based tracking. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 2805–2813

  39. Gao P, Zhang QQ, Wang F, Xiao LY, Fujita H, Zhang Y (2020) Learning reinforced attentional representation for end-to-end visual tracking. Inf Sci 517:52–67

    Article  Google Scholar 

  40. Gao P, Yuan RY, Wang F, Xiao LY, Fujita H, Zhang Y (2020) Siamese attentional keypoint network for high performance visual tracking. Knowl-based Syst 193: Article Number: 105448

  41. Ren SQ, He KM, Girshick R, Sun J (2017) Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 39(6):1137–1149

    Article  Google Scholar 

  42. Zhu Z, Wang Q, Li B, Wu W, Yan JJ, Hu WM (2018) Distractor-aware Siamese networks for visual object tracking. In: Proceedings of the european conference on computer vision (ECCV), pp 103–119

  43. He KM, Zhang XY, Ren SQ, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 770–778

  44. Voigtlaender P, Luiten J, Torr PHS, Leibe B (2020) Siam R-CNN: Visual tracking by re-detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 70–778

  45. Zhang HW, Li XX, Zhu B, Ma Q (2020) Wide-area search tracking for Siamese region proposal network. IEEE Access 8:115184–115195

    Article  Google Scholar 

  46. Kristan M, Leonardis A, Matas J, Felsberg M, Pflugfelder R, Zajc LC, et al (2018) The sixth visual object tracking VOT2018 challenge results. In: Proceedings of the european conference on computer vision (ECCV), pp 0–0

Download references

Author information

Authors and Affiliations

  1. School of Electronics and Communication Engineering, Sun Yat-sen University, Guangzhou, Guangdong, China

    Shaolong Chen, Changzhen Qiu & Zhiyong Zhang

Authors
  1. Shaolong Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Changzhen Qiu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhiyong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhiyong Zhang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, S., Qiu, C. & Zhang, Z. An efficient method for tracking failure detection using parallel correlation filtering and Siamese network. Appl Intell 52, 7713–7722 (2022). https://doi.org/10.1007/s10489-021-02768-7

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-02768-7

Keywords

Search

Navigation