Skip to main content
SpringerLink
Log in

Learning adaptive updating siamese network for visual tracking

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Recently, Siamese network (Siam)-based visual tracking describes the tracking problems as the cross-correlation between convolutional features of the target template and searching regions and solves them by similarity learning, which has achieved great success in performance. However, most of the existing Siam-based tracking methods neglect to explore the feature correlations, which is very important to learn more representative features. Moreover, the first frame is used as the fixed template without updating the template, which leads to a reduction in accuracy. To address these issues, in this paper, we propose an Adaptive Updating Siamese Network (AU-Siam) for more powerful feature correlations and adaptive template updating. Specifically, a siamese feature extraction subnetwork is proposed to introduce the attention mechanism for more discriminative representations. Furthermore, an object template updating subnetwork is developed to dynamically learn object appearance changes for robust tracking. It’s interesting to show that the proposed AU-Siam can effectively reduce the probability of tracking drift in the case of fast motions and heavy occlusion and improve the tracking accuracy. Experimental results on public tracking benchmarks with challenging sequences demonstrate that our AU-Siam performs favorably against other state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Bertinetto L, Valmadre J, Henriques JF, Vedaldi A, Torr PH (2016) Fully-convolutional siamese networks for object tracking. In European Conference on Computer Vision Workshop, pages 850-865. Springer. 1, 2, 3, 4, 5, 7, 8

  2. Bertinetto L, Valmadre J, Golodetz S, Miksik O, Torr PHS (2016) Staple: Complementary learners for real-time tracking. In The IEEE Conference on Computer Vision and Pattern Recognition, CVPR, June

  3. Bhat G, Johnander J, Danelljan M, Khan FS, Felsberg M (2018) Unveiling the power of deep tracking. In: ECCV, pp 493–509

  4. Choi J, Chang HJ, Yun S, Fischer T, Demiris Y (2017) Attentional correlation filter network for adaptive visual tracking. In: IEEE Conference on Computer Vision and Pattern Recognition, pages 4807-4816. 2 7

  5. Danelljan M., Ager G. H., Khan F. S., Felsberg M. (2016) Adaptive decontamination of the training set: a unified formulation for discriminative visual tracking. In: IEEE Conference on Computer Vision and Pattern Recognition, pages 1430-1438. 4 7

  6. Danelljan M, Bhat G, Khan FS et al (2017) ECO: Efficient convolution operators for tracking. Proc IEEE Conf Comput Vis Pattern Recognit, page 6

  7. Danelljan M, Häger G et al (2015) Convolutional Features for Correlation Filter Based Visual Tracking. ICCV workshop

  8. Danelljan M, Robinson A, Khan FS, Felsberg M (2016) Beyond correlation filters: Learning continuous convolution operators for visual tracking. In: ECCV, pages 472-488. 1 2 5

  9. Dong X, Shen J (2018) Triplet loss in siamese network for object tracking[C]. Proceedings of the European Conference on Computer Vision (ECCV). 459-474

  10. Dong X, Shen J, Wang W et al (2018) Hyperparameter optimization for tracking with continuous deep q-learning[C].Proceedings of the IEEE conference on computer vision and pattern recognition. 518-527

  11. Dong X, Shen J, Wang W et al (2019) Dynamical Hyperparameter Optimization via Deep Reinforcement Learning in Tracking[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence PP(99):1–1

    Google Scholar 

  12. Dong X, Shen J, Wu D et al (2019) Quadruplet network with One-Shot learning for fast visual object Tracking[J]. IEEE Trans Image Process 28 (7):3516–3527

    Article  MathSciNet  Google Scholar 

  13. Dong X, Shen J, Yu D et al (2017) Occlusion-aware real-time object tracking[J], vol 19

  14. Fan DP, Cheng MM, Liu JJ et al (2018) Salient objects in clutter: Bringing salient object detection to the foreground[C]. Proceedings of the European conference on computer vision (ECCV). 186-202

  15. Fu K, Fan D P, Ji G P et al (2020) Siamese network for rgb-d salient object detection and beyond[J]. arXiv preprint arXiv:2008.12134

  16. Fu K, Fan DP, Ji GP et al (2020) Jl-dcf: Joint learning and densely-cooperative fusion framework for rgb-d salient object detection[C].Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 3052-3062

  17. Fu K, Zhao Q, Gu IYH et al (2019) Deepside: A general deep framework for salient object detection[J]. Neurocomputing 356:69–82

    Article  Google Scholar 

  18. Gong C, Tao D, Liu W et al (2015) Saliency propagation from simple to difficult[C].Proceedings of the IEEE conference on computer vision and pattern recognition. 2531-2539

  19. Guo Q, Feng W, Zhou C, Huang R, Wan L, Wang S (2017) Learning dynamic siamese network for visual object tracking. In: ICCV. 1

  20. He A, Luo C et al (2018) A twofold siamese network for real-time object tracking[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4834–4843

  21. Henriques J, Caseiro R, Martins P, Batista J (2015) Highspeed tracking with kernelized correlation filters. IEEE Transactions on Pattern Analysis and Machine Intelligence 37(3):583–596. 1 3 7

    Article  Google Scholar 

  22. Hu H, Ma B, Shen J et al (2018) Robust object tracking using manifold regularized convolutional neural networks[J]. IEEE Transactions on Multimedia 21(2):510–521

    Article  Google Scholar 

  23. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks[C].Proceedings of the IEEE conference on computer vision and pattern recognition. 7132-7141

  24. Jianbing S, Xin et al (2019) Visual Object Tracking by Hierarchical Attention Siamese Network.[J] IEEE transactions on cybernetics

  25. Kristan M, Leonardis A et al (2016) The visual object tracking vot2016 challenge results. In: ECCV, pp 777–823

  26. Kristan M, Leonardis A et al (2018) The sixth visual object tracking VOT2018 challenge results. In: ECCV, pp 3–53

  27. Krizhevsky A, Sutskever I et al (2012) Imagenet Classification with Deep Convolutional Neural Networks[C]. NIPS Curran Associates Inc

  28. Li B, Wu W et al (2019) Siamrpn++:, Evolution of siamese visual tracking with very deep networks[C].Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4282–4291

  29. Li B, Yan J, Wu W, Zhu Z, Hu X (2018) High performance visual tracking with siamese region proposal network. In: CVPR. 1, 2, 3, 4, 5, 8

  30. Liang Z, Shen J (2019) Local semantic siamese networks for fast tracking[J]. IEEE Trans Image Process 29:3351–3364

    Article  Google Scholar 

  31. Liu F, Gong C, Huang X et al (2018) Robust visual tracking revisited: From correlation filter to template matching[J]. IEEE Trans Image Process 27 (6):2777–2790

    Article  MathSciNet  Google Scholar 

  32. Liu F, Gong C, Huang X et al (2018) Robust visual tracking revisited: From correlation filter to template matching[J]. IEEE Trans Image Process 27 (6):2777–2790

    Article  MathSciNet  Google Scholar 

  33. Liu P, Yu H, Cang S (2019) Adaptive neural network tracking control for underactuated systems with matched and mismatched disturbances[J]. Nonlinear Dynamics 98(2):1447–1464

    Article  Google Scholar 

  34. Lu X, Ni B, Ma C et al (2019) Learning transform-aware attentive network for object tracking[J]. Neurocomputing 349:133–144

    Article  Google Scholar 

  35. Lu X, Wang W, Ma C et al (2019) See more, know more: Unsupervised video object segmentation with co-attention siamese networks[C].Proceedings of the IEEE conference on computer vision and pattern recognition. 3623-3632

  36. Lukezic A, Vojir T, Cehovin Zajc L, Matas J, Kristan M (2017) Discriminative correlation filter with channel and spatial reliability

  37. Ren S, Girshick R et al (2017) Faster r-CNN: towards Real-Time object detection with region proposal Networks[J]. IEEE Transactions on Pattern Analysis Machine Intelligence 39(6):1137–1149

    Article  Google Scholar 

  38. Sun L, Zhao C, Yan Z et al (2018) A novel weakly-supervised approach for RGB-D-based nuclear waste object detection[J]. IEEE Sensors J 19 (9):3487–3500

    Article  Google Scholar 

  39. Tang Z, Li C, Wu J et al (2019) Classification of EEG-based single-trial motor imagery tasks using a B-CSP method for BCI[J]. Frontiers of Information Technology & Electronic Engineering 20(8):1087–1098

    Article  Google Scholar 

  40. Tang Z, Yu H, Lu C et al (2019) Single-Trial Classification of Different Movements on One Arm Based on ERD/ERS and Corticomuscular Coherence[J]. IEEE Access 7:128185–128197

    Article  Google Scholar 

  41. Tao R, Gavves E, Smeulders AWM (2016) Siamese instance search for tracking. In IEEE Conference on Computer Vision and Pattern Recognition. 1, 2, 3, 7

  42. Valmadre J, Bertinetto L, Henriques JF, Vedaldi A, Torr PH (2017) End-to-end representation learning for correlation filter based tracking. In: CVPR. 1, 2, 3, 4, 8

  43. Wang W, Lu X, Shen J et al (2019) Zero-shot video object segmentation via attentive graph neural networks[C].Proceedings of the IEEE international conference on computer vision. 9236–9245

  44. Wang W, Shen J, Ling H (2018) A deep network solution for attention and aesthetics aware photo cropping[J]. IEEE transactions on pattern analysis and machine intelligence 41(7):1531–1544

    Article  Google Scholar 

  45. Wang Q, Teng Z, Xing J, Gao J, Hu WS (2018) Maybank.Learning attentions: Residual attentional siamese network for high performance online visual tracking. In: CVPR. 1 2

  46. Woo S, Park J, Lee JY et al (2018) CBAM : Convolutional Block Attention Module[J]

  47. Wu Y, Lim J, Yang M-H (2013) Online object tracking: a benchmark. In: CVPR. 2

  48. Wu Y, Lim J, Yang M-H (2015) Object tracking benchmark.TPAMI, 1, 2, 5, 6, 7

  49. Xiao Y, Li J, Du B, Wu J, Chang J, Zhang W (2020) Memu: Metric Correlation Siamese Network and Multi-class Negative Sampling for Visual Tracking. Pattern Recognition, Volume 100. https://doi.org/10.1016/j.patcog.2019.107170

  50. Yilmaz A., Javed O., Shah M. (2006) Object tracking: a survey. ACM Comput Surv 38(4):1–45

    Article  Google Scholar 

  51. Z T, Ghanem B, Liu S, Ahuja N (2012) Robust visual tracking via multi-task sparse learning. In: IEEE Conference on Computer Vision and Pattern Recognition CVPR

  52. Zhang Z, Lai Z, Huang Z et al (2019) Scalable supervised asymmetric hashing with semantic and latent factor Embedding[J] IEEE transactions on image processing

  53. Zhang Z, Peng H (2019) Deeper and wider siamese networks for real-time visual tracking[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4591-4600

  54. Zhang Y, Wang L et al (2018) Structured siamese network for real-time visual tracking[C].Proceedings of the European conference on computer vision (ECCV). 351-366

  55. Zhou Y, Li J, Du B, Chang J, Xiao Y (2020) A Target Response Adaptive Correlation Filter Tracker with Spatial Attention. Multimedia Tools and Applications. https://doi.org/10.1007/s11042-020-08839-0

  56. Zhu Z, Wang Q, Li B, Wu W, Yan J, Hu W (2018) Distractor-aware siamese networks for visual object tracking. In: ECCV. 1, 2, 3, 6, 7, 8

Download references

Author information

Authors and Affiliations

  1. School of Computer Science, Wuhan University, Wuhan, 430072, China

    Yifei Zhou, Jing Li, Bo Du & Jun Chang

  2. Sichuan Institute of Aerospace Electronic Equipment, Chengdu, 610100, China

    Zhiquan Ding & Tianqi Qin

Authors
  1. Yifei Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jing Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bo Du
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jun Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zhiquan Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Tianqi Qin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jing Li.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work was supported in part by the Science and Technology Major Project of Hubei Province (Next-Generation AI Technologies) under Grant 2019AEA170.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhou, Y., Li, J., Du, B. et al. Learning adaptive updating siamese network for visual tracking. Multimed Tools Appl 80, 29849–29873 (2021). https://doi.org/10.1007/s11042-021-11154-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-11154-x

Keywords

Search

Navigation