Skip to main content
SpringerLink
Log in

Hierarchical correlation siamese network for real-time object tracking

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Under the influence of deep learning, many trackers have emerged recently. Among them, Siamese network reaches a pleasant balance between accuracy and speed, but its tracking performance still lags behind other trackers. In this paper, we have proposed a Hierarchical Correlation Siamese Network (HC-Siam) for object tracking. The tracker uses convolutional features of each layer to compare the correlation and identifies the position of the tracking object depending on the greatest correlation. Meanwhile, we have designed a Correlation Attention Module (CA-Module). For various objects, this module can assign different weights to the hierarchical correlation and help the network choose the distinct correlation from the hierarchical correlation. Besides, objects’ size and scale constantly varied during tracking, we claimed to use the separate scale factor in the wide and high directions to decrease the deformation of bounding boxes and increase the accuracy of our tracker. On the OTB dataset, the accuracy of HC-Siam is 6.5% higher than the baseline, and the speed of our tracker can reach 85 fps. On the VOT dataset, HC-Siam also has better performance in speed and accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Liu L, Xing J, Ai H, Ruan X (2012) Hand posture recognition using finger geometric feature. In: Proceedings of the 21st international conference on pattern recognition (ICPR2012). IEEE, pp 565–568

  2. Lee K -H, Hwang J-N (2015) On-road pedestrian tracking across multiple driving recorders. IEEE Trans Multimed 17(9):1429–1438

    Article  Google Scholar 

  3. Tang S, Andriluka M, Andres B, Schiele B (2017) Multiple people tracking by lifted multicut and person re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3539–3548

  4. Yilmaz A, Javed O, Shah M (2006) Object tracking: A survey. ACM Computing Survey 38(4):13

    Article  Google Scholar 

  5. Jia X, Lu H, Yang M-H (2012) Visual tracking via adaptive structural local sparse appearance model. In: 2012 IEEE conference on computer vision and pattern recognition. IEEE, pp 1822–1829

  6. Danelljan M, Bhat G, Shahbaz Khan F, Felsberg M (2017) ECO: Efficient convolution operators for tracking. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 6638–6646

  7. Bolme DS, Beveridge JR, Draper BA, Lui YM (2010) Visual object tracking using adaptive correlation filters. In: 2010 IEEE computer society conference on computer vision and pattern recognition, IEEE, pp 2544–2550

  8. Kiani Galoogahi H, Fagg A, Lucey S (2017) Learning background-aware correlation filters for visual tracking. In: Proceedings of the IEEE international conference on computer vision, pp 1135–1143

  9. Lukezic A, Vojir T, Cehovin Zajc L, Matas J, Kristan M (2017) Discriminative correlation filter with channel and spatial reliability. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 6309–6318

  10. Zhang M, Xing J, Gao J, Hu W (2015) Robust visual tracking using joint scale-spatial correlation filters. In: 2015 IEEE international conference on image processing (ICIP), IEEE, pp 1468–1472

  11. Zhang M, Xing J, Gao J, Shi X, Wang Q, Hu W (2015) Joint scale-spatial correlation tracking with adaptive rotation estimation. In: Proceedings of the IEEE international conference on computer vision workshops, pp 32–40

  12. Nam H, Han B (2016) Learning multi-domain convolutional neural networks for visual tracking. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4293–4302

  13. Sun C, Wang D, Lu H, Yang M-H (2018) Learning spatial-aware regressions for visual tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 8962–8970

  14. Wang L, Ouyang W, Wang X, Lu H (2015) Visual tracking with fully convolutional networks. In: Proceedings of the IEEE international conference on computer vision, pp 3119–3127

  15. Henriques J F, Caseiro R, Martins P, Batista J (2014) High-speed tracking with kernelized correlation filters. IEEE Trans Pattern Anal Mach Intell 37(3):583–596

    Article  Google Scholar 

  16. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M, Berg AC, Li F-F (2015) Imagenet large scale visual recognition challenge. International journal of computer vision 115(3):211–252

    Article  MathSciNet  Google Scholar 

  17. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105

  18. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556

  19. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  20. Held D, Thrun S, Savarese S (2016) Learning to track at 100 fps with deep regression networks. In: European conference on computer vision. Springer, pp 749–765

  21. Bertinetto L, Valmadre J, Henriques JF, Vedaldi A, Torr PH (2016) Fully-convolutional siamese networks for object tracking. In: European conference on computer vision. Springer, pp 850–865

  22. Wang X, Li C, Luo B, Tang J (2018) Sint++: robust visual tracking via adversarial positive instance generation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4864–4873

  23. Yang T, Chan AB (2017) Recurrent filter learning for visual tracking. In: Proceedings of the IEEE international conference on computer vision, pp 2010–2019

  24. He A, Luo C, Tian X, Zeng W (2018) A twofold siamese network for real-time object tracking. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4834–4843

  25. Zhang Y, Wang L, Qi J, Wang D, Feng M, Lu H (2018) Structured siamese network for real-time visual tracking. In: Proceedings of the European conference on computer vision (ECCV), pp 351–366

  26. Wang Q, Gao J, Xing J, Zhang M, Hu W (2017) Discriminant correlation filters network for visual tracking. arXiv

  27. Huang C, Lucey S, Ramanan D (2017) Learning policies for adaptive tracking with deep feature cascades. In: Proceedings of the IEEE international conference on computer vision, pp 105–114

  28. Guo Q, Feng W, Zhou C, Huang R, Wan L, Wang S (2017) Learning dynamic siamese network for visual object tracking. In: Proceedings of the IEEE international conference on computer vision, pp 1763–1771

  29. Yang T, Chan AB (2018) Learning dynamic memory networks for object tracking. In: Proceedings of the European conference on computer vision (ECCV), pp 152—167

  30. Dong X, Shen J (2018) Triplet loss in siamese network for object tracking. In: Proceedings of the European conference on computer vision (ECCV), pp 459–474

  31. Wu Y, Lim J, Yang M-H (2013) Online object tracking: A benchmark. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2411–2418

  32. Kristan M, Matas J, Leonardis A, Felsberg M, Cehovin L, Fernandez G, Vojir T, Hager G, Nebehay G, Pflugfelder R (2015) The visual object tracking vot2015 challenge results. In: Proceedings of the IEEE international conference on computer vision workshops, pp 1–23

  33. Qi Y, Qin L, Zhang S, Huang Q, Yao H (2019) Robust visual tracking via scale-and-state-awareness. Neurocomputing 329(2019):75–85

    Article  Google Scholar 

  34. Wang Q, Gao J, Xing J, Zhang M (2017) Hu WDJapa Discriminant correlation filters network for visual tracking. arXiv

  35. Danelljan M, Robinson A, Khan FS, Felsberg M (2016) Beyond correlation filters: Learning continuous convolution operators for visual tracking. In: European conference on computer vision. Springer, pp 472–488

  36. Lee H, Kim D (2018) Salient region-based online object tracking. In: 2018 IEEE winter conference on applications of computer vision (WACV). IEEE, pp 1170–1177

  37. Chi Z, Li H, Lu H, Yang M -H (2017) Dual deep network for visual tracking. IEEE Trans. Image Process 26(4):2005–2015

    Article  MathSciNet  Google Scholar 

  38. Nam H, Baek M, Han B (2016) Modeling and propagating cnns in a tree structure for visual tracking. arXiv preprint arXiv:1608.07242

  39. Bertinetto L, Valmadre J, Golodetz S, Miksik O (2016) Torr PH staple: Complementary learners for real-time tracking. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1401–1409

  40. Zhu G, Porikli F, Li H (2016) Beyond local search: Tracking objects everywhere with instance-specific proposals. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 943–951

Download references

Author information

Authors and Affiliations

  1. School of Mechanical Engineering, University of Science and Technology Beijing, Beijing, China

    Yu Meng, Zaixu Deng, Kun Zhao, Yan Xu & Hao Liu

  2. Institute of Artificial Intelligence, University of Science and Technology Beijing, Beijing, China

    Yu Meng

Authors
  1. Yu Meng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zaixu Deng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kun Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yan Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hao Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yu Meng.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Meng, Y., Deng, Z., Zhao, K. et al. Hierarchical correlation siamese network for real-time object tracking. Appl Intell 51, 3202–3211 (2021). https://doi.org/10.1007/s10489-020-01992-x

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-020-01992-x

Keywords

Search

Navigation