Skip to main content
SpringerLink
Log in

Multi-kernel support correlation filters with temporal filtering constraint for object tracking

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

This paper proposes the adaptive multi-kernel support correlation filters with hedge parameter strategy and temporal filtering constraint for real-time tracking. In order to fuse the excellent properties of various views that characterize the object robust appearance accurately, support correlation filtering responses from multiple kernels can be adaptively integrated into one strong and accurate filtering response map by hedge parameter strategy in a parallel way. It absorbs the strongly discriminative ability from different feature-based support correlation filters, which tolerate sampling outliers of circulant structures with the help of SVM learning way. Also, it exploits the intense information of multi-view appearance representations which guarantee the fusion of reliable correlation filtering maps with reasonable parameters. Meanwhile, with the temporal filtering constraint to maintain historical appearance characteristics, alternating fixed-point algorithm improves complementary memory-updated model that keeps the stability of tracking process continuously and alleviates the target drifting situation for each support correlation filter. Experimental results demonstrate that the proposed approach achieves favorable performance on multiple dynamic scenes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Avidan S (2004) Support vector tracking. IEEE Trans Pattern Anal Mach Intell 26(8):1064–1072

    Article  Google Scholar 

  2. Avidan S (2007) Ensemble tracking. IEEE Trans Pattern Anal Mach Intell 29(2):261–271

    Article  Google Scholar 

  3. Bai Y, Tang M (2011) Robust visual tracking via ranking SVM. In: Macq B, Schelkens P (eds) 18th IEEE international conference on image processing, ICIP 2011. IEEE, Brussels, pp 517–520

  4. Bertinetto L, Valmadre J, Golodetz S, Miksik O, Torr P H S (2016) Staple: Complementary learners for real-time tracking. In: 2016 IEEE conference on computer vision and pattern recognition, CVPR 2016. IEEE Computer Society, Las Vegas, pp 1401–1409

  5. Bertinetto L, Valmadre J, Henriques J F, Vedaldi A, Torr P H S (2016) Fully-convolutional siamese networks for object tracking. In: Hua G, Jégou H (eds) Computer vision - ECCV 2016 workshops - amsterdam, The netherlands, proceedings, part II, Lecture Notes in Computer Science, vol 9914, pp 850–865

  6. Bibi A, Ghanem B (2015) Multi-template scale-adaptive kernelized correlation filters. In: 2015 IEEE international conference on computer vision workshop, ICCV Workshops 2015. IEEE Computer Society, Santiago, pp 613–620

  7. Bibi A, Mueller M, Ghanem B (2016) Target response adaptation for correlation filter tracking. In: Leibe B, Matas J, Sebe N, Welling M (eds) Computer vision - ECCV 2016 - 14th european conference, Amsterdam, proceedings, part VI, Lecture Notes in Computer Science, vol 9910. Springer, pp 419–433

  8. Boddeti V N, Kumar B V K V (2014) Maximum margin vector correlation filter. CoRR arXiv:1404.6031

  9. Bolme D S, Beveridge J R, Draper B A, Lui Y M (2010) Visual object tracking using adaptive correlation filters. In: The twenty-third ieee conference on computer vision and pattern recognition, CVPR 2010. IEEE Computer Society, San Francisco, pp 2544–2550

  10. Cai B, Xu X, Xing X, Jia K, Miao J, Tao D (2016) BIT: biologically inspired tracker. IEEE Trans Image Process 25(3):1327–1339

    Article  MathSciNet  Google Scholar 

  11. Chaudhuri K, Freund Y, Hsu D J (2009) A parameter-free hedging algorithm. In: Bengio Y, Schuurmans D, Lafferty J D, Williams C K I, Culotta A (eds) Advances in neural information processing systems 22: 23rd annual conference on neural information processing systems 2009. Proceedings of a meeting held. Curran Associates, Inc., Vancouver, pp 297–305

  12. Dai K, Wang D, Lu H, Sun C, Li J (2019) Visual tracking via adaptive spatially-regularized correlation filters. In: IEEE conference on computer vision and pattern recognition, CVPR 2019. Computer Vision Foundation / IEEE, Long Beach, pp 4670–4679

  13. Danelljan M, Häger G, Khan F S, Felsberg M (2014) Accurate scale estimation for robust visual tracking. In: Valstar M F, French A P, Pridmore T P (eds) British machine vision conference, BMVC 2014. BMVA Press, Nottingham

  14. Danelljan M, Häger G, Khan F S, Felsberg M (2015) Convolutional features for correlation filter based visual tracking. In: 2015 IEEE international conference on computer vision workshop, ICCV Workshops 2015. IEEE Computer Society, Santiago, pp 621–629

  15. Danelljan M, Häger G, Khan F S, Felsberg M (2016) Learning spatially regularized correlation filters for visual tracking. CoRR arXiv:1608.05571

  16. Danelljan M, Robinson A, Khan F S, Felsberg M (2016) Beyond correlation filters: Learning continuous convolution operators for visual tracking. In: Leibe B, Matas J, Sebe N, Welling M (eds) Computer vision - ECCV 2016 - 14th European conference, proceedings, part V. Lecture Notes in Computer Science, vol 9909. Springer, Amsterdam, pp 472–488

  17. Danelljan M, Häger G, Khan F S, Felsberg M (2017) Discriminative scale space tracking. IEEE Trans Pattern Anal Mach Intell 39(8):1561–1575

    Article  Google Scholar 

  18. Danelljan M, Bhat G, Khan F S, Felsberg M (2017) ECO: efficient convolution operators for tracking. In: 2017 IEEE conference on computer vision and pattern recognition, CVPR 2017. IEEE Computer Society, Honolulu, pp 6931–6939

  19. Ding G, Chen W, Zhao S, Han J, Liu Q (2018) Real-time scalable visual tracking via quadrangle kernelized correlation filters. IEEE Trans Intell Transp Syst 19(1):140–150

    Article  Google Scholar 

  20. Dollar P (2015) Piotr’s computer vision matlab toolbox. línea]. Available: https://github.com/pdollar/toolbox

  21. Fan H, Ling H (2017) Sanet: Structure-aware network for visual tracking. In: 2017 IEEE conference on computer vision and pattern recognition workshops, CVPR Workshops 2017. IEEE Computer Society, Honolulu, pp 2217–2224

  22. Galoogahi H K, Fagg A, Lucey S (2017) Learning background-aware correlation filters for visual tracking. In: IEEE international conference on computer vision, ICCV 2017. IEEE Computer Society, Venice, pp 1144–1152

  23. Gan Q, Guo Q, Zhang Z, Cho K (2015) First step toward model-free, anonymous object tracking with recurrent neural networks. CoRR arXiv:1511.06425

  24. Hare S, Golodetz S, Saffari A, Vineet V, Cheng M-M, Hicks S L, Torr P H S (2016) Struck: Structured output tracking with kernels. IEEE Trans Pattern Anal Mach Intell 38(10):2096–2109

    Article  Google Scholar 

  25. Held D, Thrun S, Savarese S (2016) Learning to track at 100 FPS with deep regression networks. In: Leibe B, Matas J, Sebe N, Welling M (eds) Computer vision - ECCV 2016 - 14th European conference, proceedings, part I, Lecture Notes in Computer Science, vol 9905. Springer, Amsterdam, pp 749–765

  26. Henriques J F, Caseiro R, Martins P, Batista J P (2012) Exploiting the circulant structure of tracking-by-detection with kernels. In: Fitzgibbon A W, Lazebnik S, Perona P, Sato Y, Schmid C (eds) Computer vision - ECCV 2012 - 12th European conference on computer vision, proceedings, part IV, Lecture Notes in Computer Science, vol 7575. Springer, Florence, pp 702–715

  27. Henriques J F, Carreira J , Caseiro R, Batista J (2013) Beyond hard negative mining: Efficient detector learning via block-circulant decomposition. In: IEEE international conference on computer vision, ICCV 2013. IEEE Computer Society, Sydney, pp 2760–2767

  28. Henriques J F, Caseiro R, Martins P, Batista J (2015) High-speed tracking with kernelized correlation filters. IEEE Trans Pattern Anal Mach Intell 37(3):583–596

    Article  Google Scholar 

  29. Hu H, Ma B, Shen J, Sun H, Shao L, Porikli F (2019) Robust object tracking using manifold regularized convolutional neural networks. IEEE Trans Multim 21(2):510–521

    Article  Google Scholar 

  30. Ji Z, Feng K, Qian Y (2019) Part-based visual tracking via structural support correlation filter. J Vis Commun Image Represent 64

  31. Jung I, Son J, Baek M, Han B (2018) Real-time mdnet. In: Ferrari V, Hebert M, Sminchisescu C, Weiss Y (eds) Computer vision - ECCV 2018 - 15th European conference, proceedings, part IV, Lecture Notes in Computer Science, vol 11208. Springer, Munich, pp 89–104

  32. Kahou S E, Michalski V, Memisevic R, Pal C J, Vincent P (2017) RATM: recurrent attentive tracking model. In: 2017 IEEE conference on computer vision and pattern recognition workshops, CVPR Workshops 2017. IEEE Computer Society, Honolulu, pp 1613–1622

  33. Kalal Z (2011) Tracking learning detection. Ph.D. Thesis, University of Surrey, Guildford

  34. Li Y, Zhu J (2014) A scale adaptive kernel correlation filter tracker with feature integration. In: Agapito L, Bronstein M M, Rother C (eds) Computer vision - ECCV 2014 workshops, proceedings, part II, Lecture Notes in Computer Science, vol 8926. Springer, Zurich, pp 254–265

  35. Li Y, Zhu J, Hoi S C H (2015) Reliable patch trackers: Robust visual tracking by exploiting reliable patches. In: IEEE conference on computer vision and pattern recognition, CVPR 2015. IEEE Computer Society, Boston, pp 353–361

  36. Li B, Yan J, Wu W, Zhu Z, Hu X (2018) High performance visual tracking with siamese region proposal network. In: 2018 IEEE conference on computer vision and pattern recognition, CVPR 2018. IEEE Computer Society, Salt Lake City, pp 8971–8980

  37. Li B, Wu W, Wang Q, Zhang F, Xing J, Yan J (2019) Siamrpn++: Evolution of siamese visual tracking with very deep networks. In: IEEE conference on computer vision and pattern recognition, CVPR 2019. Computer Vision Foundation / IEEE, Long Beach, pp 4282–4291

  38. Lin Y, Zhong B, Li G, Zhao S, Chen Z, Fan W (2019) Localization-aware meta tracker guided with adversarial features. IEEE Access 7:99441–99450

    Article  Google Scholar 

  39. Liu G, Lin Z, Yan S, Sun J, Yu Y, Ma Y (2013) Robust recovery of subspace structures by low-rank representation. IEEE Trans Pattern Anal Mach Intell 35(1):171–184

    Article  Google Scholar 

  40. Liu T, Wang G, Yang Q (2015) Real-time part-based visual tracking via adaptive correlation filters. In: IEEE conference on computer vision and pattern recognition, CVPR 2015. IEEE Computer Society, Boston, pp 4902–4912

  41. Lu X, Ni B, Ma C, Yang X (2019) Learning transform-aware attentive network for object tracking. Neurocomputing 349:133–144

    Article  Google Scholar 

  42. Ma C, Huang J-B, Yang X, Yang M-H (2015) Hierarchical convolutional features for visual tracking. In: 2015 IEEE international conference on computer vision, ICCV 2015. IEEE Computer Society, Santiago, pp 3074–3082

  43. Ma C, Yang X, Zhang C, Yang M-H (2015) Long-term correlation tracking. In: IEEE conference on computer vision and pattern recognition, CVPR 2015. IEEE Computer Society, Boston, pp 5388–5396

  44. Mei X, Hong Z, Prokhorov D V, Tao D (2015) Robust multitask multiview tracking in videos. IEEE Trans Neural Netw Learn Syst 26(11):2874–2890

    Article  MathSciNet  Google Scholar 

  45. Mueller M, Smith N, Ghanem B (2017) Context-aware correlation filter tracking. In: 2017 IEEE conference on computer vision and pattern recognition, CVPR 2017. IEEE Computer Society, Honolulu, pp 1387–1395

  46. Nam H, Baek M, Han B (2016) Modeling and propagating cnns in a tree structure for visual tracking. CoRR arXiv:1608.07242

  47. Nam H, Han B (2016) Learning multi-domain convolutional neural networks for visual tracking. In: 2016 IEEE conference on computer vision and pattern recognition, CVPR 2016. IEEE Computer Society, Las Vegas, pp 4293–4302

  48. Ning J, Yang J, Jiang S, Zhang L, Yang M-H (2016) Object tracking via dual linear structured SVM and explicit feature map. In: 2016 IEEE conference on computer vision and pattern recognition, CVPR 2016. IEEE Computer Society, Las Vegas, pp 4266–4274

  49. Qi Y, Zhang S, Qin L, Huang Q, Yao H, Lim J, Yang M-H (2019) Hedging deep features for visual tracking. IEEE Trans Pattern Anal Mach Intell 41(5):1116–1130

    Article  Google Scholar 

  50. Radford A, Metz L, Chintala S (2016) Unsupervised representation learning with deep convolutional generative adversarial networks. In: Bengio Y, LeCun Y (eds) 4th international conference on learning representations, ICLR 2016. Conference Track Proceedings, San Juan

  51. Rao C, Yao C, Bai X, Qiu W, Liu W (2012) Online random ferns for robust visual tracking. In: Proceedings of the 21st international conference on pattern recognition, ICPR 2012. IEEE Computer Society, Tsukuba, pp 1447–1450

  52. Roffo G, Melzi S (2016) Online feature selection for visual tracking. In: Wilson R C, Hancock E R, Smith W A P (eds) Proceedings of the British machine vision conference 2016, BMVC 2016. BMVA Press, New York

  53. Ross D A, Lim J, Lin R-S, Yang M-H (2008) Incremental learning for robust visual tracking. Int J Comput Vis 77(1-3):125–141

    Article  Google Scholar 

  54. Su Z, Li J, Chang J, Du B, Xiao Y (2020) Real-time visual tracking using complementary kernel support correlation filters. Frontiers Comput Sci 14(2):417–429

    Article  Google Scholar 

  55. Sui Y, Wang G, Zhang L (2018) Correlation filter learning toward peak strength for visual tracking. IEEE Trans Cybern 48(4):1290–1303

    Article  Google Scholar 

  56. Tao R, Gavves E, Smeulders A W M (2016) Siamese instance search for tracking. In: 2016 IEEE conference on computer vision and pattern recognition, CVPR 2016. IEEE Computer Society, Las Vegas, pp 1420–1429

  57. Valmadre J, Bertinetto L, Henriques J F, Vedaldi A, Torr P H S (2017) End-to-end representation learning for correlation filter based tracking. In: 2017 IEEE conference on computer vision and pattern recognition, CVPR 2017. IEEE Computer Society, Honolulu, pp 5000–5008

  58. Vedaldi A, Lenc K (2015) Matconvnet: Convolutional neural networks for matlab. In: Proceedings of the 23rd ACM international conference on multimedia, pp 689–692

  59. Wang D, Lu H (2012) Object tracking via 2dpca and l1 regularization. IEEE Signal Process Lett 19(11):711–714

    Article  Google Scholar 

  60. Wang D, Lu H (2014) Visual tracking via probability continuous outlier model. In: 2014 IEEE conference on computer vision and pattern recognition, CVPR 2014. IEEE Computer Society, Columbus, pp 3478–3485

  61. Wang G, Qin X, Zhong F, Liu Y, Li H, Peng Q, Yang M-H (2015) Visual tracking via sparse and local linear coding. IEEE Trans Image Process 24(11):3796–3809

    Article  MathSciNet  Google Scholar 

  62. Wang L, Ouyang W, Wang X, Lu H (2015) Visual tracking with fully convolutional networks. In: 2015 IEEE international conference on computer vision, ICCV 2015. IEEE Computer Society, Santiago, pp 3119–3127

  63. Wang N, Wang J, Yeung D-Y (2013) Online robust non-negative dictionary learning for visual tracking. In: IEEE international conference on computer vision, ICCV 2013. IEEE Computer Society, Sydney, pp 657–664

  64. Wang X, Shrivastava A, Gupta A (2017) A-fast-rcnn: Hard positive generation via adversary for object detection. In: 2017 IEEE conference on computer vision and pattern recognition, CVPR 2017. IEEE Computer Society, Honolulu, pp 3039–3048

  65. Wang C, Zhang L, Xie L, Yuan J (2018) Kernel cross-correlator. In: McIlraith S A, Weinberger K Q (eds) Proceedings of the thirty-second aaai conference on artificial intelligence, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18). AAAI Press, New Orleans, pp 4179–4186

  66. Wu Y, Lim J, Yang M-H (2013) Online object tracking: A benchmark. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

  67. Xu T, Feng Z-H, Wu X-J, Kittler J (2019) Learning adaptive discriminative correlation filters via temporal consistency preserving spatial feature selection for robust visual object tracking. IEEE Trans Image Process 28(11):5596–5609

    Article  MathSciNet  Google Scholar 

  68. Yilmaz A, Javed O, Shah M (2006) Object tracking:a survey. ACM Comput Surv 38(4):13

    Article  Google Scholar 

  69. Yin M, Gao J, Lin Z (2016) Laplacian regularized low-rank representation and its applications. IEEE Trans Pattern Anal Mach Intell 38(3):504–517

    Article  Google Scholar 

  70. Zhang K, Zhang L, Yang M-H (2012) Real-time compressive tracking. In: Fitzgibbon A W, Lazebnik S, Perona P, Sato Y, Schmid C (eds) Computer vision - ECCV 2012 - 12th European conference on computer vision, proceedings, part III, Lecture Notes in Computer Science, vol 7574. Springer, Florence, pp 864–877

  71. Zhang K, Zhang L, Yang M-H, Zhang D (2013) Fast tracking via spatio-temporal context learning. CoRR arXiv:1311.1939

  72. Zhang K, Song H (2013) Real-time visual tracking via online weighted multiple instance learning. Pattern Recognit 46(1):397–411

    Article  Google Scholar 

  73. Zhang J, Liu K, Cheng F, Li Y (2014) Visual tracking with randomly projected ferns. Signal Process Image Commun 29(9):987–997

    Article  Google Scholar 

  74. Zhang S, Sui Y, Yu X, Zhao S, Zhang L (2015) Hybrid support vector machines for robust object tracking. Pattern Recognit 48(8):2474–2488

    Article  Google Scholar 

  75. Zhang S, Yu X, Sui Y, Zhao S, Zhang L (2015) Object tracking with multi-view support vector machines. IEEE Trans Multimed 17(3):265–278

    Google Scholar 

  76. Zhang S, Zhao S, Sui Y, Zhang L (2015) Single object tracking with fuzzy least squares support vector machine. IEEE Trans Image Process 24 (12):5723–5738

    Article  MathSciNet  Google Scholar 

  77. Zhang B, Li Z, Cao X, Ye Q, Chen C, Shen L, Perina A, Ji R (2017) Output constraint transfer for kernelized correlation filter in tracking. IEEE Trans Syst Man Cybern Syst 47(4):693–703

    Article  Google Scholar 

  78. Zhang L, Suganthan P N (2017) Robust visual tracking via co-trained kernelized correlation filters. Pattern Recognit 69:82–93

    Article  Google Scholar 

  79. Zhang L, Varadarajan J, Suganthan P N, Ahuja N, Moulin P (2017) Robust visual tracking using oblique random forests. In: 2017 IEEE conference on computer vision and pattern recognition, CVPR 2017. IEEE Computer Society, Honolulu, pp 5825–5834

  80. Zhang S, Sui Y, Zhao S, Zhang L (2017) Graph-regularized structured support vector machine for object tracking. IEEE Trans Circ Syst Video Techn 27(6):1249–1262

    Article  Google Scholar 

  81. Zhong W, Lu H, Yang M-H (2014) Robust object tracking via sparse collaborative appearance model. IEEE Trans Image Process 23(5):2356–2368

    Article  MathSciNet  Google Scholar 

  82. Zhu S, Fang Z, Gao F (2018) Hierarchical convolutional features for end-to-end representation-based visual tracking. Mach Vis Appl 29(6):955–963

    Article  Google Scholar 

  83. Zuo W, Wu X, Lin L, Zhang L, Yang M-H (2019) Learning support correlation filters for visual tracking. IEEE Trans Pattern Anal Mach Intell 41(5):1158–1172

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported by ‘Leading Talents of Shandong University of Science and Technology’, ‘863 project Physical Model Based Dynamic Evolution Technology of Complex Scene’ (2015AA016404), ‘Shandong Province Higher Educational Science and Technology Program’ (J17KA075) and ‘National Nature Science Foundation of China’ (61801270).

Author information

Authors and Affiliations

  1. College of Electrical Engineering and Automation, Shandong University of Science and Technology, Qingdao, 266590, Shandong Province, People’s Republic of China

    Xiaowei An

  2. College of Electronics and Information Engineering, Shandong University of Science and Technology, Qingdao, 266590, Shandong Province, People’s Republic of China

    Quanquan Liang & Nongliang Sun

Authors
  1. Xiaowei An
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Quanquan Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nongliang Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Quanquan Liang or Nongliang Sun.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

An, X., Liang, Q. & Sun, N. Multi-kernel support correlation filters with temporal filtering constraint for object tracking. Multimed Tools Appl 80, 14041–14073 (2021). https://doi.org/10.1007/s11042-020-10345-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-10345-2

Keywords

Search

Navigation