Skip to main content
SpringerLink
Log in

Real-time visual tracking using complementary kernel support correlation filters

  • Research Article
  • Published:
Frontiers of Computer Science Aims and scope Submit manuscript

Abstract

Despite demonstrated success of SVM based trackers, their performance remains a boosting room if carefully considering the following factors: first, the tradeoff between sampling and budgeting samples affects tracking accuracy and efficiency much; second, how to effectively fuse different types of features to learn a robust target representation plays a key role in tracking accuracy. In this paper, we propose a novel SVM based tracking method that handles the first factor with the help of the circulant structures of the samples and the second one by a multi-kernel learning mechanism. Specifically, we formulate an SVM classification model for visual tracking that incorporates two types of kernels whose matrices are circulant, fully taking advantage of the complementary traits of the color and HOG features to learn a robust target representation. Moreover, it is fortunate that the SVM model has a closed-form solution in terms of both the classifier weights and the kernel weights, and both can be efficiently computed via fast Fourier transforms (FFTs). Extensive evaluations on OTB100 and VOT2016 visual tracking benchmarks demonstrate that the proposed method achieves a favorable performance against various state-of-the-art trackers with a speed of 50 fps on a single CPU.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Li X, Hu W, Shen C, Zhang Z, Dick A, Hengel A V D. A survey of appearance models in visual object tracking. ACM Transactions on Intelligent Systems and Technology, 2013, 4(4): 1–48

    Google Scholar 

  2. Song Y, Ma C, Wu X, Gong L, Bao L, Zuo W, Shen C, Lau R, Yang M H. VITAL: visual tracking via adversarial learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018, 8990–8999

    Google Scholar 

  3. Zhang K, Zhang L, Yang M H. Real-time compressive tracking. In: Proceedings of European Conference on Computer Vision. 2012, 864–877

    Google Scholar 

  4. Jia X, Lu H, Yang M H. Visual tracking via adaptive structural local sparse appearance model. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2012, 1822–1829

    Google Scholar 

  5. Zhang K, Zhang L, Yang MH. Fast compressive tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36(10): 2002–2015

    Google Scholar 

  6. Zhong W, Lu H, Yang M H. Robust object tracking via sparsity-based collaborative model. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2012, 1838–1845

    Google Scholar 

  7. Chen W, Zhang K, Liu Q. Robust visual tracking via patch based kernel correlation filters with adaptive multiple feature ensemble. Neurocomputing, 2016, 214: 607–617

    Google Scholar 

  8. Yang J, Zhang K, Liu Q. Robust object tracking by online fisher discrimination boosting feature selection. Computer Vision and Image Understanding, 2016, 153: 100–108

    Google Scholar 

  9. Song H, Zheng Y, Zhang K. Robust visual tracking via self-similarity learning. Electronics Letters, 2016, 53(1): 20–22

    Google Scholar 

  10. Song H, Wang G, Zhang K. Hyperspectral image denoising via lowrank matrix recovery. Remote Sensing Letters, 2014, 5(10): 872–881

    Google Scholar 

  11. Zhang K, Liu Q, Song H, Li X. A variational approach to simultaneous image segmentation and bias correction. IEEE Transactions on Cybernetics, 2015, 45(8): 1426–1437

    Google Scholar 

  12. Zhang K, Zhang L, Lam K M, Zhang D. A level set approach to image segmentation with intensity inhomogeneity. IEEE Transactions on Cybernetics, 2016, 46(2): 546–557

    Google Scholar 

  13. Song H. Robust visual tracking via online informative feature selection. Electronics Letters, 2014, 50(25): 1931–1933

    Google Scholar 

  14. Zhang K, Song H. Real-time visual tracking via online weighted multiple instance learning. Pattern Recognition, 2013, 46(1): 397–411

    MathSciNet  MATH  Google Scholar 

  15. Zhang K, Liu Q, Wu Y, Yang M H. Robust visual tracking via convolutional networks without training. IEEE Transactions on Image Processing, 2016, 25(4): 1779–1792

    MathSciNet  MATH  Google Scholar 

  16. Wu Y, Lim J, Yang MH. Online object tracking: a benchmark. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2013, 2411–2418

    Google Scholar 

  17. Wang N, Shi J, Yeung D Y, Jia J. Understanding and diagnosing visual tracking systems. In: Proceedings of IEEE International Conference on Computer Vision. 2015, 3101–3109

    Google Scholar 

  18. Avidan S. Support vector tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2004, 26(8): 1064–1072

    Google Scholar 

  19. Bai Y, Tang M. Robust tracking via weakly supervised ranking SVM. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2012, 1854–1861

    Google Scholar 

  20. Hare S, Golodetz S, Saffari A, Vineet V, Cheng MM, Hicks S L, Torr P H. Struck: structured output tracking with kernels. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(104): 2096–2109

    Google Scholar 

  21. Zhang J, Ma S, Sclaroff S. MEEM: robust tracking via multiple experts using entropy minimization. In: Proceedings of European Conference on Computer Vision. 2014, 188–203

    Google Scholar 

  22. Song H, Huang B, Liu Q, Zhang K. Improving the spatial resolution of landsat TM/ETM+ through fusion with SPOT5 images via learningbased super-resolution. IEEE Transactions on Geoscience and Remote Sensing, 2015, 53(3): 1195–1204

    Google Scholar 

  23. Ning J, Yang J, Jiang S, Zhang L, Yang M H. Object tracking via dual linear structured SVM and explicit feature map. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016, 4266–4274

    Google Scholar 

  24. Song H. Active contours driven by regularised gradient flux flows for image segmentation. Electronics Letters, 2014, 50(14): 992–994

    Google Scholar 

  25. Zuo WM, Wu X H, Lin L, Zhang L, Yang M H. Learning support correlation filters for visual tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 41(5): 1158–1172

    Google Scholar 

  26. Song H, Liu Q, Wang G, Hang R, Huang B. Spatiotemporal satellite image fusion using deep convolutional neural networks. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2018, 11(3): 821–829

    Google Scholar 

  27. Song H, Zheng Y, Zhang K. Efficient algorithm for piecewise-smooth model with approximately explicit solutions. Electronics Letters, 2017, 53(4): 233–235

    Google Scholar 

  28. Wang M, Liu Y, Huang Z. Large margin object tracking with circulant feature maps. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017, 4021–4029

    Google Scholar 

  29. Henriques J F, Caseiro R, Martins P, Batista J. High-speed tracking with kernelized correlation filters. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(3): 583–596

    Google Scholar 

  30. Bertinetto L, Valmadre J, Golodetz S, Miksik O, Torr P H. Staple: complementary learners for real-time tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016, 1401–1409

    Google Scholar 

  31. Zhang K, Zhang L, Liu Q, Zhang D, Yang M H. Fast visual tracking via dense spatio-temporal context learning. In: Proceedings of European Conference on Computer Vision. 2014, 127–141

    Google Scholar 

  32. Song H, Wang G, Cao A, Liu Q, Huang B. Improving the spatial resolution of FY-3 microwave radiation imager via fusion with FY-3/MERSI. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2017, 10(7): 3055–3063

    Google Scholar 

  33. Bolme D S, Beveridge J R, Draper B A, Lui Y M. Visual object tracking using adaptive correlation filters. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2010, 2544–2550

    Google Scholar 

  34. Zhang K, Liu Q, Yang J, Yang M H. Visual tracking via boolean map representations. Pattern Recognition, 2018, 81: 147–160

    Google Scholar 

  35. Zhang K, Li X, Song H, Liu Q, Lian W. Visual tracking using spatiotemporally nonlocally regularized correlation filter. Pattern Recognition, 2018, 83: 185–195

    Google Scholar 

  36. Tang M, Feng J. Multi-kernel correlation filter for visual tracking. In: Proceedings of the IEEE International Conference on Computer Vision. 2015, 3038–3046

    Google Scholar 

  37. Song H, Wang G, Zhang K. Multiple change detection for multispectral remote sensing images via joint sparse representation. Optical Engineering, 2014, 53(12): 123103

    Google Scholar 

  38. Qi Y, Qin L, Zhang J, Zhang S, Huang Q, Yang M H. Structure-aware local sparse coding for visual tracking. IEEE Transactions on Image Processing, 2018, 27(8): 3857–3869

    MathSciNet  MATH  Google Scholar 

  39. Sun C, Wang D, Lu H, Yang M H. Learning spatial-aware regressions for visual tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018, 8962–8970

    Google Scholar 

  40. Qi Y, Zhang S, Qin L, Huang Q, Yao H, Lim J, Yang M H. Hedging deep features for visual tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 41(5): 1116–1130

    Google Scholar 

  41. Sun C, Wang D, Lu H, Yang M H. Correlation tracking via joint discrimination and reliability learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018, 489–497

    Google Scholar 

  42. Zhang T, Liu S, Xu C, Liu B, Yang M H. Correlation particle filter for visual tracking. IEEE Transactions on Image Processing, 2018, 27(6): 2676–2687

    MathSciNet  MATH  Google Scholar 

  43. Zhang T, Xu C, Yang M H. Learning multi-task correlation particle filters for visual tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 41(2): 365–378

    Google Scholar 

  44. Ma C, Huang J B, Yang X, Yang M H. Hierarchical convolutional features for visual tracking. In: Proceedings of the IEEE International Conference on Computer Vision. 2015, 3074–3082

    Google Scholar 

  45. Varma M, Ray D. Learning the discriminative power-invariance tradeoff. In: Proceedings of International Conference on Computer Vision. 2007, 1–8

    Google Scholar 

  46. Wu Y, Lim J, Yang M H. Object tracking benchmark. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(9): 1834–1848

    Google Scholar 

  47. Tang F, Brennan S, Zhao Q, Tao H. Co-tracking using semi-supervised support vector machines. In: Proceedings of International Conference on Computer Vision. 2007, 1–8

    Google Scholar 

  48. Li X, Dick A, Wang H, Shen C, Van Den Hengel A. Graph mode-based contextual kernels for robust SVM tracking. In: Proceedings of International Conference on Computer Vision. 2011, 1156–1163

    Google Scholar 

  49. Supancic J S, Ramanan D. Self-paced learning for long-term tracking. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2013, 2379–2386

    Google Scholar 

  50. Danelljan M, Shahbaz Khan F, Felsberg M, Van De Weijer J. AdapZhenyang SU et al. Real-time visual tracking using complementary kernel support correlation filters 13 tive color attributes for real-time visual tracking. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2014, 1090–1097

    Google Scholar 

  51. Li Y, Zhu J. A scale adaptive kernel correlation filter tracker with feature integration. In: Proceedings of European Conference on Computer Vision. 2014, 254–265

    Google Scholar 

  52. Ma C, Yang X, Zhang C, Yang M H. Long-term correlation tracking. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2015, 5388–5396

    Google Scholar 

  53. Hong Z, Chen Z, Wang C, Mei X, Prokhorov D, Tao D. Multi-store tracker (muster): a cognitive psychology inspired approach to object tracking. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2015, 749–758

    Google Scholar 

  54. Choi J, Jin Chang H, Jeong J, Demiris Y, Young Choi J. Visual tracking using attention-modulated disintegration and integration. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016, 4321–4330

    Google Scholar 

  55. Danelljan M, Hager G, Shahbaz Khan F, Felsberg M. Learning spatially regularized correlation filters for visual tracking. In: Proceedings of the IEEE International Conference on Computer Vision. 2015, 4310–4318

    Google Scholar 

  56. Qi Y, Zhang S, Qin L, Yao H, Huang Q, Lim J, Yang M H. Hedged deep tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016, 4303–4311

    Google Scholar 

  57. Liu S, Zhang T, Cao X, Xu C. Structural correlation filter for robust visual tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016, 4312–4320

    Google Scholar 

  58. Zhang T, Xu C, Yang M H. Multi-task correlation particle filter for robust object tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017, 4335–4343

    Google Scholar 

  59. Kiani Galoogahi H, Fagg A, Lucey S. Learning background-aware correlation filters for visual tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017, 1135–1143

    Google Scholar 

  60. Mueller M, Smith N, Ghanem B. Context-aware correlation filter tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017, 1396–1404

    Google Scholar 

  61. Lukezic A, Vojír T, Zajc L C, Matas J, Kristan M. Discriminative correlation filter with channel and spatial reliability. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017, 6309–6318

    Google Scholar 

  62. Scholkopf B, Smola A J. Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. Massachusetts: MIT Press, 2001

    Google Scholar 

  63. Lanckriet G R, De Bie T, Cristianini N, Jordan M I, Noble W S. A statistical framework for genomic data fusion. Bioinformatics, 2004, 20(16): 2626–2635

    Google Scholar 

  64. Kristan M, Leonardis A, Matas J, Felsberg M, Pflugfelder R, Čehovin L, Vojír T, Häger G, Lukežič A, Fernández G, others. The visual object tracking VOT2016 challenge results. In: Proceedings of European Conference on Computer Vision Workshops. 2016, 777–823

    Google Scholar 

Download references

Acknowledgments

This work was supported in part by the National Nature Science Foundation of China (Grant No. 61471274).

Author information

Authors and Affiliations

  1. School of Computer Science, Wuhan University, Wuhan, 430072, China

    Zhenyang Su, Jing Li, Jun Chang, Bo Du & Yafu Xiao

  2. Department of Digital Media Technology, Huanggang Normal University, Huangzhou, 438000, China

    Zhenyang Su

Authors
  1. Zhenyang Su
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jing Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jun Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bo Du
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yafu Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jing Li.

Additional information

Zhenyang Su received the Master degree in educational technology from Nanjing Normal University, China in 2009. Now he is a PhD candidate in Computer School of Wuhan University, China. His research interests include computer vision, visual tracking, and object recognition.

Jing Li received the PhD degree from Wuhan University, China in 2006. He is currently a professor in Computer School of Wuhan University, China. His research interests include data mining and multimedia technology.

Jun Chang received the PhD degree from Wuhan University, China in 2011. He is currently an assistant professor in School of Computer Science, Wuhan University, China. His current research interests include computer vision, large-scale machine learning, and stream data mining.

Bo Du (Senior Member, IEEE) received the BS degree and the PhD degree in photogrammetry and remote sensing from State Key Lab of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, China in 2005 and 2010, respectively. His major research interests include data mining, pattern recognition, hyperspectral image processing, and signal processing.

Yafu Xiao received the ME degree in Computer School of Hubei University of Technology, China in 2014. He is currently pursuing the PhD degree with Computer School of Wuhan University, China. His research interests include visual tracking, face recognition, and object recognition.

Electronic supplementary material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Su, Z., Li, J., Chang, J. et al. Real-time visual tracking using complementary kernel support correlation filters. Front. Comput. Sci. 14, 417–429 (2020). https://doi.org/10.1007/s11704-018-8116-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11704-018-8116-1

Keywords

Search

Navigation