Skip to main content
SpringerLink
Log in

A multiple feature fused model for visual object tracking via correlation filters

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Common tracking algorithms only use a single feature to describe the target appearance, which makes the appearance model easily disturbed by noise. Furthermore, the tracking performance and robustness of these trackers are obviously limited. In this paper, we propose a novel multiple feature fused model into a correlation filter framework for visual tracking to improve the tracking performance and robustness of the tracker. In different tracking scenarios, the response maps generated by the correlation filter framework are different for each feature. Based on these response maps, different features can use an adaptive weighting function to eliminate noise interference and maintain their respective advantages. It can enhance the tracking performance and robustness of the tracker efficiently. Meanwhile, the correlation filter framework can provide a fast training and accurate locating mechanism. In addition, we give a simple yet effective scale variation detection method, which can appropriately handle scale variation of the target in the tracking sequences. We evaluate our tracker on OTB2013/OTB50/OBT2015 benchmarks, which are including more than 100 video sequences. Extensive experiments on these benchmark datasets demonstrate that the proposed MFFT tracker performs favorably against the state-of-the-art trackers.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Adam A, Rivlin E, Shimshoni I (2006) Robust fragments-based tracking using the integral histogram. In: IEEE conference on computer vision and pattern recognition, pp 798–805

  2. Avidan S (2007) Ensemble tracking. IEEE Trans Pattern Anal Mach Intell 29 (2):261–271

    Article  Google Scholar 

  3. Babenko B, Yang MH, Belongie S (2009) Visual tracking with online multiple instance learning. In: IEEE conference on computer vision and pattern recognition, pp 983–990

  4. Bao C, Wu Y, Ling H, Ji H (2012) Real time robust l1 tracker using accelerated proximal gradient approach. In: IEEE conference on computer vision and pattern recognition, pp 1830–1837

  5. Bertinetto L, Valmadre J, Golodetz S, Miksik O, Torr P (2016) Staple: complementary learners for real-time tracking. In: IEEE conference on computer vision and pattern recognition, pp 1401–1409

  6. Bibi A, Ghanem B (2015) Multi-template scale-adaptive kernelized correlation filters. In: IEEE international conference on computer vision workshop, pp 613–620

  7. Bolme DS, Beveridge JR, Draper BA, Lui YM (2010) Visual object tracking using adaptive correlation filters. In: IEEE conference on computer vision and pattern recognition, pp 2544–2550

  8. Cehovin L, Kristan M, Leonardis A (2014) Is my new tracker really better than yours?. In: Applications of computer vision, pp 540–547

  9. Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: IEEE conference on computer vision and pattern recognition, pp 886–893

  10. Danelljan M, Hager G, Khan FS, Felsberg M (2014) Accurate scale estimation for robust visual tracking. In: British machine vision conference, vol 65, pp 1–11

  11. Danelljan M, Hager G, Khan FS, Felsberg M (2015) Learning spatially regularized correlation filters for visual tracking. In: IEEE international conference on computer vision, pp 4310–4318

  12. Danelljan M, Khan FS, Felsberg M, Weijer JVD (2014) Adaptive color attributes for real-time visual tracking. In: IEEE conference on computer vision and pattern recognition, pp 1090–1097

  13. Everingham M, Gool LV, Williams CKI, Winn J, Zisserman A (2010) The pascal visual object classes (voc) challenge. Int J Comput Vis 88(2):303–338

    Article  Google Scholar 

  14. Fan N, Li J, He Z, Zhang C, Li X (2019) Region-filtering correlation tracking. Knowl-Based Syst 172:95–103

    Article  Google Scholar 

  15. Galoogahi HK, Sim T, Lucey S (2013) Multi-channel correlation filters. In: IEEE international conference on computer vision, pp 3072–3079

  16. Gao J, Ling H, Hu W, Xing J (2014) Transfer learning based visual tracking with gaussian processes regression. European Conference on Computer Vision, 188–203

  17. Hare S, Golodetz S, Saffari A, et al. (2016) Struck: Structured output tracking with kernels. IEEE Trans Pattern Anal Mach Intell 38(10):2096–2109

    Article  Google Scholar 

  18. He Z, Chung AC (2010) 3-D b-spline wavelet-based local standard deviation (bwlsd): its application to edge detection and vascular segmentation in magnetic resonance angiography. Int J Comput Vis 87(3):235–265

    Article  Google Scholar 

  19. He Z, Li X, You X, Tao D, Tang Y (2016) Connected component model for multi-object tracking. IEEE Trans Image Process 25(8):3698–3711

    Article  MathSciNet  MATH  Google Scholar 

  20. He Z, Yi S, Cheung Y-M, You X, Tang Y (2017) Robust object tracking via key patch sparse representation. IEEE Trans Cybern 47:354–364

    Google Scholar 

  21. He Z, You X, Tang Y (2008) Writer identification of chinese handwriting documents using hidden markov tree model. Pattern Recogn 41(4):1295–1307

    Article  MATH  Google Scholar 

  22. He Z, You X, Zhou L, Cheung Y-M, Du J (2010) Writer identification using fractal dimension of wavelet subbands in gabor domain. Integrated Computer Aided Engineering 17(17):157–165

    Article  Google Scholar 

  23. Henriques JF, Caseiro R, Martins P, Batista J (2012) Exploiting the circulant structure of tracking-by-detection with kernels. In: European conference on computer vision, pp 702–715

  24. Henriques JF, Rui C, Martins P, Batista J (2014) High-speed tracking with kernelized correlation filters. IEEE Trans Pattern Anal Mach Intell 37(3):583–596

    Article  Google Scholar 

  25. Hong Z, Mei X, Prokhorov D, Tao D (2014) Tracking via robust multi-task multi-view joint sparse representation. In: IEEE international conference on computer vision, pp 649–656

  26. Jia X, Lu H, Yang MH (2012) Visual tracking via adaptive structural local sparse appearance model. In: IEEE conference on computer vision and pattern recognition, pp 1822–1829

  27. Jian M, Lam K, Dong J, Shen L (2015) Visual-patch-attention-aware saliency detection. IEEE Trans Cybern 45(8):1575–1586

    Article  Google Scholar 

  28. Jian M, Qiang Q, Dong J, Yin Y, Lam KM (2018) Integrating qdwd with pattern distinctness and local contrast for underwater saliency detection <î. J Vis Commun Image Represent 53:31–41

    Article  Google Scholar 

  29. Kalal Z, Mikolajczyk K, Matas J (2012) Tracking-learning-detection. IEEE Trans Pattern Anal Mach Intell 34(7):1409–1422

    Article  Google Scholar 

  30. Kwon J, Lee KM (2010) Visual tracking decomposition. In: IEEE conference on computer vision and pattern recognition, pp 1269–1276

  31. Li F, Yao Y, Li P, Zhang D, Zuo W, Yang MH (2017) Integrating boundary and center correlation filters for visual tracking with aspect ratio variation. In: IEEE international conference on computer vision workshop, pp 2001–2009

  32. Li X, Liu Q, Fan N, He Z, Wang H (2019) Hierarchical spatial-aware siamese network for thermal infrared object tracking. Knowl-Based Syst 166:71–81

    Article  Google Scholar 

  33. Li X, Liu Q, He Z, Wang H, Zhang C, Chen WS (2016) A multi-view model for visual tracking via correlation filters. Knowl-Based Syst 113:88–99

    Article  Google Scholar 

  34. Li X, Ma C, Wu B, He Z, Yang M. (2019) Target-aware deep tracking, arXiv:1904.01772

  35. Li Y, Zhu J (2014) A scale adaptive kernel correlation filter tracker with feature integration. In: European conference on computer vision, pp 254–265

  36. Li Y, Zhu J, Hoi SCH (2015) Reliable patch trackers: robust visual tracking by exploiting reliable patches. In: IEEE conference on computer vision and pattern recognition, pp 353–361

  37. Liu Q, Lu X, He Z, Zhang C, Chen W (2017) Deep convolutional neural networks for thermal infrared object tracking. Knowl-Based Syst 134:189–198

    Article  Google Scholar 

  38. Liu S, Zhang T, Cao X, Xu C (2016) Structural correlation filter for robust visual tracking. In: IEEE conference on computer vision and pattern recognition, pp 4312–4320

  39. Liu T, Wang G, Yang Q (2015) Real-time part-based visual tracking via adaptive correlation filters. In: IEEE conference on computer vision and pattern recognition, pp 4902–4912

  40. Lu X, Lei H, Hao Z (2010) Automatic camshift tracking algorithm based on multi-feature. J Comput Appl 30(3):650–652

    Google Scholar 

  41. Ma L, Lu J, Feng J, Zhou J (2016) Multiple feature fusion via weighted entropy for visual tracking. In: IEEE international conference on computer vision, pp 3128–3136

  42. Ma X, Liu Q, He Z, Zhang X, Chen WS (2016) Visual tracking via exemplar regression model. Knowl-Based Syst 106:26–37

    Article  Google Scholar 

  43. Ou W, You X, Tao D, Zhang P, Tang Y, Zhu Z (2014) Robust face recognition via occlusion dictionary learning. Pattern Recogn 47(4):1559–1572

    Article  Google Scholar 

  44. Ou W, Yuan D, Liu Q, Cao Y (2018) Object tracking based on online representative sample selection via non-negative least square. Multimed Tools Appl 77 (9):10569–10587

    Article  Google Scholar 

  45. Qi Y, Zhang S, Qin L, Yao H, Huang Q, Lim J, Yang MH (2016) Hedged deep tracking. In: IEEE conference on computer vision and pattern recognition, pp 4303–4311

  46. Tang M, Feng J (2015) Multi-kernel correlation filter for visual tracking. In: IEEE international conference on computer vision, pp 3038–3046

  47. Valmadre J, Bertinetto L, Henriques JF, Vedaldi A, Torr PHS (2017) End-to-end representation learning for correlation filter based tracking. In: IEEE conference on computer vision and pattern recognition, pp 2805–2813

  48. Wang N, Shi J, Yeung DY, Jia J (2015) Understanding and diagnosing visual tracking systems. In: IEEE international conference on computer vision, pp 3101–3109

  49. Wang Q, Tang S, Zhai D, Hu X (2018) Salience based object tracking in complex scenes. Neurocomputing 314:132–142

    Article  Google Scholar 

  50. Wu Y, Lim J, Yang M-H (2015) Object tracking benchmark. IEEE Trans Pattern Anal Mach Intell 37(9):1834–1848

    Article  Google Scholar 

  51. Wu Y, Lim J, Yang MH (2013) Online object tracking: a benchmark. In: IEEE conference on computer vision and pattern recognition, pp 2411–2418

  52. Yi S, Lai Z, He Z, Cheung Y-M, Liu Y (2017) Joint sparse principal component analysis. Pattern Recogn 61:524–536

    Article  Google Scholar 

  53. Yin Z, Porikli F, Collins RT (2008) Likelihood map fusion for visual object tracking. In: IEEE workshop on applications of computer vision, pp 1–7

  54. Yuan D, Lu X, Li D, He Z, Luo N (2017) Multiple feature fused for visual tracking via correlation filters. In: International conference on security, pattern analysis, and cybernetics, pp 88–93

  55. Yuan D, Lu X, Li D, Liang Y, Zhang X (2018) Particle filter re-detection for visual tracking via correlation filters. Multimed Tools Appl, pp 1–25

  56. Zhang K, Liu Q, Wu Y, Yang MH (2016) Robust visual tracking via convolutional networks without training. IEEE Trans Image Process 25(4):1779–1792

    MathSciNet  MATH  Google Scholar 

  57. Zhang K, Zhang L, Liu Q, Zhang D, Yang MH (2014) Fast visual tracking via dense spatio-temporal context learning. In: European conference on computer vision, pp 127–141

  58. Zhang T, Bibi A, Ghanem B (2016) In defense of sparse tracking: circulant sparse tracker. In: IEEE conference on computer vision and pattern recognition, pp 3880–3888

  59. Zhang T, Xu C, Yang MH (2017) Multi-task correlation particle filter for robust object tracking. In: IEEE conference on computer vision and pattern recognition, pp 4819–4827

  60. Zhang T, Xu C, Yang MH (2019) Learning multi-task correlation particle filters for visual tracking. IEEE Trans Pattern Anal Mach Intell 41(2):365–378

    Article  Google Scholar 

  61. Zhong W, Lu H, Yang MH (2012) Robust object tracking via sparsity-based collaborative model. In: IEEE conference on computer vision and pattern recognition, pp 1838–1845

  62. Zhou Y, Rao C, Lu Q, Bai X, Liu W (2011) Multiple feature fusion for object tracking. In: Sino-foreign-interchange conference on intelligent science and intelligent data engineering, pp 145–152

  63. Zhou Z, Wu D, Peng X, Zhu Z, Luo K (2014) Object tracking based on camshift with multi-feature fusion. J Softw 9(1):147–153

    Google Scholar 

Download references

Acknowledgment

This research was supported by the Shenzhen Research Council (Grant No. JCYJ2016040 6161948211, JCYJ20160226201453085, JSGG20150331152017052, JCYJ20160531194006833), by the National Natural Science Foundation of China (Grant No. 61672183, 61272366, 61672444), by Science and Technology Planning Project of Guangdong Province (Grant No. 2016B090918047).

Author information

Authors and Affiliations

  1. School of Computer Science, Harbin Institute of Technology (Shenzhen), Shenzhen, China

    Di Yuan & Donghao Li

  2. School of Science, Harbin Institute of Technology (Shenzhen), Shenzhen, China

    Di Yuan & Xinming Zhang

  3. College of Finance and Statistics, Hunan University, Changsha, China

    Jiaqi Liu

Authors
  1. Di Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xinming Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jiaqi Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Donghao Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Di Yuan.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yuan, D., Zhang, X., Liu, J. et al. A multiple feature fused model for visual object tracking via correlation filters. Multimed Tools Appl 78, 27271–27290 (2019). https://doi.org/10.1007/s11042-019-07828-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-019-07828-2

Keywords

Search

Navigation