Skip to main content
SpringerLink
Log in

Multi-view correlation tracking with adaptive memory-improved update model

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Recently, some researchers concentrate on applying multi-view learning to the correlation filter tracking to achieve both the efficiency and accuracy. However, most of them fail to effectively collaborate multiple views to deal with more complex environment. Moreover, their methods are prone to drift in case of long-term occlusion due to the memory loss. In this paper, we propose a novel multi-view correlation filters-based tracker for robust visual tracking. First, we present an adaptive multi-view collaboration strategy to highlight different contributions of different views by jointly considering the reliability and discrimination. Second, an effective memory-improved model update rule is introduced to avoid falling into a contaminated target model. Compared with the conventional linear interpolation update rule, it can effectively deal with long-term occlusion by improving the memory of historical models. Furthermore, instead of assigning a unified learning rate for all views in each frame, we design varying learning rates for different views according to their respective evaluations on the current tracking result, which can prevent the target models of all views from being contaminated at the same time. Finally, a failure-aware scale update scheme is developed to avoid noisy scale estimation in case of temporal tracking failure. Extensive experimental results on the recent benchmark demonstrate that our tracker performs favorably against other state-of-the-art trackers with a real-time performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. Note that the concept “multi-view” in this paper is different from that in 3D video domain. Generally, in visual object tracking, multiple views indicate that multiple features are extracted to capture the different appearance characteristics of the target within a single camera view. But in 3D video domain, multiple views commonly refer to multiple camera views of the same scene.

References

  1. Bertinetto L, Valmadre J, Golodetz S, Miksik O, Torr PHS (2016) Staple: complementary learners for real-time tracking. In: 2016 IEEE conference on computer vision and pattern recognition, CVPR 2016, Las Vegas, NV, USA, June 27–30, 2016, pp 1401–1409

  2. Bertinetto L, Valmadre J, Henriques JF, Vedaldi A, Torr PHS (2016) Fully-convolutional siamese networks for object tracking. In: Computer vision-ECCV 2016 Workshops, Amsterdam, The Netherlands, October 8–10 and 15–16, 2016, proceedings, part II, pp 850–865

  3. Bibi A, Ghanem B (2015) Multi-template scale-adaptive kernelized correlation filters. In: 2015 IEEE international conference on computer vision workshop, ICCV workshops 2015, Santiago, Chile, December 7–13, 2015, pp 613–620

  4. Bibi A, Mueller M, Ghanem B (2016) Target response adaptation for correlation filter tracking. In: Computer vision-ECCV 2016—14th European conference, Amsterdam, The Netherlands, October 11–14, 2016, proceedings, part VI, pp 419–433

  5. Bolme DS, Beveridge JR, Draper BA, Lui YM (2010) Visual object tracking using adaptive correlation filters. In: The twenty-third IEEE conference on computer vision and pattern recognition, CVPR 2010, San Francisco, CA, USA, 13–18 June 2010, pp 2544–2550

  6. Chen K, Tao W, Han S (2017) Visual object tracking via enhanced structural correlation filter. Inf Sci 394:232–245

    Article  Google Scholar 

  7. Chen W, An J, Li R, Fu L, Xie G, Bhuiyan MZA, Li K (2018) A novel fuzzy deep-learning approach to traffic flow prediction with uncertain spatial-temporal data features. Future Gener Comput Syst 89:78–88

    Article  Google Scholar 

  8. Danelljan M, Häger G, Khan FS, Felsberg M (2014) Accurate scale estimation for robust visual tracking. In: British machine vision conference, BMVC 2014, Nottingham, UK, September 1–5, 2014

  9. Danelljan M, Häger G, Khan FS, Felsberg M (2015) Convolutional features for correlation filter based visual tracking. In: 2015 IEEE international conference on computer vision workshop, ICCV workshops 2015, Santiago, Chile, December 7–13, 2015, pp 621–629

  10. Danelljan M, Häger G, Khan FS, Felsberg M (2015) Learning spatially regularized correlation filters for visual tracking. In: 2015 IEEE international conference on computer vision, ICCV 2015, Santiago, Chile, December 7–13, 2015, pp 4310–4318

  11. Dong X, Shen J, Yu D, Wang W, Liu J, Huang H (2017) Occlusion-aware real-time object tracking. IEEE Trans Multimed 19(4):763–771

    Article  Google Scholar 

  12. Fang Y, Zhang H, Ye Y, Li X (2014) Detecting hot topics from twitter: a multiview approach. J Inf Sci 40(5):578–593

    Article  Google Scholar 

  13. Gao J, Ling H, Hu W, Xing J (2014) Transfer learning based visual tracking with gaussian processes regression. In: Computer vision-ECCV 2014—13th European conference, Zurich, Switzerland, September 6–12, 2014, proceedings, part III, pp 188–203

  14. Hare S, Saffari A, Torr PHS (2011) Struck: structured output tracking with kernels. In: IEEE International conference on computer vision, ICCV 2011, Barcelona, Spain, November 6–13, 2011, pp 263–270. https://doi.org/10.1109/ICCV.2011.6126251

  15. Held D, Thrun S, Savarese S (2016) Learning to track at 100 FPS with deep regression networks. In: Computer vision-ECCV 2016—14th European conference, Amsterdam, The Netherlands, October 11–14, 2016, proceedings, part I, pp 749–765

  16. Henriques JF, Caseiro R, Martins P, Batista JP (2012) Exploiting the circulant structure of tracking-by-detection with kernels. In: Computer cision-ECCV 2012—12th European conference on computer vision, Florence, Italy, October 7–13, 2012, proceedings, part IV, pp 702–715. https://doi.org/10.1007/978-3-642-33765-9_50

  17. Henriques JF, Caseiro R, Martins P, Batista J (2015) High-speed tracking with kernelized correlation filters. IEEE Trans Pattern Anal Mach Intell 37(3):583–596. https://doi.org/10.1109/TPAMI.2014.2345390

    Article  Google Scholar 

  18. Hong S, You T, Kwak S, Han B (2015) Online tracking by learning discriminative saliency map with convolutional neural network. In: Proceedings of the 32nd international conference on machine learning, ICML 2015, Lille, France, 6–11 July 2015, pp 597–606

  19. Li G, Peng M, Nai K, Li Z, Li K (2018) Visual tracking via context-aware local sparse appearance model. J Vis Commun Image Represent 56:92–105

    Article  Google Scholar 

  20. Li H, Wu H, Zhang H, Lin S, Luo X, Wang R (2017) Distortion-aware correlation tracking. IEEE Trans Image Process 26(11):5421–5434

    Article  MathSciNet  Google Scholar 

  21. Li X, Hu W, Shen C, Zhang Z, Dick AR, van den Hengel A (2013) A survey of appearance models in visual object tracking. ACM Trans Intell Syst Technol 4(4):58:1–58:48. https://doi.org/10.1145/2508037.2508039

    Article  Google Scholar 

  22. Li X, Liu Q, He Z, Wang H, Zhang C, Chen W (2016) A multi-view model for visual tracking via correlation filters. Knowl Based Syst 113:88–99

    Article  Google Scholar 

  23. Li Y, Zhu J (2014) A scale adaptive kernel correlation filter tracker with feature integration. In: Computer vision-ECCV 2014 workshops—Zurich, Switzerland, September 6–7 and 12, 2014, proceedings, part II, pp 254–265. https://doi.org/10.1007/978-3-319-16181-5_18

  24. Li Y, Zhu J, Hoi SCH (2015) Reliable patch trackers: robust visual tracking by exploiting reliable patches. In: IEEE conference on computer vision and pattern recognition, CVPR 2015, Boston, MA, USA, June 7–12, 2015, pp 353–361

  25. Li Z, Gao S, Nai K (2017) Robust object tracking based on adaptive templates matching via the fusion of multiple features. J Vis Commun Image Represent 44:1–20. https://doi.org/10.1016/j.jvcir.2017.01.012

    Article  Google Scholar 

  26. Lukezic A, Vojir T, Zajc LC, Matas J, Kristan M (2017) Discriminative correlation filter with channel and spatial reliability. In: 2017 IEEE conference on computer vision and pattern recognition, CVPR 2017, Honolulu, HI, USA, July 21–26, 2017, pp 4847–4856

  27. Ma C, Yang X, Zhang C, Yang M (2015) Long-term correlation tracking. In: IEEE conference on computer vision and pattern recognition, CVPR 2015, Boston, MA, USA, June 7–12, 2015, pp 5388–5396

  28. Ma L, Lu J, Feng J, Zhou J (2015) Multiple feature fusion via weighted entropy for visual tracking. In: 2015 IEEE international conference on computer vision, ICCV 2015, Santiago, Chile, December 7–13, 2015, pp 3128–3136

  29. Nai K, Li Z, Li G, Wang S (2018) Robust object tracking via local sparse appearance model. IEEE Trans Image Process 27(10):4958–4970

    Article  MathSciNet  Google Scholar 

  30. Nai K, Xiao D, Li Z, Jiang S, Gu Y (2019) Multi-pattern correlation tracking. Knowl Based Syst. https://doi.org/10.1016/j.knosys.2019.05.032

    Article  Google Scholar 

  31. Ning J, Yang J, Jiang S, Zhang L, Yang M (2016) Object tracking via dual linear structured SVM and explicit feature map. In: 2016 IEEE conference on computer vision and pattern recognition, CVPR 2016, Las Vegas, NV, USA, June 27–30, 2016, pp 4266–4274

  32. Smeulders AWM, Chu DM, Cucchiara R, Calderara S, Dehghan A, Shah M (2014) Visual tracking: an experimental survey. IEEE Trans Pattern Anal Mach Intell 36(7):1442–1468. https://doi.org/10.1109/TPAMI.2013.230

    Article  Google Scholar 

  33. Sui Y, Zhang Z, Wang G, Tang Y, Zhang L (2016) Real-time visual tracking: Promoting the robustness of correlation filter learning. In: Computer vision-ECCV 2016—14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, proceedings, part VIII, pp 662–678

  34. Sun S, An Z, Jiang X, Zhang B, Zhang J (2019) Robust object tracking with the inverse relocation strategy. Neural Comput Appl 31:123–132

    Article  Google Scholar 

  35. Tang M, Feng J (2015) Multi-kernel correlation filter for visual tracking. In: 2015 IEEE international conference on computer vision, ICCV 2015, Santiago, Chile, December 7–13, 2015, pp 3038–3046

  36. Valmadre J, Bertinetto L, Henriques JF, Vedaldi A, Torr PHS (2017) End-to-end representation learning for correlation filter based tracking. In: 2017 IEEE conference on computer vision and pattern recognition, CVPR 2017, Honolulu, HI, USA, July 21–26, 2017, pp 5000–5008

  37. Wang N, Li S, Gupta A, Yeung D (2015) Transferring rich feature hierarchies for robust visual tracking. CoRR arXiv:1501.04587

  38. Wang X, Hou Z, Yu W, Pu L, Jin Z, Qin X (2018) Robust occlusion-aware part-based visual tracking with object scale adaptation. Pattern Recognit 81:456–470

    Article  Google Scholar 

  39. Wu Y, Lim J, Yang M (2013) Online object tracking: a benchmark. In: 2013 IEEE conference on computer vision and pattern recognition, Portland, OR, USA, June 23–28, 2013, pp 2411–2418. https://doi.org/10.1109/CVPR.2013.312

  40. Wu Y, Lim J, Yang M (2015) Object tracking benchmark. IEEE Trans Pattern Anal Mach Intell 37(9):1834–1848

    Article  Google Scholar 

  41. Xie G, Zeng G, Jiang J, Fan C, Li R, Li K (2017) Energy management for multiple real-time workflows on cyber-physical cloud systems. Future Gener Comput Syst. https://doi.org/10.1016/j.future.2017.05.033

    Article  Google Scholar 

  42. Xie K, Li X, Wang X, Xie G, Wen J, Cao J, Zhang D (2017) Fast tensor factorization for accurate internet anomaly detection. IEEE ACM Trans Netw 25(6):3794–3807

    Article  Google Scholar 

  43. Xie K, Li X, Wang X, Cao J, Xie G, Wen J, Zhang D, Qin Z (2018) On-line anomaly detection with high accuracy. IEEE ACM Trans Netw 26(3):1222–1235

    Article  Google Scholar 

  44. Xie K, Peng C, Wang X, Xie G, Wen J, Cao J, Zhang D, Qin Z (2018) Accurate recovery of internet traffic data under variable rate measurements. IEEE ACM Trans Netw 26(3):1137–1150

    Article  Google Scholar 

  45. Xu C, Tao D, Xu C (2013) A survey on multi-view learning. CoRR arXiv:1304.5634

  46. Yang B, Li Z, Jiang S, Li K (2018) Envy-free auction mechanism for VM pricing and allocation in clouds. Future Gener Comput Syst 86:680–693

    Article  Google Scholar 

  47. Yoon JH, Yang M, Yoon K (2016) Interacting multiview tracker. IEEE Trans Pattern Anal Mach Intell 38(5):903–917

    Article  Google Scholar 

  48. Zhang J, Ma S, Sclaroff S (2014) MEEM: robust tracking via multiple experts using entropy minimization. In: Computer vision-ECCV 2014—13th European conference, Zurich, Switzerland, September 6–12, 2014, proceedings, part VI, pp 188–203

  49. Zhang K, Liu Q, Wu Y, Yang M (2016) Robust visual tracking via convolutional networks without training. IEEE Trans Image Process 25(4):1779–1792. https://doi.org/10.1109/TIP.2016.2531283

    Article  MathSciNet  MATH  Google Scholar 

  50. Zhang L, Suganthan PN (2017) Robust visual tracking via co-trained kernelized correlation filters. Pattern Recognit 69:82–93

    Article  Google Scholar 

  51. Zhang S, Yu X, Sui Y, Zhao S, Zhang L (2015) Object tracking with multi-view support vector machines. IEEE Trans Multimed 17(3):265–278. https://doi.org/10.1109/TMM.2015.2390044

    Article  Google Scholar 

  52. Zhang T, Xu C, Yang M (2017) Multi-task correlation particle filter for robust object tracking. In: 2017 IEEE conference on computer vision and pattern recognition, CVPR 2017, Honolulu, HI, USA, July 21–26, 2017, pp 4819–4827

  53. Zhong W, Lu H, Yang M (2014) Robust object tracking via sparse collaborative appearance model. IEEE Trans Image Process 23(5):2356–2368

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

This work was supported by the National key R&D Program of China (Grant Nos. 2017YFB0202901, 2017YFB0202905). The corresponding author of this paper is Manman Peng (pengmanman@hnu.edu.cn).

Author information

Authors and Affiliations

  1. College of Information Science and Engineering, Hunan University, Changsha, 410082, Hunan, China

    Guiji Li, Manman Peng, Ke Nai, Zhiyong Li & Keqin Li

  2. Department of Computer Science, State University of New York, New Paltz, NY, 12561, USA

    Keqin Li

Authors
  1. Guiji Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Manman Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ke Nai
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhiyong Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Keqin Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Manman Peng.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, G., Peng, M., Nai, K. et al. Multi-view correlation tracking with adaptive memory-improved update model. Neural Comput & Applic 32, 9047–9063 (2020). https://doi.org/10.1007/s00521-019-04413-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-019-04413-4

Keywords

Search

Navigation