Skip to main content
SpringerLink
Log in

Long-term correlation tracking via spatial–temporal context

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

In this paper, we mainly deal with the problems of long-term visual tracking while the target objects undergo sophisticated scenarios such as occlusion, out-of-view, and scale changes. We employ two discriminative correlation filters (DCFs) for achieving long-term object tracking, which is performed by learning a spatial–temporal context correlation filter for translation estimation. As for the scale estimation, which is achieved by learning a scale DCF centered on the estimated target position to estimate scale from the best confident results. In addition, we proposed an efficient model update and redetecting activate strategy to avoid unrecoverable drift due to noisy updates, and achieve robust long-term tracking in the case of tracking failure. We evaluate our algorithm carry on OTB benchmark datasets, and the tracking results of both qualitative and quantitative evaluations on challenging sequences demonstrate that the proposed algorithm performs superiorly against several state-of-the-art DCFs methods including some methods which follow deep learning paradigm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Smeulders, A.W., Chu, D.M., Cucchiara, R., Calderara, S., Dehghan, A., Shah, M.: Visual tracking: an experimental survey. IEEE TPAMI 36(7), 1442–1468 (2014)

    Article  Google Scholar 

  2. Yilmaz, A., Javed, O., Shah, M.: Object tracking: a survey. ACM Comput. Surv. 38(4), 13 (2006)

    Article  Google Scholar 

  3. Comaniciu, D., Ramesh, V., Meer, P.: Kernel-based object tracking. Pattern Anal. Mach. Intell. 25(5), 564–575 (2003)

    Article  Google Scholar 

  4. Cauwenberghs, G., Poggio, T.: Incremental and decremental support vector machine learning. In: Proceedings of International Conference on Neural Information Processing Systems, pp. 388–394 (2000)

  5. Adam, A., Rivlin, E., Shimshoni, I.: Robust fragments-based tracking using the integral histogram. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 798–805 (2006)

  6. Mei, X., Ling, H.: Robust visual tracking using ℓ 1 minimization. In: Proceedings of IEEE International Conference on Computer Vision, pp. 1436–1443 (2009)

  7. Ahuja, N.: Robust visual tracking via multi-task sparse learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2042–2049 (2012)

  8. Zhang, T., Ghanem, B., Liu, S., et al.: Low-rank sparse learning for robust visual tracking. In: Proceedings of European Conference on Computer Vision, pp. 470–484 (2012)

    Chapter  Google Scholar 

  9. Fan, H., Xiang, J.: Robust visual tracking with multitask joint dictionary learning. IEEE Trans. Circuits Syst. Video Technol. 27(5), 1018–1030 (2017)

    Article  Google Scholar 

  10. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Proceedings of International Conference on Neural Information Processing Systems, pp. 1097–1105 (2012)

  11. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: Proceedings of International Conference on Learning Representations (2015)

  12. He, K., Zhang, X., Ren, S., et al.: Deep residual learning for image recognition, pp 770–778 (2015)

  13. Babenko, B., Yang, M.H., Belongie, S.: Robust object tracking with online multiple instance learning. IEEE Trans. Pattern Anal. Mach. Intell. 33(8), 1619 (2011)

    Article  Google Scholar 

  14. Avidan, S.: Ensemble tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 494–501 (2005)

  15. Zhang, K., Zhang, L., Yang, M.H.: Real-time compressive tracking. In: Proceedings of European Conference on Computer Vision, pp. 864–877 (2012)

    Chapter  Google Scholar 

  16. Avidan, S.: Support vector tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, vol. 1, pp. I-184–I-191 (2003)

  17. Kalal, Z., Mikolajczyk, K., Matas, J.: Tracking-learning-detection. IEEE Trans. Pattern Anal. Mach. Intell. 34(7), 1409–1422 (2012)

    Article  Google Scholar 

  18. Hare, S., Saffari, A., Torr, P.H.S.: Struck: structured output tracking with kernels. In: Proceedings of IEEE International Conference on Computer Vision, pp. 263–270 (2011)

  19. Grabner, H., Leistner, C., Bischof, H.: Semi-supervised on-line boosting for robust tracking. In: Proceedings of European Conference on Computer Vision, pp. 234–247 (2008)

    Google Scholar 

  20. Mueller, M., Smith, N., Ghanem, B.: Context-aware correlation filter tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 138–1395 (2017)

  21. Ozuysal, M., Fua, P., Lepetit, V.: Fast keypoint recognition in ten lines of code. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2007, pp. 1–8 (2007)

  22. Henriques, J.F., Rui, C., Martins, P., Batista, J.: High-speed tracking with kernelized correlation filters. IEEE Trans. Pattern Anal. Mach. Intell. 37(3), 583–596 (2014)

    Article  Google Scholar 

  23. Ma, C., Yang, X., Zhang, C., Yang, M.H.: Long-term correlation tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5388–5396 (2015)

  24. Danelljan, M., Häger, G., Khan, F.S., Felsberg, M.: Accurate scale estimation for robust visual tracking. In: Proceedings of British Machine Vision Conference, pp. 65.1–65.11 (2014)

  25. Bertinetto, L., Valmadre, J., Golodetz, S., Miksik, O., Torr, P.H.S.: Staple: complementary learners for real-time tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1401–1409 (2016)

  26. Ning, J., Yang, J., Jiang, S., et al.: Object tracking via dual linear structured SVM and explicit feature map. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4266–4274 (2016)

  27. Wu, Y., Lim, J., Yang, M.H.: Online object tracking: a benchmark. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2411–2418 (2013)

  28. Wu, Y., Lim, J., Yang, M.H.: Object tracking benchmark. IEEE Trans. Pattern Anal. Mach. Intell. 37(9), 1834–1848 (2015)

    Article  Google Scholar 

  29. Bolme, D.S., Beveridge, J.R., Draper, B.A., Liu, Y.M.: Visual object tracking using adaptive correlation filters. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2544–2550 (2010)

  30. Henriques, J.F., Caseiro, R., Martins, P., et al.: Exploiting the circulant structure of tracking-by-detection with Kernels. In: Computer Vision – ECCV 2012. Springer, Berlin, Heidelberg (2012)

    Google Scholar 

  31. Quan, W., Jiang, Y., Zhang, J., et al.: Robust object tracking with active context learning. Vis Comput 31, 1307 (2015). https://doi.org/10.1007/s00371-014-1012-8

    Article  Google Scholar 

  32. Hua, Y., Alahari, K., Schmid, C.: Occlusion and motion reasoning for long-term tracking. In: Computer Vision—ECCV 2014. Springer, pp 172–187 (2014)

  33. Li, Y., Zhu, J.: A scale adaptive kernel correlation filter tracker with feature integration. In: Proceedings of European Conference on Computer Vision, vol. 8926, pp. 254–265 (2014)

    Chapter  Google Scholar 

  34. Danelljan, M., Häger, G., Khan, F.S., Felsberg, M.: Learning spatially regularized correlation filters for visual tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4310–4318 (2015)

  35. Li, Z., He, S., Hashem, M.: Robust object tracking via multi-feature adaptive fusion based on stability: contrast analysis. Vis Comput 31, 1319 (2015). https://doi.org/10.1007/s00371-014-1014-6

    Article  Google Scholar 

  36. Wang, M., Liu, Y., Huang, Z.: Large margin object tracking with circulant feature maps. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4800–4808 (2017)

  37. Tang, M., Yu, B., Zhang, F., et al.: High-speed tracking with multi-kernel correlation filters. In: The IEEE Conference on Computer Vision and Pattern Recognition (2018)

  38. Wang, N., Zhou, W., Tian, Q., Hong, R., Wang, M., Li, H.: Multi-cue correlation filters for robust visual tracking. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018, pp. 4844–4853

  39. Zhang, K., Zhang, L., Liu, Q., Zhang, D., Yang, M.-H.: Fast visual tracking via dense spatio-temporal context learning. In: Proceedings of the European Conference on Computer Vision (2014)

  40. Danelljan, M., Hager, G., Khan, F.S., Felsberg, M.: Convolutional features for correlation filter based visual tracking. In: Proceedings of IEEE International Conference on Computer Vision Workshop, pp. 621–629 (2016)

  41. Valmadre, J., Bertinetto, L., Henriques, J.F., Vedaldi, A., Torr, P.H.S.: End-to-end representation learning for correlation filter based tracking. In: The IEEE Conference on Computer Vision and Pattern Recognition (2017)

  42. Song, Y., Ma, C., Gong, L., et al.: CREST: convolutional residual learning for visual tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2574–2583 (2017)

  43. Zhu, Z., Huang, G., Zou, W., et al.: UCT: learning unified convolutional networks for real-time visual tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1973–1982 (2017)

  44. Danelljan, M., Robinson, A., Khan, F.S., et al.: Beyond correlation filters: learning continuous convolution operators for visual tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 472–488 (2016)

    Chapter  Google Scholar 

  45. Bibi, A., Mueller, M., Ghanem, B.: Target response adaptation for correlation filter tracking. In: Proceedings of European Conference on Computer Vision, pp. 419–433 (2016)

    Chapter  Google Scholar 

  46. Hong, S., You, T., Kwak, S., et al.: Online tracking by learning discriminative saliency map with convolutional neural network. In: Proceedings of International Conference on Machine Learning, pp 597–606 (2015)

Download references

Funding

This work was supported by Promotion Program for Young and Middle-aged Teacher in Science and Technology Research of Huaqiao University (No.ZQN-PY518), and the grants from National Natural Science Foundation of China (Grant No.61605048), in part by Natural Science Foundation of Fujian Province, China under Grant 2015J01256, and Grant 2016J01300, in part by Fujian Provincial Big Data Research Institute of Intelligent Manufacturing, in part by the Quanzhou scientific and technological planning projects (No.2018C113R and No. 2017G024), and in part by the Subsidized Project for Postgraduates’ Innovative Fund in Scientific Research of Huaqiao University under Grant 1611422001.

Author information

Authors and Affiliations

  1. College of Engineering, Huaqiao University, Quanzhou, Fujian, China

    Zhi Chen, Peizhong Liu & Yongzhao Du

  2. College of Computer Science and Technology, Huaqiao University, Xiamen, Fujian, China

    Yanmin Luo

  3. Research Center of Applied Statistics and Big Data, Huaqiao University, No. 668, Jimei Avenue, Xiamen, 361021, China

    Peizhong Liu & Yongzhao Du

  4. Department of Electrical Engineering, National Taiwan University of Science and Technology, Taipei, Taiwan

    Jing-Ming Guo

Authors
  1. Zhi Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Peizhong Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yongzhao Du
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yanmin Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jing-Ming Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Peizhong Liu.

Ethics declarations

Conflict of interest

The authors declare no conflict of interest.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, Z., Liu, P., Du, Y. et al. Long-term correlation tracking via spatial–temporal context. Vis Comput 36, 425–442 (2020). https://doi.org/10.1007/s00371-019-01631-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-019-01631-8

Keywords

Search

Navigation