Skip to main content

Advertisement

Springer Nature Link
Log in

Structural local sparse and low-rank tracker using deep features

  • Regular Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

Visual trackers based on the sparse representation and low-rank constraints imposed on candidate targets have achieved commendable results. However, combining low-rank constraint imposed on local patches with local sparsity has not been explored for designing visual tracker yet. In this work, we propose a novel deep features-based structural local sparse low-rank tracker, which not only exploits sparsity and low-rank constraint about local patches, but also takes the spatial structure of the target regions into account. Specifically, according to the magnitude of change of targets’ appearance, all local patches are divided into stable and unstable ones since their contribution to tracking may be sharply different. Then, elegant pre-locating and pruning schemes are developed to maintain the proposed tracker’s performance even in the very challenging scenarios and meanwhile decrease the heavy computational burden caused by large number of candidate targets. Moreover, an upgraded effective template update scheme is designed by utilizing long-term and short-term memory of different templates to adapt to changes of the target’s appearance. Comprehensive experiments on OTB100, TC128, VOT2016, VOT2018 and GOT-10k benchmark datasets demonstrate that the proposed method outperforms several popular handcrafted features-based trackers and is comparable to state-of-the-art deep trackers in various tracking environments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Mei, X., Ling, H.: Robust visual tracking using \(\ell 1\) minimization. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 1436–1443 (2009)

  2. Wang, D., Lu, H., Yang, M.-H.: Online object tracking with sparse prototypes. IEEE Trans. Image Process. 22(1), 314–325 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  3. Zhuang, B., Lu, H., Xiao, Z., Wang, D.: Visual tracking via discriminative sparse similarity map. IEEE Trans. Image Process. 23(4), 1872–1881 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  4. Zhang, T., Bibi, A., Ghanem, B.: In defense of sparse tracking: circulant sparse tracker. In: Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3880–3888 (2016)

  5. Zhang, T., Liu, S., Ahuja, N., Yang, M.-H., Ghanem, B.: Robust visual tracking via consistent low-rank sparse learning. Int. J. Comput. Vis. 111(2), 171–190 (2015)

    Article  MATH  Google Scholar 

  6. Yang, Y., Hu, W., Xie, Y., Zhang, W., Zhang, T.: Temporal restricted visual tracking via reverse-low-rank sparse learning. IEEE Trans. Cybern. 47(2), 485–498 (2017)

    Google Scholar 

  7. Jia, X., Lu, H., Yang, M.-H.: Visual tracking via adaptive structural local sparse appearance model. In: Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1822–1829 (2012)

  8. Liu, B., Huang, J., Kulikowski, C., Yang, L.: Robust visual tracking using local sparse appearance model and k-selection. IEEE Trans. Pattern Anal. Mach. Intell. 35(12), 2968–2981 (2013)

    Article  Google Scholar 

  9. Zhong, W., Lu, H., Yang, M.-H.: Robust object tracking via sparse collaborative appearance model. IEEE Trans. Image Process. 23(5), 2356–2368 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  10. Zhang, T., Liu, S., Xu, C., Yan, S., Ghanem, B., Ahuja, N., Yang, M.-H.: Structural sparse tracking. In: Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), pp. 150–158 (2015)

  11. Nai, K., Li, Z., Li, G., Wang, S.: Robust object tracking via local sparse appearance model. IEEE Trans. Image Process. 27(10), 4958–4970 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  12. Qi, Y., Qin, L., Zhang, J., Zhang, S., Huang, Q., Yang, M.-H.: Structure-aware local sparse coding for visual tracking. IEEE Trans. Image Process. 27(8), 3857–3869 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  13. Zhou, T., Liu, F., Bhaskar, H., Yang, J.: Robust visual tracking via online discriminative and low-rank dictionary learning. IEEE Trans. Cybern. 48(9), 2643–2655 (2018)

    Article  Google Scholar 

  14. Zhang, T., Xu, C., Yang, M.-H.: Robust structural sparse tracking. IEEE Trans. Pattern Anal. Mach. Intell. 41(2), 473–486 (2019)

    Article  Google Scholar 

  15. Fiaz, M., Mahmood, A., Javed, S., Jung, S.K.: Handcrafted and deep trackers: recent visual object tracking approaches and trends. ACM Comput. Surv. 52(2), 1–44 (2019)

    Article  Google Scholar 

  16. Han, R., Feng, W., Guo, Q., Hu: Single object tracking research: a survey. arXiv:2204.11410 (2022)

  17. Marvasti-Zadeh, S.M., Cheng, L., Ghanei-Yakhdan, H., Kasaei, S.: Deep learning for visual tracking: a comprehensive survey. IEEE Trans. Intell. Transp. Syst. 23(5), 3943–3968 (2022)

    Article  Google Scholar 

  18. Zolfaghari, M., Ghanei-Yakhdan, H., Yazdi, M.: Real-time object tracking based on sparse representation and adaptive particle drawing. Vis. Comput. 38(3), 849–869 (2022)

    Article  Google Scholar 

  19. Jia, X., Lu, H., Yang, M.-H.: Visual tracking via coarse and fine structural local sparse appearance models. IEEE Trans. Image Process. 25(10), 4555–4564 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  20. Zhong, W., Lu, H., Yang, M.-H.: Robust object tracking via sparsity-based collaborative model. In: Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1838–1845 (2012)

  21. Zhang, J., Feng, W., Yuan, T., Wang, J., Sangaiah, A.K.: Scstcf: spatial-channel selection and temporal regularized correlation filters for visual tracking. Appl. Soft Comput. 118, 108485 (2022)

    Article  Google Scholar 

  22. Zhang, J., Liu, Y., Liu, H., Wang, J., Zhang, Y.: Distractor-aware visual tracking using hierarchical correlation filters adaptive selection. Appl. Intell. 52(6), 6129–6147 (2022)

    Article  Google Scholar 

  23. Nai, K., Li, Z., Gan, Y., Wang, Q.: Robust visual tracking via multitask sparse correlation filters learning. IEEE Trans. Neural Netw. Learn. Syst. 1–14 (2021)

  24. Xu, T., Feng, Z.-H., Wu, X.-J., Kittler, J.: Learning low-rank and sparse discriminative correlation filters for coarse-to-fine visual object tracking. IEEE Trans. Circuits Syst. Video Technol. 30(10), 3727–3739 (2020)

    Article  Google Scholar 

  25. Yu, M., Wang, C., Zhang, Y., Lin, Z.: Rotation-aware dynamic temporal consistency with spatial sparsity correlation tracking. Image Vis. Comput. 104546 (2022)

  26. Ma, C., Huang, J.-B., Yang, X., Yang, M.-H.: Hierarchical convolutional features for visual tracking. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 3074–3082 (2015)

  27. Danelljan, M., Bhat, G., Khan, F.S., Felsberg, M.: ECO: efficient convolution operators for tracking. In: Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6931–6939 (2017)

  28. Bertinetto, L., Valmadre, J., Henriques, J.F., Vedaldi, A., Torr, P.H.: Fully-convolutional siamese networks for object tracking. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 850–865 (2016)

  29. Wang, Q., Zhang, L., Bertinetto, L., Hu, W., Torr, P.H.: Fast online object tracking and segmentation: a unifying approach. In: Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1328–1338 (2019)

  30. Guo, D., Shao, Y., Cui, Y., Wang, Z., Zhang, L., Shen, C.: Graph attention tracking. In: Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9538–9547 (2021)

  31. Danelljan, M., Bhat, G., Khan, F.S., Felsberg, M.: ATOM: accurate tracking by overlap maximization. In: Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4660–4669 (2019)

  32. Zhou, Z., Li, X., Fan, N., Wang, H., He, Z.: Target-aware state estimation for visual tracking. IEEE Trans. Circuits Syst. Video Technol. 32(5), 2908–2920 (2022)

    Article  Google Scholar 

  33. Wang, N., Zhou, W., Wang, J., Li, H.: Transformer meets tracker: exploiting temporal context for robust visual tracking. In: Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1571–1580 (2021)

  34. Mayer, C., Danelljan, M., Bhat, G., Paul, M., Paudel, D.P., Yu, F., Van Gool, L.: Transforming model prediction for tracking. In: Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8731–8740 (2022)

  35. Song, Z., Yu, J., Chen, Y.-P.P., Yang, W.: Transformer tracking with cyclic shifting window attention. In: Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8791–8800 (2022)

  36. Javed, S., Mahmood, A., Dias, J., Werghi, N.: Robust structural low-rank tracking. IEEE Trans. Image Process. 29, 4390–4405 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  37. Zhang, T., Ghanem, B., Liu, S., Ahuja, N.: Low-rank sparse learning for robust visual tracking. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 470–484 (2012)

  38. Li, Y., Zhu, J., Hoi, S.C.: Reliable patch trackers: Robust visual tracking by exploiting reliable patches, In: Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), pp. 353–361 (2015)

  39. Li, G., Peng, M., Nai, K., Li, Z., Li, K.: Visual tracking via context-aware local sparse appearance model. J. Vis. Commun. Image Represent. 56, 92–105 (2018)

    Article  Google Scholar 

  40. Lin, Z., Chen, M., Ma, Y.: The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices. arXiv:1009.5055, (2010)

  41. Wu, Y., Lim, J., Yang, M.-H.: Object tracking benchmark. IEEE Trans. Pattern Anal. Mach. Intell. 37(9), 1834–1848 (2015)

    Article  Google Scholar 

  42. Liang, P., Blasch, E., Ling, H.: Encoding color information for visual tracking: algorithms and benchmark. IEEE Trans. Image Process. 24(12), 5630–5644 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  43. Kristan, M., Leonardis, A., Matas, J., Felsberg, M.: The visual object tracking vot2016 challenge results. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 777–823 (2016)

  44. Kristan, M., Leonardis, A., Matas, J., Felsberg, M., Pflugfelder, R., Cehovin Zajc, L., Vojir, T., Bhat, G., Lukezic, A., Eldesokey, A., et al.: The sixth visual object tracking vot2018 challenge results. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 3–53 (2018)

  45. Huang, L., Zhao, X., Huang, K.: GOT-10k: a large high-diversity benchmark for generic object tracking in the wild. IEEE Trans. Pattern Anal. Mach. Intell. 43(5), 1562–1577 (2019)

    Article  Google Scholar 

  46. Henriques, J.F., Caseiro, R., Martins, P., Batista, J.: High-speed tracking with kernelized correlation filters. IEEE Trans. Pattern Anal. Mach. Intell. 37(3), 583–596 (2015)

    Article  Google Scholar 

  47. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: International Conference on Learning Representations, pp. 1–14 (2015)

  48. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), vol. 1, pp. 886–893 (2005)

  49. Fan, H., Ling, H.: Parallel tracking and verifying: a framework for real-time and high accuracy visual tracking. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 5487–5495 (2017)

  50. Danelljan, M., Hager, G., Shahbaz Khan, F., Felsberg, M.: Convolutional features for correlation filter based visual tracking. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 58–66 (2015)

  51. Galoogahi, H.K., Fagg, A., Lucey, S.: Learning background-aware correlation filters for visual tracking. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 1144–1152 (2017)

  52. Valmadre, J., Bertinetto, L., Henriques, J., Vedaldi, A., Torr, P.H.S.: End-to-end representation learning for correlation filter based tracking. In: Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5000–5008 (2017)

  53. Wang, M., Liu, Y., Huang, Z.: Large margin object tracking with circulant feature maps. In: Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4021–4029 (2017)

  54. Wang, N., Yeung, D.-Y.: Learning a deep compact image representation for visual tracking. Adv. Neural Inf. Process. Syst. 809–817 (2013)

  55. Li, P., Chen, B., Ouyang, W., Wang, D., Yang, X., Lu, H.: Gradnet: gradient-guided network for visual object tracking. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 6161–6170 (2019)

  56. Zhu, Z., Wang, Q., Li, B., Wu, W., Yan, J., Hu, W.: Distractor-aware siamese networks for visual object tracking. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 101–117 (2018)

  57. Wang, N., Song, Y., Ma, C., Zhou, W., Liu, W., Li, H.: Unsupervised deep tracking. In: Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1308–1317 (2019)

  58. Li, X., Ma, C., Wu, B., He, Z., Yang, M.-H.: Target-aware deep tracking. In: Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1369–1378 (2019)

  59. Zheng, J., Ma, C., Peng, H., Yang, X.: Learning to track objects from unlabeled videos. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 13546–13555 (2021)

  60. Bhat, G., Danelljan, M., Gool, L.V., Timofte, R.: Learning discriminative model prediction for tracking. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 6182–6191 (2019)

  61. Shen, Q., Qiao, L., Guo, J., Li, P., Li, X., Li, B., Feng, W., Gan, W., Wu, W., Ouyang, W.: Unsupervised learning of accurate siamese tracking. In: Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8101–8110 (2022)

  62. Danelljan, M., Robinson, A., Khan, F.S., Felsberg, M.: Beyond correlation filters: Learning continuous convolution operators for visual tracking. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 472–488 (2016)

  63. Yang, T., Xu, P., Hu, R., Chai, H., Chan, A.B.: ROAM: recurrently optimizing tracking model. In: Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6718–6727 (2020)

  64. Ma, F., Shou, M.Z., Zhu, L., Fan, H., Xu, Y., Yang, Y., Yan, Z.: Unified transformer tracker for object tracking. In: Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8781–8790 (2022)

Download references

Acknowledgements

This study was funded by the National Natural Science Foundation of China under the Grants no. 11501351. Thank all the referees and the editorial board members for their insightful comments and suggestions, which improved our paper significantly.

Author information

Authors and Affiliations

  1. Department of Mathematics, Shanghai University, Shanghai, 200444, People’s Republic of China

    Pengqing Li, Hongjuan Zhang & Yansong Chen

Authors
  1. Pengqing Li
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Hongjuan Zhang
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Yansong Chen
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

PL: conceptualization, methodology, software. HZ: supervision, writing—reviewing. YC: writing-reviewing.

Corresponding author

Correspondence to Hongjuan Zhang.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, P., Zhang, H. & Chen, Y. Structural local sparse and low-rank tracker using deep features. Multimedia Systems 29, 1481–1498 (2023). https://doi.org/10.1007/s00530-023-01064-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-023-01064-3

Keywords