Skip to main content
SpringerLink
Log in

Real-time object tracking based on sparse representation and adaptive particle drawing

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Sparsity-based trackers describe a candidate region by solving the \({l}_{1}\) minimization problem. This process is done for many candidates generated in a particle filter framework. This results in a high computational cost, and thus it can preclude the use of these trackers in real-time applications. To tackle this issue, we proposed a novel method to draw particles from a Gaussian distribution which not only reduces the number of candidates but also maintains the accuracy of object tracking. In the proposed method, the appearance and motion information of object is used to adaptively estimate the number of particles in each frame. In addition, a normalized correlation (NC) filter is integrated to the sparsity-based tracker in which they cooperate with each other to increase the performance of object tracking. In other words, the \({l}_{1}\) minimization problem is controlled by a NC filter and the template region of correlation filter is set by a sparsity-based tracker. The occlusion degree over object tracking is accurately estimated by the sparse coefficients that its energy is adaptively controlled in the \({l}_{1}\) minimization problem. When an object is partially occluded, the templates of dictionary are adaptively updated based on occlusion degree which causes to increase the accuracy of object tracking. Experimental results on two tracking benchmark datasets demonstrate that the proposed method performs favorably against several popular trackers.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. We use the source code that is available at https://github.com/lukacu/visual-tracking-matlab/tree/master/ l1apg.

References

  1. Li, X., et al.: A survey of appearance models in visual object tracking. ACM Trans. Intell. Syst. Technol. (TIST) 4(4), 58 (2013)

    Google Scholar 

  2. Cannons, K.: A review of visual tracking. Dept. Comput. Sci. Eng., York Univ., Toronto, Canada, Tech. Rep. CSE-2008–07, 2008

  3. Wu, Y., Lim, J., M.-H. Yang.: 2013 IEEE Conference on Online object tracking: a benchmark. in Computer vision and pattern recognition (CVPR). 2013. IEEE

  4. Smeulders, A.W., et al.: Visual tracking: an experimental survey. IEEE Trans. Pattern Anal. Mach. Intell. 36(7), 1442–1468 (2014)

    Google Scholar 

  5. Pang, Y., Ling, H.: Finding the best from the second bests-inhibiting subjective bias in evaluation of visual tracking algorithms. in Proceedings of the IEEE International Conference on Computer Vision. 2013

  6. Liu, T., Wang, G., Yang, Q.: Real-time part-based visual tracking via adaptive correlation filters. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015

  7. Avidan, S.: Support vector tracking. IEEE Trans. Pattern Anal. Mach. Intell. 26(8), 1064–1072 (2004)

    Google Scholar 

  8. Babenko, B., Yang, M.-H., Belongie, S.: Robust object tracking with online multiple instance learning. IEEE Trans. Pattern Anal. Mach. Intell. 33(8), 1619–1632 (2010)

    Google Scholar 

  9. Jiang, N., Liu, W., Wu, Y.: Learning adaptive metric for robust visual tracking. IEEE Trans. Image Process. 20(8), 2288–2300 (2011)

    MathSciNet  MATH  Google Scholar 

  10. Gao, J., et al.: Transfer learning based visual tracking with Gaussian processes regression. In: European Conference on Computer Vision. 2014. Springer

  11. Wang, L., et al.: Visual tracking with fully convolutional networks. In: Proceedings of the IEEE International Conference on Computer Vision. 2015

  12. Adam, A., Rivlin, E., Shimshoni, I.: Robust fragments-based tracking using the integral histogram. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06). 2006. IEEE

  13. Kwon, J., Lee, K.M.: Visual tracking decomposition. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 2010. IEEE

  14. Wang, Q., Chen, F., Xu, W.: Tracking by third-order tensor representation. IEEE Trans. Syst. Man Cybern. Part B (Cybern.) 41(2), 385–396 (2010)

    Google Scholar 

  15. Kumar, B.S., Swamy, M., Ahmad, M.O.: Visual tracking via bilateral 2DPCA and robust coding. In: 2016 IEEE Canadian Conference on Electrical and Computer Engineering (CCECE). 2016. IEEE

  16. Kumar, B.S., Swamy, M., Ahmad, M.O.: Weighted residual minimization in PCA subspace for visual tracking. In: 2016 IEEE International Symposium on Circuits and Systems (ISCAS). 2016. IEEE.

  17. Wang, D.L.: Object tracking via 2DPCA and L1-regularization. Signal Process. Lett. 19(11), 711 (2012)

    Google Scholar 

  18. Mei, X., Ling, H.: Robust visual tracking using ℓ1 minimization. In: 2009 IEEE 12th International Conference on Computer Vision. 2009. IEEE.

  19. Zhang, T., Bibi, A., Ghanem, B.: In defense of sparse tracking: circulant sparse tracker. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016.

  20. Zeng, X., et al.: Dual-scale weighted structural local sparse appearance model for object tracking. IET Comput. Vis. 13(2), 146–156 (2018)

    Google Scholar 

  21. Li, Z., et al.: Visual tracking with weighted adaptive local sparse appearance model via spatio-temporal context learning. IEEE Trans. Image Process. 27(9), 4478–4489 (2018)

    MathSciNet  MATH  Google Scholar 

  22. Nai, K., et al.: Robust object tracking via local sparse appearance model. IEEE Trans. Image Process. 27(10), 4958–4970 (2018)

    MathSciNet  MATH  Google Scholar 

  23. Zhang, T., Xu, C., Yang, M.-H.: Robust structural sparse tracking. IEEE Trans. Pattern Anal. Mach. Intell. 41(2), 473–486 (2018)

    Google Scholar 

  24. Ma, B., et al.: Visual tracking using strong classifier and structural local sparse descriptors. IEEE Trans. Multimed. 17(10), 1818–1828 (2015)

    Google Scholar 

  25. Liu, N., Huo, H., Fang, T.: Robust object tracking via constrained online dictionary learning. Multimed. Tools Appl. 78(3), 3689–3703 (2019)

    Google Scholar 

  26. Jansi, R., et al.: Robust object tracking using sparse based representative dictionary learning. In: 2017 International Conference on Computation of Power, Energy Information and Communication (ICCPEIC). 2017. IEEE.

  27. Ma, B., et al.: Robust object tracking by nonlinear learning. IEEE Trans. Neural Netw. Learn. Syst. 29(10), 4769–4781 (2017)

    MathSciNet  Google Scholar 

  28. Dong, X., Shen, J.: Triplet loss in siamese network for object tracking. In: Proceedings of the European Conference on Computer Vision (ECCV). 2018

  29. Dong, X., et al.: Quadruplet network with one-shot learning for fast visual object tracking. IEEE Trans. Image Process. 28(7), 3516–3527 (2019)

    MathSciNet  MATH  Google Scholar 

  30. Liang, Z., Shen, J.: Local semantic siamese networks for fast tracking. IEEE Trans. Image Process. 29, 3351–3364 (2019)

    Google Scholar 

  31. Shen, J.: Tang X, Dong X, Shao L (2019) Visual object tracking by hierarchical attention siamese network. IEEE Trans. Cybern. 50(7), 3068–3080 (2019)

    Google Scholar 

  32. Hu, H., Ma, B., Shen, J., Hanqiu, S., Shao, L., Porikli, F., et al.: Robust tracking using manifold convolutional neural networks with Laplacian regularization. IEEE. Trans. Multi 21(2), 510–521 (2018)

    Google Scholar 

  33. Dong, X., et al.: Hyperparameter optimization for tracking with continuous deep q-learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2018)

  34. Dong, X., Shen, J., Wang, W., et al.: Dynamical hyperparameter optimization via deep reinforcement learning in tracking. IEEE. Trans. Pattern Anal. Mach. Intell. (2019). https://doi.org/10.1109/tpami.2019.2956703

    Article  Google Scholar 

  35. Hu, H., et al.: Manifold regularized correlation object tracking. IEEE Trans. Neural Netw. Learn. Syst. 29(5), 1786–1795 (2017)

    MathSciNet  Google Scholar 

  36. Shen, J., et al.: Multiobject tracking by submodular optimization. IEEE Trans. Cybern. 49(6), 1990–2001 (2018)

    Google Scholar 

  37. Zhong, W., Lu, H., Yang, M.-H.: Robust object tracking via sparse collaborative appearance model. IEEE Trans. Image Process. 23(5), 2356–2368 (2014)

    MathSciNet  MATH  Google Scholar 

  38. Liu, H., Li, S., Fang, L.: Robust object tracking based on principal component analysis and local sparse representation. IEEE Trans. Instrum. Meas. 64(11), 2863–2875 (2015)

    Google Scholar 

  39. Shan, C., Tan, T., Wei, Y.: Real-time hand tracking using a mean shift embedded particle filter. Pattern Recognit. 40(7), 1958–1970 (2007)

    MATH  Google Scholar 

  40. Mei, X., Ling, H.: Robust visual tracking and vehicle classification via sparse representation. IEEE Trans. Pattern Anal. Mach. Intell. 33(11), 2259–2272 (2011)

    Google Scholar 

  41. He, L., et al.: Robust object tracking based on motion consistency. Sensors 18(2), 572 (2018)

    Google Scholar 

  42. Mei, X., et al.: Minimum error bounded efficient ℓ1 tracker with occlusion detection. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2011. IEEE.

  43. Kumar, N., Parate, P.: Fragment-based real-time object tracking: a sparse representation approach. In: 2012 19th IEEE International Conference on Image Processing. 2012. IEEE

  44. Zhang, T., Xu, C., Yang, M.-H.: Multi-task correlation particle filter for robust object tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2017)

  45. Bao, C., et al.: Real time robust l1 tracker using accelerated proximal gradient approach. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2012. IEEE

  46. Zhuang, B., et al.: Visual tracking via discriminative sparse similarity map. IEEE Trans. Image Process. 23(4), 1872–1881 (2014)

    MathSciNet  MATH  Google Scholar 

  47. Li, H., Shen, C., Shi, Q.: Real-time visual tracking using compressive sensing. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2011. IEEE

  48. An, X., Sun, N., Cao, M.: Structural regression model based inverse sparse representation for tracking objects. IEEE Access 6, 69978–69987 (2018)

    Google Scholar 

  49. Wang, D., et al.: Inverse sparse tracker with a locally weighted distance metric. IEEE Trans. Image Process. 24(9), 2646–2657 (2015)

    MathSciNet  MATH  Google Scholar 

  50. Smith, A.: Sequential Monte Carlo Methods in Practice. Springer, Berlin (2013)

    Google Scholar 

  51. Wong, S.: Advanced correlation tracking of objects in cluttered imagery. In: Acquisition, Tracking, and Pointing XIX. 2005. International Society for Optics and Photonics

  52. Henriques, J.F., et al.: High-speed tracking with kernelized correlation filters. IEEE Trans. Pattern Anal. Mach. Intell. 37(3), 583–596 (2015)

    Google Scholar 

  53. Ahmed, J., et al.: Real-time edge-enhanced dynamic correlation and predictive open-loop car-following control for robust tracking. Mach. Vis. Appl. 19(1), 1–25 (2008)

    Google Scholar 

  54. Mbelwa, J.T., Zhao, Q., Wang, F.: Visual tracking tracker via object proposals and co-trained kernelized correlation filters. Vis. Comput. 36, 1173–1187 (2020)

    Google Scholar 

  55. Zhang, H., Liu, G.: Coupled-layer based visual tracking via adaptive kernelized correlation filters. Vis. Comput. 34(1), 41–54 (2018)

    MathSciNet  Google Scholar 

  56. Hadi, I., Sabah, M.: Behavior formula extraction for object trajectory using curve fitting method. Int. J. Comput. Appl. 104(2), 21–27 (2014)

    Google Scholar 

  57. Bai, T., Li, Y.: Robust visual tracking using flexible structured sparse representation. IEEE Trans. Industr. Inf. 10(1), 538–547 (2013)

    Google Scholar 

  58. Bai, T., Li, Y.F.: Robust visual tracking with structured sparse representation appearance model. Pattern Recognit. 45(6), 2390–2404 (2012)

    MATH  Google Scholar 

  59. Ross, D.A., et al.: Incremental learning for robust visual tracking. Int. J. Comput. Vis. 77(1–3), 125–141 (2008)

    Google Scholar 

  60. Jia, X., Lu, H., Yang, M.-H.: Visual tracking via adaptive structural local sparse appearance model. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE (2012)

  61. Hu, Q., et al.: Robust and real-time object tracking using scale-adaptive correlation filters. In: 2016 International Conference on Digital Image Computing: Techniques and Applications (DICTA). IEEE (2016)

  62. Danelljan, M., et al.: Accurate scale estimation for robust visual tracking. In: British Machine Vision Conference, Nottingham, September 1–5, 2014. 2014. BMVA Press

  63. Zolfaghari, M., Ghanei-Yakhdan, H., Yazdi, M.: Real-time object tracking based on an adaptive transition model and extended Kalman filter to handle full occlusion. Vis. Comput. 36, 701–715 (2020)

    Google Scholar 

  64. Wu, Y., Lim, J., Yang, M.-H.: Object tracking benchmark. IEEE Trans. Pattern Anal. Mach. Intell. 37(9), 1834–1848 (2015)

    Google Scholar 

  65. Kristan, M., et al.: The visual object tracking vot2015 challenge results. In: Proceedings of the IEEE International Conference on Computer Vision Workshops (2015)

  66. Kristan, M., et al.: The seventh visual object tracking vot2019 challenge results. In: Proceedings of the IEEE International Conference on Computer Vision Workshops (2019)

  67. Hare, S., et al.: Struck: structured output tracking with kernels. IEEE Trans. Pattern Anal. Mach. Intell. 38(10), 2096–2109 (2015)

    MathSciNet  Google Scholar 

  68. Zhang, K., et al.: Robust visual tracking via convolutional networks without training. IEEE Trans. Image Process. 25(4), 1779–1792 (2016)

    MathSciNet  MATH  Google Scholar 

  69. Kalal, Z., Mikolajczyk, K., Matas, J.: Tracking-learning-detection. IEEE Trans. Pattern Anal. Mach. Intell. 34(7), 1409–1422 (2011)

    Google Scholar 

  70. Van De Weijer, J., et al.: Learning color names for real-world applications. IEEE Trans. Image Process. 18(7), 1512–1523 (2009)

    MathSciNet  MATH  Google Scholar 

  71. Felzenszwalb, P.F., et al.: Object detection with discriminatively trained part-based models. IEEE Trans. Pattern Anal. Mach. Intell. 32(9), 1627–1645 (2010)

    Google Scholar 

  72. Fazl-Ersi, E., Nooghabi, M.K. Revisiting correlation-based filters for low-resolution and long-term visual tracking. Vis. Comput., 1–13 (2018)

  73. Ma, C., et al.: Hierarchical convolutional features for visual tracking. In: Proceedings of the IEEE International Conference on Computer Vision (2015)

  74. Dong, X., et al.: Occlusion-aware real-time object tracking. IEEE Trans. Multimed. 19(4), 763–771 (2016)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, Yazd University, Yazd, Iran

    Mohammad Zolfaghari & Hossein Ghanei-Yakhdan

  2. Department of Electronics and Telecommunication Engineering, Shiraz University, Shiraz, Iran

    Mehran Yazdi

Authors
  1. Mohammad Zolfaghari
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hossein Ghanei-Yakhdan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mehran Yazdi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hossein Ghanei-Yakhdan.

Ethics declarations

Conflict of interest

All authors declare that they have no conflict of interest.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zolfaghari, M., Ghanei-Yakhdan, H. & Yazdi, M. Real-time object tracking based on sparse representation and adaptive particle drawing. Vis Comput 38, 849–869 (2022). https://doi.org/10.1007/s00371-020-02055-5

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-020-02055-5

Keywords

Search

Navigation