Skip to main content
SpringerLink
Log in

Efficient object tracking using hierarchical convolutional features model and correlation filters

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Visual object tracking is a very important task in computer vision. This paper develops a method based on the convolutional neural network (CNN) and correlation filters for visual object tracking. To implement a superior tracking method, we develop a multiple correlation tracker. This paper presents an effective method to track an object based on a combination of feature hierarchies of CNNs. We combine several feature hierarchies and compute the more discriminative map to track the object. Firstly, the correlation filters framework is selected to build the new tracker. Secondly, three feature maps from the CNN, which are inserted into the correlation filters framework, are adopted to evaluate the object location independently. Finally, a novel method of feature hierarchies integration based on Kullback–Leibler (KL) divergence is adopted. Experiments on the different sequences are carried out, and the outputs reveal that the proposed tracker has better results than those of the state-of-the-art methods, and it has the ability to handle various challenges.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Comaniciu, D., Ramesh, V., Meer, P.: Kernel-based object tracking. IEEE Trans. Pattern Anal. Mach. Int. 5, 564–575 (2003)

    Article  Google Scholar 

  2. Ross, D.A., Lim, J., Lin, R.S., Yang, M.H.: Incremental learning for robust visual tracking. Int. J. Comput. Vis. 77(1–3), 125–141 (2008)

    Article  Google Scholar 

  3. Tian, S., Shen, S., Tian, G., et al.: End-to-end deep metric network for visual tracking. Vis. Comput. (2019). https://doi.org/10.1007/s00371-019-01730-6

    Article  Google Scholar 

  4. Wang, D., Huchuan, L., Chen, Y.-W.: IncrementalMPCA for color object tracking. In: 2010 20th International Conference on Pattern Recognition, pp. 1751–1754 (2010)

  5. Hu, W., Li, X., Zhang, X., Shi, X., Maybank, S., Zhang, Z.: Incremental tensor subspace learning and its applications to foreground segmentation and tracking. Int. J. Comput. Vis. 91(3), 303–327542 (2011)

    Article  Google Scholar 

  6. Mbelwa, J.T., Zhao, Q., Wang, F.: Visual tracking tracker via object proposals and co-trained kernelized correlation filters. Vis. Comput. (2019). https://doi.org/10.1007/s00371-019-01727-1

    Article  Google Scholar 

  7. Avidan, S.: Ensemble tracking. IEEE Trans. Pattern Anal. Mach. Int. 29(2), 261–271 (2007)

    Article  Google Scholar 

  8. He, Z., Li, Q., Feng, H., et al.: Fast and sub-pixel precision target tracking algorithm for intelligent dual resolution camera. Vis. Comput. (2019). https://doi.org/10.1007/s00371-019-01724-4

    Article  Google Scholar 

  9. Grabner, H., Leistner, C., Bischof, H.: Semi-supervised on-line boosting for robust tracking. In: European Conference on Computer Vision, pp. 234–247 (2008)

  10. Vidanpathirana, M., Sudasingha, I., Vidanapathirana, J., et al.: Tracking and frame-rate enhancement for real-time 2Dhuman pose estimation. Vis Comput (2019). https://doi.org/10.1007/s00371-019-01757-9

    Article  Google Scholar 

  11. Tang, F., Brennan, S., Zhao, Q., Tao, H.: Co-tracking using semi-supervised support vector machines. In: 2007 IEEE 11th International Conference on Computer Vision, pp. 1–8 (2007)

  12. Li, Y., Zhu, J.: Ascale adaptive kernel correlation filter tracker with feature integration. In: European Conference on Computer Vision (2014)

  13. Xu, F., Zhao, L.: A particle filter tracking algorithm based on adaptive feature fusion strategy. In: Proceedings of the 10th World Congress on Intelligent Control and Automation, pp. 4612–4616. Beijing (2012)

  14. Jiang, H., Li, J., Wang, D., Lu, H.: Multi-feature tracking via adaptive weights. Neurocomputing 207, 189–201 (2016)

    Article  Google Scholar 

  15. Leang, I., Herbin, S., Girard, B., Droulez, J., Leang, I., Herbin, S., Girard, B., Droulez, J.: On-line fusion of trackers for single-object tracking. Pattern Recogn. 74, 459–473 (2017)

    Article  Google Scholar 

  16. He, Y.J., Li, M., Zhang, J.L., Yao, J.P.: Infrared target tracking via weighted correlation filter. Infrared Phys. Technol. 73, 103–114 (2015)

    Article  Google Scholar 

  17. Asha, C.S., Narasimhadhan, A.V.: Robust infrared target tracking using discriminative and generative approaches. Infrared Phys. Technol. 85, 114–127 (2017)

    Article  Google Scholar 

  18. Bolme, D.S., Beveridge, J.R., Draper, B.A., Lui, Y.M.: Visual object tracking using adaptive correlation filters. In: 2010 IEEE Computer Vision and Pattern Recognition, pp. 2544–2550. IEEE (2010)

  19. Henriques, J.F., Caseiro, R., Martins, P., Batista, J.J.: Exploiting the circulant structure of tracking-by detection with kernels. In: European Conference on Computer Vision, pp. 702–715 (2012)

  20. Danelljan, M., Häger, G., Khan, F., Felsberg, M.: Accurate scale estimation for robust visual tracking. In: British Machine Vision Conference, Nottingham, September 1–5, 2014. BMVA Press, London (2014)

  21. Wu, Y., Lim, J., Yang, M.H.: Object tracking benchmark. IEEE Trans. Pattern Anal. Mach. Intell. 37(9), 1834–1848 (2015)

    Article  Google Scholar 

  22. Henriques, J.F., Caseiro, R., Martins, P., Batista, J.: High-speed tracking with kernelized correlation filters. IEEE Trans. Pattern Anal. Mach. Intell. 37(3), 583–596 (2015)

    Article  Google Scholar 

  23. Danelljan, M., Khan, F.S., Felsberg, M., Weijer, J.V.D.: Adaptive color attributes for real-time visual tracking. In: IEEE Conference on Computer Vision and Pattern Recognition (2014)

  24. Li, X., Liu, Q., He, Z., Wang, H., Zhang, C., Chen, W.S.: A multi-view model for visual tracking via correlation filters. Knowl. Based Syst. 113(1), 88–99 (2016)

    Article  Google Scholar 

  25. Zhang, K., Lei, Z., Liu, Q., Zhang, D., Yang, M.H.: Fast visual tracking via dense spatio-temporal context learning. In: European Conference on Computer Vision (2014)

  26. Li, Y., Zhu, J., Hoi, S.C.: Reliable patch trackers: robust visual tracking by exploiting reliable patches. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 353–361 (2015)

  27. Chen, Z., Guo, Q., Wan, L., Feng, W.: Background-suppressed correlation filters for visual tracking. In: Proceedings of the IEEE International Conference on Multimedia Expo, pp. 1–6 (July 2018)

  28. Zhang, P., Guo, Q., Feng, W.: Fast spatially-regularized correlation filters for visual object tracking. In: Proceedings of the Pacific Rim International Conference on Artificial Intelligence, pp. 57–70 (2018)

  29. Zhang, P., Guo, Q., Feng, W.: Fast and object-adaptive spatial regularization for correlation filters based tracking. Neurocomputing 337, 129–143 (2019). https://doi.org/10.1016/j.neucom.2019.01.060

    Article  Google Scholar 

  30. Galoogahi, H.K., Sim, T., Lucey, S.: Correlation filters with limited boundaries. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4630–4638 (June 2015)

  31. Galoogahi, H.K., Fagg, A., Lucey, S.: Learning background-aware correlation filters for visual tracking. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1135–1143 (July 2017)

  32. Lukežič, A., Vojiř, T., Čehovin, L., Matas, J., Kristan, M.: Discriminative correlation filter with channel and spatial reliability. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6309–6318 (July 2017)

  33. Danelljan, M., Häger, G., Khan, F.S., Felsberg, M.: Learning spatially regularized correlation filters for visual tracking. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4310–4318 (Dec. 2015)

  34. Danelljan, M., Robinson, A., Khan, F.S., Felsberg, M.: Beyond correlation filters: learning continuous convolution operators for visual tracking. In: Proceedings of the European Conference on Computer Vision, pp. 472–488 (2016)

  35. Danelljan, M., Bhat, G., Khan, F.S., Felsberg, M.: Eco: efficient convolution operators for tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–3 641 (July 2017)

  36. Guo, Q., Feng, W., Zhou, C., Huang, R., Wan, L., Wang, S.: Learning dynamic siamese network for visual object tracking. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1781–1789 (Oct. 2017)

  37. Ma, C., Huang, J.-B., Yang, X., Yang, M.-H.: Hierarchical convolutional features for visual tracking. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3074–3082 (Dec. 2015)

  38. Danelljan, M., Hager, G., Khan, F.S., Felsberg, M.: Convolutional features for correlation filter based visual tracking. In: Proceedings of the ICCV Workshop, pp. 58–66 (Dec. 2015)

  39. Fan, J., Xu, W., Wu, Y., Gong, Y.: Human tracking using convolutional neural networks. IEEE Trans. Neural Netw. 21(10), 1610–1623 (2010)

    Article  Google Scholar 

  40. Bertinetto, L., Valmadre, J., Henriques, J.F., Vedaldi, A., Torr, P.H.S.: Fully-convolutional siamese networks for object tracking. In: Proceedings of the ECCV, pp. 850–865 (2016)

  41. Valmadre, J., Bertinetto, L., Henriques, J., Vedaldi, A., Torr, P.H.S.: End-to-end representation learning for correlation filter based tracking. In: Proceedings of the CVPR, pp. 2805–2813 (July 2017)

  42. Wang, Q., Gao, J., Xing, J., Zhang, M., Hu,W.: DCFNet: discriminant correlation filters network for visual tracking. arXiv: 1704.04057 (2017)

  43. Qi, Y. et al.: Hedged deep tracking. In: Proceedings of the CVPR, pp. 4303–4311 (June 2016)

  44. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv: 1409.1556 (2015)

  45. Ma, C., Huang, J.B., Yang, X., Yang, M.H.: Long-term correlation tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2015)

  46. Lukežič, A., Vojíř, T., Zajc, Č.: Discriminative correlation filter tracker with channel and spatial reliability. Int. J. Comput. Vis. (2018). https://doi.org/10.1007/s11263-017-1061-3

    Article  MathSciNet  Google Scholar 

  47. Danelljan, M., Häger, G., Khan, F.S., Felsberg, M.: Discriminative scale space tracking. IEEE Trans. Pattern Anal. Mach. Intell. 39(8), 1561–1575 (2017)

    Article  Google Scholar 

  48. Bertinetto, L., Valmadre, J., Golodetz, S., Miksik, O., Torr, P.H.S.: Staple: complementary learners for real-time tracking. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)

  49. Zhang, J., Ma, S., Sclaroff, S.: Robust tracking via multiple experts using entropy minimization. In: Proceedings of the European Conference on Computer Vision (2014)

  50. Ma, C., Huang, J.-B., Yang, X., Yang, M.-H.: Robust visual tracking via hierarchical convolutional features. IEEE Trans. Pattern Anal. Mach. Intell. 41, 2709–2723 (2018)

    Article  Google Scholar 

  51. Zhang, K., Zhang, L., Yang, M.-H.: Fast compressive tracking. IEEE Trans. Pattern Anal. Mach. Intell. 36(10), 2002–2015 (2014)

    Article  Google Scholar 

  52. Kalal, Z., Mikolajczyk, K., Matas, J.: Tracking-learning-detection. IEEE Trans. Pattern Anal. Mach. Intell. 34(7), 1409–1422 (2012)

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by ‘The Cross-Ministry Giga KOREA Project’ Grant funded by the Korea government (No. GK17C0200, Development of full-3D mobile display terminal and its contents).

Author information

Authors and Affiliations

  1. Engineering Department, Nuclear Research Center, Atomic Energy Authority, Cairo, Egypt

    Mohammed Y. Abbass & Safey A. Abdelwahab

  2. School of Information and Communication Engineering, Chungbuk National University, Cheongju, 28644, South Korea

    Ki-Chul Kwon & Nam Kim

  3. Department of Electronics and Electrical Communications, Faculty of Electronic Engineering, Menoufia University, Menouf, 32952, Egypt

    Fathi E. Abd El-Samie

  4. Electronics and Communications Department, Faculty of Engineering, Minia University, Minia, Egypt

    Ashraf A. M. Khalaf

Authors
  1. Mohammed Y. Abbass
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ki-Chul Kwon
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nam Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Safey A. Abdelwahab
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Fathi E. Abd El-Samie
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Ashraf A. M. Khalaf
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nam Kim.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Abbass, M.Y., Kwon, KC., Kim, N. et al. Efficient object tracking using hierarchical convolutional features model and correlation filters. Vis Comput 37, 831–842 (2021). https://doi.org/10.1007/s00371-020-01833-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-020-01833-5

Keywords

Search

Navigation