Skip to main content
SpringerLink
Log in

Robust kernelized correlation filter with scale adaption for real-time single object tracking

  • Special Issue Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

Kernelized correlation filter (KCF) is a kind of efficient method for real-time tracking, but remains being challenged by the drifting problem due to inaccurate localization caused by the scale variation and wrong candidate selection. In this paper, we propose a new scale adaptive kernelized correlation filter tracker, termed as SKCF, which estimates an accurate scale and models the distribution of correlation response to address the template drifting problem. In SKCF, a scale adaption method is used to find an accurate candidate. Thus we improve its capacity to drastic scale change which usually happens for unmanned aerial vehicles (UAVs)-based applications. The SKCF also introduces a Gaussian distribution to model the correlation response of the target image to select a better candidate in tracking procedure. Extensive experiments are performed on two commonly used tracking benchmarks and also a new benchmark for UAV tracking with complex scale variations. The results show that the proposed SKCF significantly improves the performance compared to the baseline KCF and achieves better performance than state-of-the-art single object trackers at real-time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Yilmaz, A., Javed, O., Shah, M.: Object tracking: a survey. ACM Comput. Surv. (CSUR) 38(4), 13 (2006)

    Article  Google Scholar 

  2. Yao, R., Shi, Q., Shen, C., Zhang, Y., Van Den Hengel, A.: Part-based visual tracking with online latent structural learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2363–2370 (2013)

  3. Adam, A., Rivlin, E., Shimshoni, I.: Robust fragments-based tracking using the integral histogram. In: IEEE computer society conference on computer vision and pattern recognition, pp. 798–805 (2006)

  4. Ross, D.A., Lim, J., Lin, R.S., Yang, M.H.: Incremental learning for robust visual tracking. Int. J. Comput. Vis. 77(1–3), 125–141 (2008)

    Article  Google Scholar 

  5. Nummiaro, K., Koller-Meier, E., Van Gool, L.: An adaptive color-based particle filter. Image Vis. Comput. 21(1), 99–110 (2003)

    Article  Google Scholar 

  6. Zhang, K., Zhang, L., Liu, Q., Zhang, D., Yang, M.H.: Fast Visual Tracking via Dense Spatio-temporal Context Learning, pp. 127–141. Springer, Cham (2014c)

    Google Scholar 

  7. Henriques, J.F., Caseiro, R., Martins, P., Batista, J.: Exploiting the circulant structure of tracking-by-detection with kernels. In: European conference on computer vision, pp. 702–715. Springer, Berlin (2012)

    Chapter  Google Scholar 

  8. Henriques, J.F., Caseiro, R., Martins, P., Batista, J.: High-speed tracking with kernelized correlation filters. IEEE Trans. Pattern Anal. Mach. Intell. 37(3), 583–596 (2015)

    Article  Google Scholar 

  9. Danelljan, M., Häger, G., Khan, F., Felsberg, M.: Accurate scale estimation for robust visual tracking. In: British machine vision conference, Nottingham, September 1–5, 2014, BMVA Press (2014)

  10. Ma, C., Yang, X., Zhang, C., Yang, M.H.: Long-term correlation tracking. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5388–5396 (2015)

  11. Su, X., Zhang, B., Yang, L., Li, Z., Yang, Y.: Scale invariant kernelized correlation filter based on gaussian output. In: International conference on cloud computing (ICCC), pp. 567–577 (2016)

    Chapter  Google Scholar 

  12. Wright, J., Yang, A.Y., Ganesh, A., Sastry, S.S., Ma, Y.: Robust face recognition via sparse representation. IEEE Trans. Pattern Anal. Mach. Intell. 31(2), 210–227 (2009)

    Article  Google Scholar 

  13. Ding, M., Fan, G.: Multi-layer joint gait-pose manifold for human motion modeling. IEEE Trans. Cybern. 45(11), 2413–2424 (2013a)

    Article  Google Scholar 

  14. Ding, M., Fan, G., Zhang, X., Ge, S., Chou, L.S.: Structure-guided manifold learning for video-based motion estimation. In: IEEE international conference on image processing, pp. 1977–1980 (2013b)

  15. Zhang, B., Perina, A., Li, Z., Murino, V., Liu, J., Ji, R.: Bounding multiple gaussians uncertainty with application to object tracking. Int. J. Comput. Vis. 118, 1–16 (2016)

    Article  MathSciNet  Google Scholar 

  16. Kalal, Z., Mikolajczyk, K., Matas, J.: Tracking-learning-detection. IEEE Trans. Pattern Anal. Mach. Intell. 34(7), 1409–1422 (2012)

    Article  Google Scholar 

  17. Cabanes, G., Bennani, Y.: Learning topological constraints in self-organizing map. In: International conference on neural information processing, pp. 367–374. Springer, Berlin (2010)

    Google Scholar 

  18. Zhang, B., Li, Z., Cao, X., Ye, Q., Chen, C., Shen, L., Perina, A., Ji, R.: Output constraint transfer for kernelized correlation filter in tracking. IEEE Trans. Syst. Man Cybern. Syst. 47(4), 693–703 (2017)

    Article  Google Scholar 

  19. Wu, Y., Lim, J., Yang, M.H.: Online object tracking: a benchmark. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2411–2418 (2013)

  20. Mueller, M., Smith, N., Ghanem, B.: A benchmark and simulator for uav tracking. In: European conference on computer vision, pp. 445–461 (2016)

    Chapter  Google Scholar 

  21. Wang, N., Shi, J., Yeung, D., Jia, J.: Understanding and diagnosing visual tracking systems. In: Proceedings of the IEEE international conference on computer vision, pp. 3101–3109 (2015)

  22. Wu, Y., Lim, J., Yang, M.H.: Object tracking benchmark. IEEE Trans. Pattern Anal. Mach. Intell. 37(9), 1834–1848 (2015)

    Article  Google Scholar 

  23. Comaniciu, D., Ramesh, V., Meer, P.: Kernel-based object tracking. IEEE Trans. Pattern Anal. Mach. Intell. 25(5), 564–577 (2003)

    Article  Google Scholar 

  24. Oron, S., Barhillel, A., Levi, D., Avidan, S.: Locally orderless tracking. Comput. In: Computer Vision and Pattern Recognition (CVPR), pp. 1940–1947 (2012)

  25. Dinh, T.B, Vo, N., Medioni, G.: Context tracker: exploring supporters and distracters in unconstrained environments. In: Computer Vision and Pattern Recognition (CVPR), pp. 1177–1184 (2011)

  26. Zhang, K., Zhang, L., Liu, Q., Zhang, D., Yang, M.: Fast visual tracking via dense spatio-temporal context learning. In: European conference on computer vision, pp. 127–141 (2014b)

    Google Scholar 

  27. Sevilla-Lara, L.: Distribution fields for tracking. In: Computer Vision and Pattern Recognition (CVPR), pp. 1910–1917 (2012)

  28. Held, D., Thrun, S., Savarese, S.: Learning to track at 100 fps with deep regression networks. In: European conference computer vision (ECCV) (2016)

    Chapter  Google Scholar 

  29. Nam, H., Han, B.: Learning multi-domain convolutional neural networks for visual tracking. In: The IEEE conference on computer vision and pattern recognition (CVPR) (2016)

  30. Hare, S., Saffari, A., Torr, P.H.S.: Struck: structured output tracking with kernels. In: IEEE international conference on computer vision, pp. 263–270 (2012)

  31. Grabner, H., Grabner, M., Bischof, H.: Real-time tracking via on-line boosting. Br. Mach. Vis. Conf. 1, 47–56 (2006)

    Google Scholar 

  32. Zhang, J., Ma, S., Sclaroff, S.: Meem: robust tracking via multiple experts using entropy minimization. In: European conference on computer vision, pp. 188–203 (2014a)

    Google Scholar 

  33. Bolme, D.S., Beveridge, J.R., Draper, B.A., Lui, Y.M.: Visual object tracking using adaptive correlation filters. In: Computer Vision and Pattern Recognition (CVPR), pp. 2544–2550 (2010)

  34. Ning, J., Yang, J., Jiang, S., Zhang, L., Yang, M.-H.: Object tracking via dual linear structured svm and explicit feature map. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4266–4274 (2016)

  35. Sui, Y., Zhang, Z., Wang, G., Tang, Y., Zhang, L.: Real-time visual tracking: promoting the robustness of correlation filter learning. In: European conference on computer vision, pp. 662–678. Springer, Berlin (2016)

    Chapter  Google Scholar 

  36. Danelljan, M., Hager, G., Khan, F.S., Felsberg, M.: Learning spatially regularized correlation filters for visual tracking. In: International conference on computer vision, pp. 4310–4318 (2015)

  37. Li, Y., Zhu, J.: A scale adaptive kernel correlation filter tracker with feature integration. In: European conference on computer vision, pp. 254–265 (2014)

    Google Scholar 

  38. Zhang, B., Gu, J., Chen, C., Han, J., Su, X., Cao, X., Liu, J.: One-two-one network for compression artifacts reduction in remote sensing. ISPRS J. Photogramm. Remote Sens. (2018b). https://doi.org/10.1016/j.isprsjprs.2018.01.003

    Article  Google Scholar 

  39. Yang, L., Li, C., Han, J., Chen, C., Ye, Q., Zhang, B.: Image reconstruction via manifold constrained convolutional sparse coding for image sets. J. Sel. Top. Signal Process. 11(7), 1072–1081 (2017b)

    Article  Google Scholar 

  40. Hou, R., Chen, C., Shah, M.: Tube Convolutional neural network (T-CNN) for action detection in videos. In: International conference on computer vision, pp. 5823–5832 (2017)

  41. Felzenszwalb, P.F., Girshick, R., Mcallester, D., Ramanan, D.: Object detection with discriminatively trained part-based models. IEEE Trans. Pattern Anal. Mach. Intell. 32(9), 1627–1645 (2010)

    Article  Google Scholar 

  42. Tuzel, O., Porikli, F., Meer, P.: Region covariance: a fast descriptor for detection and classification. In: European conference on computer vision, pp. 589–600 (2006)

    Chapter  Google Scholar 

  43. Zhang, B., Yang, Y., Chen, C., Yang, L., Han, J., Shao, L.: Action recognition using 3D histograms of texture and a multi-class boosting classifier. IEEE Trans. Image Process. 26(10), 4648–4660 (2017a)

    Article  MathSciNet  Google Scholar 

  44. Zhang, B., Luan, S., Chen, C., Han, J., Wang, W., Perina, A., Shao, L.: Latent constrained correlation filter. IEEE Trans. Image Process. 27(3), 1038–1048 (2018a)

    Article  MathSciNet  Google Scholar 

  45. Porikli, F., Tuzel, O., Meer, P.: Covariance tracking using model update based on lie algebra. Comput. Vis. Pattern Recognit. 1, 728–735 (2006)

    Google Scholar 

  46. Nussberger, A., Grabner, H., Van Gool, L.: Aerial object tracking from an airborne platform. In: International conference on unmanned aircraft systems, pp. 1284–1293 (2014)

  47. Qadir, A., Neubert, J., Semke, W., Schultz, R.: On-board visual tracking with unmanned aircraft system (uas). In: AIAA infotech at aerospace conference and exhibit (2011)

  48. Kendall, A.G., Salvapantula, N.N., Stol, K.A.: On-board object tracking control of a quadcopter with monocular vision. In: International conference on unmanned aircraft systems, pp. 404–411 (2014)

  49. Lim, H., Sinha, S.N.: Monocular localization of a moving person onboard a quadrotor mav. In: IEEE international conference on robotics and automation, pp. 2182–2189 (2015)

  50. Pestana, J., Sanchez-Lopez, J.L., Campoy, P., Saripalli, S.: Vision based gps-denied object tracking and following for unmanned aerial vehicles. In: IEEE international symposium on safety, security, and rescue robotics, pp. 1–6 (2014)

Download references

Acknowledgements

The authors would like to thank Dr. Jing Dai (China Academy of Launch Vehicle Technology R&D Center) for constructive suggestions on modifications on experimental parts. The work was supported by the Natural Science Foundation of China under Contract 61672079, 61473086, 61601466. This work is supported by the Open Projects Program of National Laboratory of Pattern Recognition and Shenzhen peacock plan.

Author information

Authors and Affiliations

  1. School of Automation Science and Electrical Engineering, Beihang University, Beijing, China

    Xingchao Liu, Xiangbo Su & Baochang Zhang

  2. Department of Computer Science, China University of Mining and Technology, Beijing, China

    Ce Li

Authors
  1. Ce Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xingchao Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiangbo Su
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Baochang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Baochang Zhang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, C., Liu, X., Su, X. et al. Robust kernelized correlation filter with scale adaption for real-time single object tracking. J Real-Time Image Proc 15, 583–596 (2018). https://doi.org/10.1007/s11554-018-0758-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-018-0758-z

Keywords

Search

Navigation