Skip to main content
SpringerLink
Log in

Real-time object tracking based on an adaptive transition model and extended Kalman filter to handle full occlusion

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

In this paper, a tracker scheme is proposed that not only can face object tracking challenges but also can estimate object positions over occluded frames. In the proposed scheme, kernelized correlation filter (KCF) is considered as our basic tracker due to its high efficiency in the most object tracking challenges except occlusion and illumination variation. To improve the efficiency of KCF, the proposed method integrates an occlusion detection method, an adaptive model update and a prediction system into the KCF tracker. The occlusion detection method is based on the peak-to-sidelobe ratio of the confidence map to determine the type of occlusion. When an object is partially occluded, the object appearance model is adaptively updated to increase the accuracy of object tracking. When full occlusion occurs, the proposed predictor is run and exploits the available motion information before the occurrence of full occlusion to predict the location of the tracked object. The proposed predictor uses adaptive transition state equations to estimate the acceleration and velocity of the object needed in the extended Kalman filter (EKF) to predict object position. It also uses two quadratic equations to estimate the object trajectory. Finally, a method is proposed that exploits the estimated object positions by both EKF and the object trajectory to predict object positions over fully occluded frames. Experimental results on open datasets show that the proposed method achieved a better performance in comparison with several state-of-the-art trackers.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. The sequences of this dataset are available at https://pws.yazd.ac.ir/ghanei/DIVPL/Datasets.htm.

References

  1. Adhikari, G., Sahani, S.K., Chauhan, M.S., Das, B.K.: Fast real time object tracking based on normalized cross correlation and importance of thresholding segmentation. In: 2016 International Conference on Recent Trends in Information Technology (ICRTIT), pp. 1–5 (2016)

  2. Yilmaz, A., Javed, O., Shah, M.: Object tracking: a survey. ACM Comput. Surv (CSUR) 38, 13 (2006)

    Article  Google Scholar 

  3. Liu, T., Wang, G., Yang, Q.: Real-time part-based visual tracking via adaptive correlation filters. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4902–4912 (2015)

  4. Liu, Q., Zhao, X., Hou, Z.: Survey of single-target visual tracking methods based on online learning. IET Comput. Vision 8, 419–428 (2014)

    Article  Google Scholar 

  5. Wu, Y., Lim, J., Yang, M.-H.: Object tracking benchmark. IEEE Trans. Pattern Anal. Mach. Intell. 37, 1834–1848 (2015)

    Article  Google Scholar 

  6. Henriques, J.F., Caseiro, R., Martins, P., Batista, J.: High-speed tracking with kernelized correlation filters. IEEE Trans. Pattern Anal. Mach. Intell. 37, 583–596 (2015)

    Article  Google Scholar 

  7. Bolme, D.S., Beveridge, J.R., Draper, B.A., Lui, Y.M.: Visual object tracking using adaptive correlation filters. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2544–2550 (2010)

  8. Henriques, J.F., Caseiro, R, Martins, P., Batista, J.: Exploiting the circulant structure of tracking-by-detection with kernels. In: European Conference on Computer Vision, pp. 702–715 (2012)

    Chapter  Google Scholar 

  9. Felzenszwalb, P.F., Girshick, R.B., McAllester, D., Ramanan, D.: Object detection with discriminatively trained part-based models. IEEE Trans. Pattern Anal. Mach. Intell. 32, 1627–1645 (2010)

    Article  Google Scholar 

  10. Van De Weijer, J., Schmid, C., Verbeek, J., Larlus, D.: Learning color names for real-world applications. IEEE Trans. Image Process. 18, 1512–1523 (2009)

    Article  MathSciNet  Google Scholar 

  11. Fazl-Ersi, E., Nooghabi, M.K.: Revisiting correlation-based filters for low-resolution and long-term visual tracking. Vis. Comput. 1–13 (2018). https://doi.org/10.1007/s00371-018-1510-1

    Article  Google Scholar 

  12. Ma, C., Huang, J.-B., Yang, X., Yang, M.-H.: Hierarchical convolutional features for visual tracking. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3074–3082 (2015)

  13. Wu, Y., Lim, J., Yang, M.-H.: Online object tracking: a benchmark. In: 2013 IEEE Conference on Computer vision and pattern recognition (CVPR), pp. 2411–2418 (2013)

  14. Li, X., Hu, W., Shen, C., Zhang, Z., Dick, A., Hengel, A.V.D.: A survey of appearance models in visual object tracking. ACM Trans. Intell. Syst. Technol. 4, 58 (2013)

    Google Scholar 

  15. Adam, A., Rivlin, E., Shimshoni, I.: Robust fragments-based tracking using the integral histogram. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), pp. 798–805 (2006)

  16. Iraei, I., Faez, K.: Object tracking with occlusion handling using mean shift, Kalman filter and edge histogram. In 2015 2nd International Conference on Pattern Recognition and Image Analysis, pp. 4799–8445 (2015)

  17. Li, D., Xu, L., Wu, Y.: Improved CAMShift object tracking based on Epanechnikov Kernel density estimation and Kalman filter. In: Control And Decision Conference (CCDC), 2017 29th Chinese, pp. 3120–3126 (2017)

  18. Jeong, J.-M., Yoon, T.-S., Park, J.-B.: Kalman filter based multiple objects detection-tracking algorithm robust to occlusion. In: 2014 Proceedings of the SICE Annual Conference (SICE), pp. 941–946 (2014)

  19. Tripathi, R.P., Ghosh, S., Chandle, J.: Tracking of object using optimal adaptive Kalman filter. In: 2016 IEEE International Conference on Engineering and Technology (ICETECH), pp. 1128–1131 (2016)

  20. Xiao, J., Oussalah, M.: Collaborative tracking for multiple objects in the presence of inter-occlusions. IEEE Trans. Circuits Syst. Video Technol. 26, 304–318 (2016)

    Article  Google Scholar 

  21. Ma, C., Yang, X., Zhang, C., Yang, M.-H.: Long-term correlation tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5388–5396 (2015)

  22. Mu, X., Che, J., Hu, T., Wang, Z.: A video object tracking algorithm combined Kalman filter and adaptive least squares under occlusion. In: International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI), pp. 6–10 (2016)

  23. Aishwarya, R., Maik, V., Chithravathi, B.: Robust object tracking using kernalized correlation filters (KCF) and Kalman predictive estimates. In 2017 International Conference on Recent Trends in Electronics Information and Communication Technology, pp. 587–591 (2017)

  24. Iswanto, I.A., Li, B.: Visual object tracking based on mean-shift and particle-Kalman filter. Proc. Comput. Sci. 116, 587–595 (2017)

    Article  Google Scholar 

  25. Wang, J., He, F., Zhang, X., Gao, Y.: Tracking objects through occlusions using improved Kalman filter. In: 2010 2nd International Conference on Advanced Computer Control (ICACC), pp. 223–228 (2010)

  26. Danelljan, M., Häger, G., Khan, F.S., Felsberg, M.: Discriminative scale space tracking. IEEE Trans. Pattern Anal. Mach. Intell. 39, 1561–1575 (2017)

    Article  Google Scholar 

  27. Hu, Q., Guo, Y., Lin, Z., An, W., Cheng, H.: Robust and real-time object tracking using scale-adaptive correlation filters. In 2016 International Conference on Digital Image Computing: Techniques and Applications (DICTA), pp. 1–8 (2016)

  28. Danelljan, M., Häger, G., Khan, F., Felsberg, M.: Accurate scale estimation for robust visual tracking. In: British Machine Vision Conference, Nottingham, September 1–5, 2014, pp. 1–65 (2014)

  29. Schölkopf, B., Smola, A.J.: Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. MIT Press, Cambridge (2002)

    Google Scholar 

  30. Simon, D.: Optimal State Estimation: Kalman, H Infinity, and Nonlinear Approaches. Wiley, New York (2006)

    Book  Google Scholar 

  31. Zhang, H., Liu, G.: Coupled-layer based visual tracking via adaptive kernelized correlation filters. Vis Comput 34, 41–54 (2018)

    Article  Google Scholar 

  32. Hadi, I., Sabah, M.: Behavior formula extraction for object trajectory using curve fitting method. Int. J. Comput. Appl. 104, 28–37 (2014)

    Google Scholar 

  33. Kristan, M., Matas, J., Leonardis, A., Felsberg, M., Cehovin, L., Fernandez, G., et al.: The visual object tracking vot2015 challenge results. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 1–23 (2015)

  34. Ross, D.A., Lim, J., Lin, R.-S., Yang, M.-H.: Incremental learning for robust visual tracking. Int. J. Comput. Vis. 77, 125–141 (2008)

    Article  Google Scholar 

  35. Babenko, B., Yang, M.-H., Belongie, S.: Visual tracking with online multiple instance learning. In: IEEE Conference on Computer Vision and Pattern Recognition, 2009. CVPR 2009, pp. 983–990 (2009)

  36. Hare, S., Golodetz, S., Saffari, A., Vineet, V., Cheng, M.-M., Hicks, S.L., et al.: Struck: structured output tracking with kernels. IEEE Trans. Pattern Anal. Mach. Intell. 38, 2096–2109 (2016)

    Article  Google Scholar 

  37. Bao, C., Wu, Y., Ling, H., Ji, H.: Real time robust l1 tracker using accelerated proximal gradient approach. In 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1830–1837 (2012)

  38. Zhang, K., Liu, Q., Wu, Y., Yang, M.-H.: Robust visual tracking via convolutional networks without training. IEEE Trans. Image Process. 25, 1779–1792 (2016)

    MathSciNet  MATH  Google Scholar 

  39. Kalal, Z., Matas, J., Mikolajczyk, K.: Pn learning: Bootstrapping binary classifiers by structural constraints. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 49–56 (2010)

  40. Ahmed, J., Jafri, M.N., Shah, M., Akbar, M.: Real-time edge-enhanced dynamic correlation and predictive open-loop car-following control for robust tracking. Mach. Vis. Appl. 19, 1–25 (2008)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, Yazd University, Yazd, Iran

    Mohammad Zolfaghari & Hossein Ghanei-Yakhdan

  2. Department of Electronics and Telecommunication Engineering, Shiraz University, Shiraz, Iran

    Mehran Yazdi

Authors
  1. Mohammad Zolfaghari
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hossein Ghanei-Yakhdan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mehran Yazdi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hossein Ghanei-Yakhdan.

Ethics declarations

Conflict of interest

All the authors declare that they have no conflict of interest.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zolfaghari, M., Ghanei-Yakhdan, H. & Yazdi, M. Real-time object tracking based on an adaptive transition model and extended Kalman filter to handle full occlusion. Vis Comput 36, 701–715 (2020). https://doi.org/10.1007/s00371-019-01652-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-019-01652-3

Keywords

Search

Navigation