Skip to main content
SpringerLink
Log in

Correlation, Kalman filter and adaptive fast mean shift based heuristic approach for robust visual tracking

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Correlation tracker is computation intensive (if the search space or the template is large), has template drift problem, and may fail in case of fast maneuvering target, rapid changes in its appearance, occlusion suffered by it and clutter in the scene. Kalman filter can predict the target coordinates in the next frame, if the measurement vector is supplied to it by a correlation tracker. Thus, a relatively small search space can be determined where the probability of finding the target in the next frame is high. This way, the tracker can become fast and reject the clutter, which is outside the search space in the scene. However, if the tracker provides wrong measurement vector due to the clutter or the occlusion inside the search region, the efficacy of the filter is significantly deteriorated. Mean-shift tracker is fast and has shown good tracking results in the literature, but it may fail when the histograms of the target and the candidate region in the scene are similar (even when their appearance is different). In order to make the overall visual tracking framework robust to the mentioned problems, we propose to combine the three approaches heuristically, so that they may support each other for better tracking results. Furthermore, we present novel method for (1) appearance model updating which adapts the template according to rate of appearance change of target, (2) adaptive threshold for similarity measure which uses the variable threshold for each forthcoming image frame based on current frame peak similarity value, and (3) adaptive kernel size for fast mean-shift algorithm based on varying size of the target. Comparison with nine state-of-the-art tracking algorithms on eleven publically available standard dataset shows that the proposed algorithm outperforms the other algorithms in most of the cases.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Similar content being viewed by others

References

  1. Yilmaz, A., Javed, O., Shah, M.: Object tracking: a survey. ACM Comput. Surv. (CSUR) 38(4), 1–45 (2006)

    Google Scholar 

  2. Hu, W., Tan, T., Wang, L., Maybank, S.: A survey on visual surveillance of object motion and behaviors. IEEE Trans. Syst. Man Cybern. 34, 334–352 (2004)

    Article  Google Scholar 

  3. Kettnaker, V., Zabih, R.: Bayesian multi-camera surveillance. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 23–25 June 1999, pp. 1–18 (1999)

  4. Collins, R.T., Lipton, A.J., Fujiyoshi, H., Kanade, T.: Algorithms for cooperative multisensor surveillance. Proc. IEEE 89(10), 1456–1477 (2001)

    Article  Google Scholar 

  5. Greiffenhagen, M., Comaniciu, D., Niemann, H., Ramesh, V.: Design, analysis, and engineering of video monitoring systems: an approach and a case study. Proc. IEEE 89(10), 1498–1517 (2001)

    Article  Google Scholar 

  6. Kumar, R., Sawhney, H., Samarasekera, S., Hsu, S., Tao, H., Guo, Y., Hanna, K., Pope, A., Wildes, R., Hirvonen, D., Hansen, M., Burt, P.: Aerial video surveillance and exploitation. Proc. IEEE 89(10), 1518–1539 (2001)

    Article  Google Scholar 

  7. Decarlo, D., Metaxas, D.: Optical flow constraints on deformable models with applications to face tracking. Int. J. Comput. Vis. 38(2), 99–127 (2000)

    Article  MATH  Google Scholar 

  8. Yang, M.H., Kriegman, D.J., Ahuja, N.: Detecting faces in images: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 24(1), 34–58 (2002)

    Article  Google Scholar 

  9. Stauffer, C., Grimson, W.E.L.: Learning patterns of activity using real-time tracking. IEEE Trans. Pattern Anal. Mach. Intell. 22(8), 747–757 (2000)

    Article  Google Scholar 

  10. Fablet, R., Black, M.J.: Automatic detection and tracking of human motion with a view-based representation. In: European Conference on Computer Vision (ECCV’02) 2002, pp. 476–491 (2002)

  11. Agarwal, A., Triggs, B.: Learning to track 3D human motion from silhouettes. In: International Conference on Machine Learning (ICML’04), Banff, Canada 2004, pp. 9–16 (2004)

  12. Rand, D., Kizony, R., Weiss, P.T.L.: The Sony PlayStation II EyeToy: low-cost virtual reality for use in rehabilitation. J. Neurol. Phys. Ther. 32(4), 155–163 (2008)

    Article  Google Scholar 

  13. Handmann, U., Kalinke, T., Tzomakas, C., Werner, M., von Seelen, W.: Computer vision for driver assistance systems. In: International Society for Optics and Photonics: Aerospace/Defense Sensing and Controls 1998, pp. 136–147 (1998)

  14. Avidan, S.: Support vector tracking. IEEE Trans. Pattern Anal. Mach. Intell. 26(8), 1064–1072 (2004)

    Article  Google Scholar 

  15. Coifman, B., Beymer, D., McLauchlan, P., Malik, J.: A real-time computer vision system for vehicle tracking and traffic surveillance. Transp. Res. Part C: Emerg. Technol. 6(4), 271–288 (1998)

    Article  Google Scholar 

  16. Bradski, G.R.: Real time face and object tracking as a component of a perceptual user interface. In: Fourth IEEE Workshop on Applications of Computer Vision (WACV’98). 1998, pp. 214–219 (1998)

  17. Papanikolopoulos, N.P., Khosla, P.K.: Adaptive robotic visual tracking: theory and experiments. IEEE Trans. Autom. Control 38(3), 429–445 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  18. Amini, A., Owen, R., Anandan, P., Duncan, J.: Non-rigid motion models for tracking the left-ventricular wall. In: Information Processing in Medical Imaging 1991, pp. 343–357 (1991)

  19. Vasconcelos, M.J.M., Ventura, S.M.R., Freitas, D.R.S., Tavares, J.M.R.S.: Using statistical deformable models to reconstruct vocal tract shape from magnetic resonance images. Proc. Inst. Mech. Eng. Part H: J. Eng. Med. 224(10), 1153–1163 (2010)

    Article  Google Scholar 

  20. Vasconcelos, M.J., Rua Ventura, S.M., Freitas, D.R.S., Tavares, J.M.R.S.: Towards the automatic study of the vocal tract from magnetic resonance images. J. Voice 25(6), 732–742 (2010)

    Google Scholar 

  21. Cafforio, C., Rocca, F.: Tracking moving objects in television images. Signal Process. 1(2), 133–140 (1979)

    Article  Google Scholar 

  22. Lucas, B.D., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: 7th International Joint Conference on Artificial Intelligence 1981 (1981)

  23. Fitts, J.M.: Precision correlation tracking via optimal weighting functions. In: 18th IEEE Conference on Decision and Control including the Symposium on Adaptive Processes 1979, pp. 280–283 (1979)

  24. Asgarizadeh, M., Pourghassem, H.: A robust object tracking synthetic structure using regional mutual information and edge correlation-based tracking algorithm in aerial surveillance application. Signal Image Video Process. 1–15 (2013)

  25. Wang, Y., Zhao, Q.: Robust object tracking via online principal component-canonical correlation analysis (P3CA). Signal Image Video Process. 1–16 (2013)

  26. Khan, M.I., Ahmed, J., Ali, A., Masood, A.: Robust edge-enhanced fragment based normalized correlation tracking in cluttered and occluded imagery. Signal Process. Image Process. Pattern Recogn. 12, 169–176 (2009)

    Google Scholar 

  27. Ahmed, J., Ali, A., Khan, A.: Stabilized active camera tracking system. J. Real-Time Image Process. 1–20 (2012)

  28. Ahmed, J.: Adaptive Edge-Enhanced Correlation Based Robust And Real-Time Visual Tracking Framework and Its Deployment in Machine Vision Systems. Research, National University of Science and Technology (NUST), Karachi (2008)

    Google Scholar 

  29. Ali, A., Kauser, H., Khan, M.I.: Automatic Visual Tracking and Firing System for Anti-Aircraft Machine Gun. In: 6th International Bhurban Conference of Applied Science and Technology, Islamabad, Pakistan, 2009, pp. 253–257 (2009)

  30. Ahmed, J., Jafri, M.N., Shah, M., Akbar, M.: Real-time edge-enhanced dynamic correlation and predictive open-loop car following control for robust tracking. Mach. Vis. Appl. J. 19(1), 1–25 (2008)

    Article  Google Scholar 

  31. Wong, S.: Advanced correlation tracking of objects in cluttered imagery. In: Defense and Security:International Society for Optics and Photonics 2005, pp. 158–169 (2005)

  32. Ali, A., Mirza, S.M.: Object tracking using correlation, Kalman filter and fast means shift algorithms. In: International Conference on Emerging Technologies, 2006. ICET’06, Islamabad, pp. 174–178 (2006)

  33. Wilson, J.N., Ritter, G.X.: Handbook of Computer Vision-Algorithms in Image Algebra. CRC Press, Boca Raton (2001)

    MATH  Google Scholar 

  34. Kuglin, C., Hines, D.: The phase correlation image alignment method. In: International Conference on Cybernetics and Society 1975, pp. 163–165 (1975)

  35. Chen, Q., Defrise, M., Deconinck, F.: Symmetric phase-only matched filtering of Fourier–Mellin transforms for image registration and recognition. IEEE Trans. Pattern Anal. Mach. Intell. 16(12), 1156–1168 (1994)

    Article  Google Scholar 

  36. Manduchi, R., Mian, G.A.: Accuracy analysis for correlation-based image registration algorithms. In: IEEE International Symposium on Circuits and Systems (ISCAS’93) 1993, pp. 834–837 (1993)

  37. Stone, H.S., Tao, B., McGuire, M.: Analysis of image registration noise due to rotationally dependent aliasing. J. Vis. Commun. Image Represent. 14(2), 114–135 (2003)

    Article  Google Scholar 

  38. Stone, H.S.: Fourier-based image registration techniques. NEC Research (2002)

  39. Ahmed, J., Jafri, M.N.: Improved phase correlation matching. In: ICISP-08: International Conference on Image and Signal Processing, France 2008, pp. 128–135 (2008)

  40. Jingying, J., Xiaodong, H., Kexin, X., Qilian, Y.: Phase correlation-based matching method with sub-pixel accuracy for translated and rotated images. In: IEEE International Conference on Signal Processing (ICSP’02) 2002, pp. 752–755 (2002)

  41. Foroosh, H., Zerubia, J.B., Berthod, M.: Extension of phase correlation to subpixel registration. IEEE Trans. Image Process. 11(3), 188–200 (2002)

    Article  Google Scholar 

  42. Keller, Y., Averbuch, A., Miller, O.: Robust Phase Correlation. In: 17th International Conference on Pattern Recognition (ICPR’04) 2004, pp. 740–743 (2004)

  43. Blackman, S., Popoli, R.: Design and Analysis of Modern Tracking Systems. Artech House, Boston (1999)

    MATH  Google Scholar 

  44. Gonzalez, R.C., Woods, R.E.: Digital Image Processing, 2nd edn. Prentice-Hall, Englewood Cliffs (2002)

    Google Scholar 

  45. Lewis, J.P.: Fast Normalized Cross-Correlation. In: Vision Interface 1995, pp. 120–123 (1995)

  46. Gonzalez, R.C., Woods, R.E., Eddins, S.L.: Digital Image Processing Using MATLAB. Pearson Education Pte. Ltd., Delhi (2004)

    Google Scholar 

  47. Nixon, M., Aguado, A.: Feature Extraction and Image Processing. Newnes, Oxford (2002)

    Google Scholar 

  48. Beleznai, C., Frühstück, B., Bischop, H.: Human detection in groups using a fast mean shift procedure. In: International Conference on Image Processing (ICIP), October 2004, pp. 349–352 (2004)

  49. Beleznai, C., Frühstück, B., Bischop, H.: Detecting humans in groups using a fast mean shift procedure. In: Proceedings of the 28th Workshop of the Austrian Association for Pattern Recognition (AAPR), June 2004, pp. 71–78 (2004)

  50. Beleznai, C., Frühstück, B., Bischop, H.: Tracking multiple humans using fast mean shift mode seeking. In: IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance, January 2005, pp. 25–32 (2005)

  51. Beleznai, C., Frühstück, B., Bischop, H.: Human tracking by mode seeking. In: Proceedings of the 4th International Symposium on Image and Signal Processing and Analysis (ISPA), September 2005, pp. 1–6 (2005)

  52. Beleznai, C., Frühstück, B., Bischop, H.: Human tracking by fast mean shift mode seeking. Trans. J. Multimed. 1(1), 1–8 (2006)

    Google Scholar 

  53. Wang, X., Liu, L., Tang, Z.: Infrared human tracking with improved mean shift algorithm based on multi-cue fusion. Trans. J. Appl. Otics 48(21), 4201–4212 (2009)

    Google Scholar 

  54. Sutor, S., Röhr, R., Pujolle, G., Reda, R.: Efficient mean shift clustering using exponential integral kernels. Trans. Int. J. Electric. Comput. Eng. 4(4), 206–210 (2009)

    Google Scholar 

  55. Shan, C., Tan, T., Wei, Y.: Real-time hand tracking using a mean shift embedded particle filter. Trans. Pattern Recogn. 40, 1958–1970 (2007)

    Article  MATH  Google Scholar 

  56. Yilmaz, A., Shafique, K., Lobo, N., Li, X., Olson, T., Shah, M.: Target tracking in FLIR imagery using mean shift and global motion compensation. In: IEEE Workshop on Computer Vision Beyond Visible Spectrum, Kauai, Hawaii 2001, pp. 54–58 (2001)

  57. Comaniciu, D., Ramesh, V., Meer, P.: Real-time tracking of non-rigid objects using mean shift. In: IEEE Conference on Computer Vision and Pattern Recognition, June 2000, pp. 142–149. Hilton Head, SC (2000)

  58. Comaniciu, D., Ramesh, V.: Mean shift and optimal prediction for efficient object tracking. In: IEEE International Conference on Image Processing (ICIP) 2000, pp. 70–73 (2000)

  59. Li, X., Zhang, T., Shen, X., Sun, J.: Object Tracking using an Adaptive Kalman Filter combined with Mean Shift. Opt. Eng. 49(2), 31–33 (2010)

    Google Scholar 

  60. Adam, A., Rivlin, E., Shimshoni, I.: Robust fragments-based tracking using the integral Histogram. In: IEEE Conference on Computer Vision and Pattern Recognition (ICPR) 2006, pp. 798–805 (2006)

  61. Brunson, R.L., Boesen, D.L., Crockett, G.A., Riker, J.F.: Precision trackpoint control via correlation track referenced to simulated imagery. In: International Society for Optics and Photonics: Aerospace Sensing 1992, pp. 325–336 (1992)

  62. Comaniciu, D., Ramesh, V., Meer, P.: Kernel-based object tracking. IEEE Trans. Pattern Anal. Mach. Intell. 25(5), 564–577 (2003)

    Google Scholar 

  63. Collins, R.T.: Mean-shift blob tracking through scale space. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition 2003, pp. 234–240 (2003)

  64. Ahmed, J., Shah, M., Miller, A., Harper, D., Jafri, M.N.: A Vision-based System for a UGV to Handle a Road Intersection. In: Proceedings of the National Conference on Artificial Intelligence 2007. Menlo Park, CA; Cambridge, MA; London; AAAI Press; MIT Press; 1999

  65. Ahmed, J., Jafri, M.N.: Best-match rectangle adjustment algorithm for persistent and precise correlation tracking. In: IEEE International Conference on Machine Vision (ICMV), Islamabad, Pakistan, 28–29 December 2007 (2007)

  66. Babenko, B., Yang, M.H., Belongie, S.: Robust object tracking with online multiple instance learning. IEEE Trans. Pattern Anal. Mach. Intell. 33(8), 1619–1632 (2011)

    Article  Google Scholar 

  67. Santner, J., Leistner, C., Saffari, A., Pock, T., Bischof, H.: PROST: Parallel robust online simple tracking. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2010, pp. 723–730 (2010)

  68. Oron, S., Bar-Hillel, A., Levi, D., Avidan, S.: Locally orderless tracking. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2012, pp. 1940–1947 (2012)

  69. Jia, X., Lu, H., Yang, M.H.: Visual tracking via adaptive structural local sparse appearance model. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2012, pp. 1822–1829 (2012)

  70. Kwon, J., Lee, K.M.: Visual tracking decomposition. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2010, pp. 1269–1276 (2010)

  71. Liu, B., Huang, J., Yang, L., Kulikowsk, C.: Robust tracking using local sparse appearance model and k-selection. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2011, pp. 1313–1320 (2011)

  72. http://vision.ucsd.edu/~bbabenko/project_miltrack.shtml

  73. http://gpu4vision.icg.tugraz.at/index.php?content=subsites/prost/prost.php

  74. http://www.cs.technion.ac.il/~amita/fragtrack/fragtrack.html

  75. http://groups.inf.ed.ac.uk/vision/caviar/caviardata1/

  76. http://cv.snu.ac.kr/research/~vtd/

  77. http://www.cs.toronto.edu/~dross/ivt/

  78. Everingham, M., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The pascal visual object classes (voc) challenge. Int. J. Comput. Vis. 88(2), 303–338 (2010)

    Article  Google Scholar 

  79. Ross, D.A., Lim, J., Lin, R.S., Yang, M.H.: Incremental learning for robust visual tracking. Int. J. Comput. Vis. 77(1), 125–141 (2008)

    Article  Google Scholar 

  80. Mei, X., Ling, H.: Robust visual tracking using \(\ell \) 1 minimization. In: IEEE 12th International Conference on Computer Vision 2009, pp. 1436–1443 (2009)

  81. Kalal, Z., Matas, J., Mikolajczyk, K.: Pn learning: Bootstrapping binary classifiers by structural constraints. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2010, pp. 49–56 (2010)

Download references

Acknowledgments

This research work is supported by PIEAS-administered HEC Endowment Fund for Higher education and R&D for IT and Telecom Sector Fund.

Author information

Authors and Affiliations

  1. Pakistan Institute of Engineering and Applied Sciences, Islamabad, Pakistan

    Ahmad Ali, Abdul Jalil, Muhammad Aksam Iftikhar & Mutawarra Hussain

  2. NUST Military College of Signals, Islamabad, Pakistan

    Javed Ahmed

Authors
  1. Ahmad Ali
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Abdul Jalil
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Javed Ahmed
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Muhammad Aksam Iftikhar
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mutawarra Hussain
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ahmad Ali.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ali, A., Jalil, A., Ahmed, J. et al. Correlation, Kalman filter and adaptive fast mean shift based heuristic approach for robust visual tracking. SIViP 9, 1567–1585 (2015). https://doi.org/10.1007/s11760-014-0612-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-014-0612-0

Keywords

Search

Navigation