Skip to main content

Advertisement

SpringerLink
Log in

Pedestrian tracking by fusion of thermal-visible surveillance videos

  • Short Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

In this paper we introduce a system to track pedestrians using a combined input from RGB and thermal cameras. Two major contributions are presented here. First is the novel probabilistic model of the scene background where each pixel is represented as a multi-modal distribution with the changing number of modalities for both color and thermal input. We demonstrate how to eliminate the influence of shadows with this type of fusion. Second, based on our background model we introduce a pedestrian tracker designed as a particle filter. We further develop a number of informed reversible transformations to sample the model probability space in order to maximize our model posterior probability. The novelty of our tracking approach also comes from a way we formulate observation likelihoods to account for 3D locations of the bodies with respect to the camera and occlusions by other tracked human bodies as well as static objects. The results of tracking on color and thermal sequences demonstrate that our algorithm is robust to illumination noise and performs well in the outdoor environments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Bhanu, B., Han, J.: Kinematic-based human motion analysis in infrared sequences. In: IEEE WS. on Applications of Computer Vision, pp. 208–212 (2002)

  2. CAVIAR: Ist 37540. Found at http://homepages.inf.ed.ac.uk/rbf/CAVIAR/ (2001)

  3. Collins R., Lipton A., Kanade T.: Introduction to the special section on video surveillance. IEEE Trans. Pattern Anal. Mach. Intell. 22(8), 745–746 (2000)

    Article  Google Scholar 

  4. Collins, R., Lipton, A., Kanade, T., Fujiyoshi, H., Duggins, D., Tsin, Y., Tolliver, D., Enomoto, N., Hasegawa, O., Burt, P., Wixson, L.: A system for video surveillance and monitoring. Tech. Rep. CMU-RI-TR-00-12. Carnegie Mellon University, Pittsburgh (2000)

  5. Connaire, C.O., O’Connor, N., Smeaton., A.F.: Thermo-visual feature fusion for object tracking using multiple spatiogram trackers. Mach. Vis. Appl., pp. 1–12 (2007). doi:10.1007/s00138-007-0078-y

  6. Dai, C., Zheng, Y., Li, X.: Layered representation for pedestrian detection and tracking in infrared imagery. In: IEEE CVPR WS on OTCBVS (2005)

  7. Davis, J., Sharma, V.: Fusion-based background-subtraction using contour saliency. In: IEEE International Workshop on Object Tracking and Classification Beyond the Visible Spectrum, IEEE OTCBVS WS Series Bench (2005)

  8. Davis, J.W., Sharma, V.: Fusion-based background-subtraction using contour saliency. In: IEEE CVPR WS on Object Tracking and Classification Beyond the Visible Spectrum, pp. 19–26 (2005)

  9. Deutscher, J., North, B., Bascle, B., Blake, A.b.: Tracking through singularities and discontinuities by random sampling. In: International Conference on Computer Vision (1999)

  10. Doucet A., de Freitas N., Gordon N.: Sequential Monte Carlo Methods in Practice. Springer, Heidelberg (2001)

    MATH  Google Scholar 

  11. Elgammal, A., Davis, L.: Probabilistic framework for segmenting people under occlusion. In: International Conference on Computer Vision (2001)

  12. Gavrila D.: The visual analysis of human movement: A survey. Comp. Vis. Image Understanding 73(1), 82–98 (1999)

    Article  MATH  Google Scholar 

  13. Haritaoglu, I., Flickner, M.: Detection and tracking of shopping groups in stores. In: International Conference on Computer Vision and Pattern Recognition (2001)

  14. Haritaoglu I., Harwood D., Davis L.: W-4: Real-time surveillance of people and their activities. IEEE Trans. Pattern Anal. Mach. Intell. 22(8), 809–830 (2000)

    Article  Google Scholar 

  15. Havasi, L., Sziranyi, T.: Motion tracking through grouped transient feature points. In: Advanced Concepts for Intelligent Vision Systems (2004)

  16. Isard, M., MacCormick, J.: Bramble: A bayesian multiple-blob tracker. In: International Conference on Computer Vision (2001)

  17. Kemp, C., Drummond, T.: Multi-modal tracking using texture changes. In: British Machine Vision Conference (2004)

  18. Kim, K., Chalidabhongse, T., Harwood, D., Davis, L.: Background modeling and subtraction by codebook construction. In: International Conference on Image Processing (2004)

  19. Leykin, A.: Visual human tracking and group activity analysis: A video mining system for retail marketing. PhD Thesis, Indiana University (2007)

  20. Leykin, A., Hammoud, R.: Robust multi-pedestrian tracking in thermal-visible surveillance videos. In: IEEE CVPR WS on Object Tracking and Classification Beyond the Visible Spectrum, p. 136. IEEE Computer Society, Los Alamitos (2006)

  21. Leykin, A., Tuceryan, M.: A vision system for automated customer tracking for marketing analysis: Low level feature extraction. In: Human Activity Recognition and Modelling Workshop (2005)

  22. Maybank S., Tan T.: Introduction to special section on visual surveillance. Int. J. Comp. Vis. 37(2), 173–173 (2000)

    Article  Google Scholar 

  23. Mittal, A., Davis, L.: M2tracker: A multi-view approach to segmenting and tracking people in a cluttered scene using region-based stereo. In: European Conference on Computer Vision (2002)

  24. Morris, D., Rehg, J.: Singularity analysis for articulated object tracking. In: International Conference on Computer Vision and Pattern Recognition (1998)

  25. Nanda, H., Davis, L.: Probabilistic template based pedestrian detection in infrared videos. In: IEEE Intell. Vehicles Symp. (2002)

  26. Oliver N., Rosario B., Pentland A.: A bayesian computer vision system for modeling human interactions. IEEE Trans. Pattern Anal. Mach. Intell. 22(8), 831–843 (2000)

    Article  Google Scholar 

  27. Owechko, Y., Medasani, S., Srinivasa, N.: Classifier swarms for human detection in infrared imagery. In: IEEE CVPR WS on Object Tracking and Classification Beyond the Visible Spectrum (2004)

  28. Rana Y., Weiss I., Zheng Q., Davis L.S.: Pedestrian detection via periodic motion analysis. Int. J. Comp. Vis 71(2), 143–160 (2007)

    Article  Google Scholar 

  29. Rittscher, J., Tu, P., Krahnstoever, N.: Simultaneous estimation of segmentation and shape. In: International Conference on Computer Vision and Pattern Recognition, vol. II, pp. 486–493 (2005)

  30. Sminchisescu, C., Triggs, B.: Kinematic jump processes for monocular 3d human tracking. In: International Conference on Computer Vision and Pattern Recognition (2003)

  31. Xu, F., Fujimura, K.: Pedestrian detection and tracking with night vision. In: IEEE Intell. Vehicles Symp. (2002)

  32. Yasuno, M., Yasuda, N., Aoki, M.: Pedestrian detection and tracking in far infrared images. In: IEEE CVPR WS on Object Tracking and Classification Beyond the Visible Spectrum (2004)

  33. Zhao, T., Nevatia, R.: Tracking multiple humans in crowded environment. In: International Conference on Computer Vision and Pattern Recognition (2004)

Download references

Author information

Authors and Affiliations

  1. Computer Science Department, Indiana University, Bloomington, IN, USA

    Alex Leykin

  2. Delphi Corporation, Electronics and Safety World Headquarters, Kokomo, IN, USA

    Riad Hammoud

Authors
  1. Alex Leykin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Riad Hammoud
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alex Leykin.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Leykin, A., Hammoud, R. Pedestrian tracking by fusion of thermal-visible surveillance videos. Machine Vision and Applications 21, 587–595 (2010). https://doi.org/10.1007/s00138-008-0176-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-008-0176-5

Keywords

Search

Navigation