Skip to main content
SpringerLink
Log in

Applications for a people detection and tracking algorithm using a time-of-flight camera

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

This paper outlines a method and applications for detection and tracking of people in depth images, acquired with a low-resolution Time-of-Flight (ToF) camera. This depth sensor is placed perpendicular to the ground in order to provide distance information from a top-view position. Usage of intrinsic and extrinsic camera parameters allows estimation of a ground plane and comparison to the measured distances of the ToF sensor in every pixel. Differences to the expected ground plane define foreground information, that is subsequently combined to associated regions. These regions of interest (ROI) are analyzed to distinguish persons from other objects by using a matched filter that is applied the height segmented depth information of each of these regions. The proposed method separates crowds into individuals and facilitates a multi-object tracking system based on Kalman filtering. Furthermore, we present several applications for the proposed method. Experiments with different crowding situations - from very low to very high density - and different heights of camera placements have proven the applicability and practicability of the system.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Bar-Shalom J, Rong Li X, Kirubarajan T (2001) Estimation with applications to tracking and navigation. Wiley, New York ChichesterWeinheim Brisbane Singapore Toronto

  2. Bevilacqua A, Di Stefano L, Azzari P (2006) People tracking using a time-of-flight depth sensor. In: IEEE fifth international conference on advanced video and signal based surveillance

  3. Beymer D, Konolige K (2003) Tracking people from a mobile platform. In: Experimental robotics VIII, springer tracts in advanced robotics, vol 5. Springer, Berlin Heidelberg, pp 234–244 doi:10.1007/3-540-36268-1_20

  4. Foix S, Alenyà G, Torras C (2011) Lock-in time-of-flight (ToF) cameras: a survey. IEEE Sensors J 11 (9):1917–1926. doi:10.1109/JSEN.2010.2101060

    Article  Google Scholar 

  5. Gavriilidis A, Schwerdtfeger T, Velten J, Schauland S, Hohmann L, Haselhoff A, Boschen F, Kummert A (2011) Multisensor data fusion for advanced driver assistance systems - the active safety car project. In: International workshop on multidimensional systems (nDs), vol 7, pp 1–5. 10.1109/nDS.2011.6076836

  6. Gonzalez RC, Woods RE (2001) Digital image processing, 2nd edn. Prentice Hall International, Boston

    Google Scholar 

  7. Han J, With P H (2011) Real-time multiple people tracking for automatic group-behavior evaluation in delivery simulation training. Multimedia Tools Appl 51 (3):913–933. doi:10.1007/s11042-009-0423-4

    Article  Google Scholar 

  8. Hansard M, Lee S, Choi O, Horaud R P (2012) Time of flight cameras: principles, methods, and applications. Springer briefs in computer science. Springer

  9. Harville M (2005) Stereo person tracking with short and long term plan-view appearance models of shape and color. In: Proceedings of international conference on advanced video and signal based surveillance (AVSS), vol 1. Santa Fe, pp 511–517

  10. Harville M, Li D (2004) Fast, integrated person tracking and activity recognition with plan-view templates from a single stereo camera. In: IEEE conference on computer vision and pattern recognition, Washington

  11. Helbing D, Mukerji P (2011) Crowd disasters as systemic failures: analysis of the love parade disaster. Tech. rep., ETH Risk Center, Zürich

  12. Ikemura S, Fujiyoshi H (2012) Human detection by haar-like filtering using depth information. In: International conference on pattern recognition (ICPR), 2012 21st, pp 813–816

  13. Mallat S (1998) A wavelet tour of signal processing. Academic Press, Elsevier Oxford

  14. PMDTechnologies (2010) PMD vision CamCube 3.0 Specsheet - High resolution 3D video camera. Tech. rep., PMDTechnologies GmbH

  15. Polus A, Schofer J, Ushpiz A (1983) Pedestrian flow and level of service. J Transp Eng 109 (1):46–56

    Article  Google Scholar 

  16. Reynolds M, Dobos J, Peel L, Weyrich T, Brostow G (2011) Capturing time-of-flight data with confidence. In: Proceedings of IEEE conference on computer vision and pattern recognition (CVPR)

  17. Ringbeck T (2007) A 3D time of flight camera for object detection. In: Optical 3-d measurement techniques

  18. Rodriguez M, Sivic J, Laptev I, Audibert JY (2011) Density-aware person detection and tracking in crowds. In: Proceedings of the international conference on computer vision (ICCV)

  19. Soille P (1999) Morphological image analysis: principles and applications. Springer, Berlin Heidelberg

  20. Stahlschmidt C, Gavriilidis A, Kummert A (2013) Density measurements from a top-view position using a time-of-flight camera. In: Proceedings of the 8th international workshop on multidimensional systems

  21. Still GK (2000) Crowd dynamics. PhD thesis, University of Warwick, Warwick

  22. Still GK (2011) Duisburg - 24th July 2010, love parade incident, expert report. Tech. rep., Bucks New University, Buckinghamshire

  23. Takahashi M, Fujii M, Naemura M, Satoh S (2013) Human gesture recognition system for TV viewing using time-of-flight camera. Multimedia Tools Appl 62 (3):761–783. doi:10.1007/s11042-011-0870-6

    Article  Google Scholar 

  24. Tanner R, Studer M, Zanoli A, Hartmann A (2008) People detection and tracking with TOF sensor. In: IEEE fifth international conference on advanced video and signal based surveillance

  25. Verstockt S, Hoecke S, Potter P, Lambert P, Hollemeersch C, Sette B, Merci B, Walle R (2012) Multi-modal time-of-flight based fire detection. Multimedia Tools Appl:1–26

  26. Zhang Z (2000) A flexible new technique for camera calibration. IEEE Trans Pattern Anal Mach Intel 22 (11):1330–1334. doi:10.1109/34.888718

    Article  Google Scholar 

Download references

Acknowledgments

The research leading to these results has received funding from the European Community’s Seventh Framework Programme (FP7 / 2007 - 2013) under grant agreement no. 218086.

Author information

Authors and Affiliations

  1. Faculty of Electrical, Information and Media Engineering, University of Wuppertal, Rainer-Gruenter-Straße 21, 42119, Wuppertal, Germany

    Carsten Stahlschmidt, Alexandros Gavriilidis, Jörg Velten & Anton Kummert

Authors
  1. Carsten Stahlschmidt
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alexandros Gavriilidis
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jörg Velten
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Anton Kummert
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Carsten Stahlschmidt.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Stahlschmidt, C., Gavriilidis, A., Velten, J. et al. Applications for a people detection and tracking algorithm using a time-of-flight camera. Multimed Tools Appl 75, 10769–10786 (2016). https://doi.org/10.1007/s11042-014-2260-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-014-2260-3

Keywords

Search

Navigation