Skip to main content
SpringerLink
Log in

Persistent people tracking and face capture using a PTZ camera

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

Pan–tilt–zoom (PTZ) camera is a powerful tool in far-field scenarios. However, most of the current PTZ surveillance systems require manual intervention to move the camera to the desired position. In this paper, we address the problem of persistent people tracking and face capture in uncontrolled scenarios using a single PTZ camera, which could prove most helpful in forensic applications. The system first detects and tracks pedestrians in zoomed-out mode. Then, according to a scheduler, the system selects a person to zoom in. In the zoomed-in mode, we detect a set of face images and solve the face–face association and face–person association problems. The system then zooms back out where tracking is continued as people re-appear in the view. The person–person association module associates the people on the schedule list with the people in the current view. The detected faces are associated with the corresponding people and trajectories. Due to the dynamic nature of our problem, e.g. the field of view of the camera changes because of the pan/tilt/zoom movement of the camera, all of the processes including receiving images from the camera and processing must be done in real time. To the best of our knowledge, the proposed method is the first to address the association of face images to people and trajectories using a single PTZ camera. Extensive experiments in challenging indoor and outdoor uncontrolled conditions demonstrate the effectiveness of the proposed system.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

References

  1. Andriluka, M., Roth, S., Schiele, B.: People-tracking-by-detection and people-detection-by-tracking. In: IEEE Conference on Computer Vision and Pattern Recognition (2008)

  2. Babenko, B., Yang, M.H., Belongie, S.: Robust object tracking with online multiple instance learning. IEEE Trans Pattern Anal Mach Intell 33(8), 1619–1632 (2011)

    Article  Google Scholar 

  3. Bagdanov, A.D., Del Bimbo, A., Nunziati, W.: Improving evidential quality of surveillance imagery through active face tracking. In: International Conference on Pattern Recognition (2006)

  4. Bagdanov, A.D., del Alberto, B., Federico, P.: Acquisition of high-resolution images through on-line saccade sequence planning. In: ACM International Workshop on Video Surveillance Sensor Networks, pp. 121–130 (2005)

  5. Benfold, B., Reid, I.: Stable multi-target tracking in real-time surveillance video. In: Computer Vision and Pattern Recognition, pp. 3457–3464 (2011)

  6. Choi, W., Savarese, S.: Multiple target tracking in world coordinate with single, minimally calibrated camera. In: European Conference on Computer Vision, pp. 553–567 (2010)

  7. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: Computer Vision and Pattern Recognition, pp. 886–893 (2005)

  8. Dinh, T., Vo, N., Medioni, G.G.: Context tracker: exploring supporters and distracters in unconstrained environments. In: Computer Vision and Pattern Recognition, pp. 1177–1184 (2011a)

  9. Dinh, T.B., Vo, N., Medioni, G.G.: High resolution face sequences from a PTZ network camera. In: IEEE Conference on Automatic Face and Gesture Recognition, pp. 531–538 (2011b)

  10. Dinh, T.B., Yu, Q., Medioni, G.G.: Real time tracking using an active pan-tilt-zoom network camera. In: International Conference on Intelligent Robots and Systems, pp. 3786–3793 (2009)

  11. Enzweiler, M., Gavrila, D.M.: Monocular pedestrian detection: survey and experiments. IEEE Trans. Pattern Anal. Mach. Intell. 31(12), 2179–2195 (2009)

    Article  Google Scholar 

  12. Felzenszwalb, P.F., Girshick, R.B., McAllester, D., Ramanan, D.: Object detection with discriminatively trained part-based models. IEEE Trans. Pattern Anal. Mach. Intell. 32(9), 1627–1645 (2010)

    Article  Google Scholar 

  13. Froba, B., Ernst, A.: Face detection with the modified census transform. In: IEEE Conference on Automatic Face and Gesture Recognition, pp. 91–96 (2004)

  14. Huang, C., Nevatia, R.: High performance object detection by collaborative learning of joint ranking of granules features. In: Computer Vision and Pattern Recognition, pp. 41–48 (2010)

  15. Huang, C., Wu, B., Nevatia, R.: Robust object tracking by hierarchical association of detection responses. In: European Conference on Computer Vision, pp. 788–801 (2008)

  16. Kalal, Z., Mikolajczyk, K., Matas, J.: Tracking-learning-detection. IEEE Trans. Pattern Anal. Mach. Intell. 34(7), 1409–1422 (2012)

    Article  Google Scholar 

  17. Lee, S.G., Batkhishig, R.: Implementation of a real-time image object tracking system for PTZ cameras. Converg. Hybrid Inf. Technol. 206, 121–128 (2011)

    Article  Google Scholar 

  18. Lee, S.C., Nevatia, R.: Robust camera calibration tool for video surveillance camera in urban environment. In: Computer Vision and Pattern Recognition Workshops, pp. 62–67 (2011)

  19. Li, Y., Bhanu, B., Lin, W.: Auction protocol for camera active control. In: International Conference on Image Processing, pp. 4325–4328 (2010)

  20. Munkres, J.: Algorithms for the assignment and transportation problems. J. Soc. Ind. Appl. Math. 5(1), 32–38 (1957)

    Article  MathSciNet  MATH  Google Scholar 

  21. Ojala, T., Pietikäinen, M., Mäenpää, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. Pattern Anal. Mach. Intell. 24(7), 971–987 (2002)

    Article  MATH  Google Scholar 

  22. Possegger, H., Mauthner, T., Roth, P.M., Bischof, H.: Occlusion geodesics for online multi-object tracking. In: IEEE Conference on Computer Vision and Pattern Recognition (2014)

  23. Qureshi, F., Terzopoulos, D.: Surveillance camera scheduling: a virtual vision approach. Multimed. Syst. 12, 269–283 (2006)

    Article  Google Scholar 

  24. Qureshi, F., Terzopoulos, D.: Distributed coalition formation in visual sensor networks: a virtual vision approach. Distrib. Comput. Sens. Syst. 4549, 1–20 (2007)

    Article  Google Scholar 

  25. Salvagnini, P., Cristani, M., Del Bue, A., Murino, V.: An experimental framework for evaluating PTZ tracking algorithms. In: International Conference on Computer Vision Systems, pp. 81–90 (2011)

  26. Salvagnini, P., Pernici, F., Cristani, M., Lisanti, G., Masi, I., del Bimbo, A., Murino, V.: Information theoretic sensor management for multi-target tracking with a single pan-tilt-zoom camera. In: IEEE Winter Applications of Computer Vision Conference (2014)

  27. Schumann, A., Bauml, M., Stiefelhagen, R.: Person tracking-by-detection with efficient selection of part-detectors. In: IEEE International Conference on Advanced Video and Signal Based Surveillance (2013)

  28. Silberschatz, A., Galvin P.B., Gagne, G.: Process scheduling. In: Operating System Concepts (2010)

  29. Smith, K., Gatica-Perez, D., Odobez, J.-M., Ba, S.: Evaluating multi-object tracking. In: Workshop on Empirical Evaluation Methods in Computer Vision (2005)

  30. Sommerlade, E., Reid, I.: Information-theoretic active scene exploration. In: Computer Vision and Pattern Recognition, pp. 1–7 (2008)

  31. Stauffer, C., Grimson, W.E.L.: Adaptive background mixture models for real-time tracking. In: IEEE Workshop on Visual Surveillance, pp. 246–252 (1999)

  32. Vaisenberg, R., Mehrotra, S., Ramanan, D.: Semartcam scheduler: semantics driven real-time data collection from indoor camera networks to maximize event detection. J. Real-Time Image Process. 5(4), 215–230 (2010)

    Article  Google Scholar 

  33. Varcheie, P.D.Z., Bilodeau, G.-A.: People tracking using a network-based PTZ camera. Mach. Vis. Appl. 22(4), 671–690 (2011)

    Article  Google Scholar 

  34. Verma, R.C., Schmid, C., Mikolajczyk, K.: Face detection and tracking in a video by propagating detection probabilities. IEEE Trans. Pattern Anal. Mach. Intell. 25(10), 1215–1228 (2003)

    Article  Google Scholar 

  35. Wu, B., Nevatia, R.: Detection of multiple, partially occluded humans in a single image by bayesian combination of edgelet part detectors. In: International Conference on Computer Vision (2005)

  36. Wu, B., Nevatia, R.: Detection and tracking of multiple, partially occluded humans by bayesian combination of edgelet based part detectors. Int. J. Comput. Vis. 75(2), 247–266 (2007)

    Article  Google Scholar 

  37. Wu, B., Nevatia, R.: Detection and segmentation of multiple, partially occluded objects by grouping, merging, assigning part detection responses. Int. J. Comput. Vis. 82, 121–128 (2009)

    Google Scholar 

  38. Xu, Y., Song, D.: Systems and algorithms for autonomously simultaneous observation of multiple objects using robotic PTZ cameras assisted by a wide-angle camera. In: International Conference on Intelligent Robots and Systems, pp. 3802–3807 (2009)

  39. Yang, B., Nevatia, R.: Online learned discriminative part-based appearance models for multi-human tracking. In: European Conference on Computer Vision, pp. 484–498 (2012)

  40. Yang, J., Wang, Y., Sowmya, A., Li, Z.: Vehicle detection and tracking with low-angle cameras. In: IEEE International Conference on Image Processing, pp. 685–688 (2010)

  41. Zhang, L., Li, Y., Nevatia, R.: Global data association for multi-object tracking using network flows. In: Computer Vision and Pattern Recognition, pp. 1–8 (2008)

  42. Zhao, T., Nevatia, R.: Tracking multiple humans in crowded environment. In: IEEE Conference on Computer Vision and Pattern Recognition, vol. 2 (2004)

Download references

Acknowledgments

This research was supported by Award No. 2011-IJ-CX-K054, awarded by the National Institute of Justice, Office of Justice Programs, U.S. Department of Justice and National Natural Science Foundation of China 61503381. The opinions, findings and conclusions or recommendations expressed in this publication are those of the authors and do not necessarily reflect those of the Department of Justice.

Author information

Authors and Affiliations

  1. Institute of Automation, Chinese Academy of Sciences, Beijing, 100190, China

    Yinghao Cai

  2. Institute for Robotics and Intelligent Systems, University of Southern California, Los Angeles, 90089, CA, USA

    Gérard Medioni

Authors
  1. Yinghao Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gérard Medioni
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yinghao Cai.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cai, Y., Medioni, G. Persistent people tracking and face capture using a PTZ camera. Machine Vision and Applications 27, 397–413 (2016). https://doi.org/10.1007/s00138-016-0758-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-016-0758-6

Keywords

Search

Navigation