Skip to main content
SpringerLink
Log in

Multi-Camera Human Activity Monitoring

  • Published:
Journal of Intelligent and Robotic Systems Aims and scope Submit manuscript

Abstract

With the proliferation of security cameras, the approach taken to monitoring and placement of these cameras is critical. This paper presents original work in the area of multiple camera human activity monitoring. First, a system is presented that tracks pedestrians across a scene of interest and recognizes a set of human activities. Next, a framework is developed for the placement of multiple cameras to observe a scene. This framework was originally used in a limited X, Y, pan formulation but is extended to include height (Z) and tilt. Finally, an active dual-camera system for task recognition at multiple resolutions is developed and tested. All of these systems are tested under real-world conditions, and are shown to produce usable results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Abrams, S., Allen, P.K., Tarabanis, K.A.: Dynamic sensor planning. In: Proceedings of the IEEE International Conference on Intelligent Autonomous Systems, pp. 206–215. Pittsburgh, PA, Feb. (1993)

  2. Atev, S., Masoud, O., Papanikolopoulos, N.: Practical mixtures of Gaussians with brightness monitoring. In: Proceedings of the IEEE International Conference on Intelligent Transportation Systems (2004)

  3. Azarbayejani, A., Pentland, A.: Real-time self-calibrating stereo person tracking using 3D shape estimation from blob features. In: Proceedings of the IEEE International Conference on Pattern Recognition (1996)

  4. Ben-Arie, J., Wang, Z., Pandit, P., Rajaram, S.: Human activity recognition using multidimensional indexing. IEEE Trans. Pattern Anal. Mach. Intell. 24(8), 1091–1104, August (2002)

    Article  Google Scholar 

  5. Beymer, D., Konolige, K.: Real-time tracking of multiple people using continuous detection. In: International Conference on Computer Vision (1999)

  6. Bodor, R.: Multi-camera human activity recognition in unconstrained indoor and outdoor environments. Ph.D. thesis, University of Minnesota, August (2005)

  7. Bodor, R., Drenner, A., Janssen, M., Schrater, P., Papanikolopoulos, N.: Mobile camera positioning to optimize the observability of human activity recognition tasks. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems. Edmonton, Canada, Aug. (2005)

  8. Bodor, R., Jackson, B., Masoud, O., Papanikolopoulos, N.: Image-based reconstruction for view-independent human motion recognition. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, Oct. (2003)

  9. Bodor, R., Jackson, B., Papanikolopoulos, N.: Vision-based human tracking and activity recognition. In: Proceedings of the 11th Mediterranean Conference on Control and Automation, June (2003)

  10. Bregler, C.: Learning and recognizing human dynamics in video sequences. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (1997)

  11. Cao, D., Masoud, O., Boley, D., Papanikolopoulos, N.: Online motion classification using support vector machines. In: IEEE International Conference on Robotics and Automation, Apr. (2004)

  12. Chen, X., Davis, J.: Camera placement considering occlusion for robust motion capture. Technical Report CS-TR-2000-07, Stanford University (2000)

  13. Collins, R., Amidi, O., Kanade, T.: An active camera system for acquiring multi-view video. In: Proceedings of the 2002 International Conference on Image Processing, Sept. (2002)

  14. Cretual, A., Chaumette, F., Bouthemy, P.: Complex object tracking by visual servoing based on 2D image motion. In: Proceedings of the IAPR International Conference on Pattern Recognition, Australia, Aug. (1998)

  15. Cutler, R., Turk, M.: View-based interpretation of real-time optical flow for gesture recognition. In: Proceedings of the Third IEEE Conference on Face and Gesture Recognition. Nara, Japan, Apr. (1998)

  16. Daniilidis, K., Krauss, C., Hansen, M., Sommer, G.: Real-time tracking of moving objects with an active camera. Real-Time Imaging 4, 3–20 (1998)

    Article  Google Scholar 

  17. Darrell, T., Gordon, G., Harville, M., Woodfill, J.: Integrated person tracking using stereo, color, and pattern detection. Int. J. Comput. Vis. 37(2), 175–185, June (2000)

    Article  MATH  Google Scholar 

  18. Davis, J., Chen, X.: Calibrating pan-tilt cameras in wide-area surveillance networks. In: Proceedings of the IEEE International Conference on Computer Vision (2003)

  19. Denzler, J., Zobel, M., Niemann, H.: On optimal camera parameter selection in Kalman filter based object tracking. In: Proceedings of the 24th DAGM Symposium on Pattern Recognition, pp. 17–25. Zurich, Switzerland (2002)

  20. Efros, A., Berg, A., Mori, G., Malik, J.: Recognizing action at a distance. In: Proceedings of the IEEE Conference on Computer Vision (2003)

  21. Elgammal, A., Duraiswami, R., Harwood, D., Davis, L.: Background and foreground modeling using nonparametric kernel density estimation for visual surveillance. Proc. IEEE 90, 1151–1163 (2002)

    Article  Google Scholar 

  22. Eveland, C., Konolige, K., Boles, R.: Background modeling for segmentation of video-rate stereo sequences. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Santa Barbara, CA (1998)

  23. Fablet, R., Black, M.J.: Automatic detection and tracking of human motion with a view-based representation. In: European Conference on Computer Vision, May (2002)

  24. Fleishman, S., Cohen-Or, D., Lischinski, D.: Automatic camera placement for image-based modeling. In: Proceedings of Pacific Graphics, vol. 99, pp. 12–20 (1999)

  25. Gasser, G., Bird, N., Masoud, O., Papanikolopoulos, N.: Human activity monitoring at bus stops. In: Proceedings of the IEEE Conference on Robotics and Automation (2004)

  26. Gerkey, B., Vaughan, R.T., Howard, A.: The player/stage project: tools for multi-robot and distributed sensor systems. In: Proceedings of the 11th International Conference on Advanced Robotics, pp. 317–323. Coimbra, Portugal, June (2003)

  27. Grimson, W., Stauffer, C., Romano, R., Lee, L.: Using adaptive tracking to classify and monitor activities in a site. In: Proceedings of the International Conference on Computer Visiona and Pattern Recognition, June (1998)

  28. Haritaoglu, I., Harwood, D., Davis, L.: W4: real-time surveillance of people and their activities. IEEE Trans. Pattern Anal. Mach. Intell. 22(8), 809–830 (2000)

    Article  Google Scholar 

  29. Heisele, B., Verri, A., Poggio, T.: Learning and vision machines. Proc. IEEE 90(7), 1164–1177 (2002)

    Article  Google Scholar 

  30. Hodgins, J., O’Brien, J., Tumblin, J.: Do geometric models affect judgements of human motion? In: Proceedings of the Graphics Interface Conference. Canada, May (1997)

  31. Isler, V., Kannan, S., Daniilidis, K.: Vc-dimension of exterior visibility. IEEE Trans. Pattern Anal. Mach. Intell. 26(5), 667–671 (2004)

    Article  Google Scholar 

  32. Lin, C., Wang, C., Chang, Y., Chen, Y.: Real-time object extraction and tracking with an active camera using image mosaics. In: Proceedings of the IEEE International Workshop on Multimedia Signal Processing, Dec. (2002)

  33. Lyons, D.: Discrete event modeling of misrecognition in PTZ tracking. In: Proceedings of the IEEE Conference on Advanced Video and Signal Based Surveillance. Miami, FL, July (2003)

  34. Masoud, O., Papanikolopoulos, N.P.: A method for human action recognition. Image Vis. Comput. 21(8), 729–743 (2003)

    Article  Google Scholar 

  35. Masoud, O., Papanikolopoulos, N.P.: Using geometric primitives to calibrate traffic scenes. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems. Japan, Oct. (2004)

  36. Maurin, B., Masoud, O., Papanikolopoulos, N.: Monitoring crowded traffic scenes. In: Proceedings of the IEEE International Conference on Intelligent Transportation Systems. Singapore, Sept. (2002)

  37. McKenna, S., Jabri, S., Duric, Z., Wechsler, H.: Tracking interacting people. In: Proceedings of the Conference on Automatic Face and Gesture Recognition. Grenoble, France, Mar. (2000)

  38. Mittal, A., Huttenlocher, D.: Scene modeling for wide area surveillance and image synthesis. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. South Carolina, June (2000)

  39. Mori, H., Charkari, M., Matsushita, T.: On-line vehicle and pedestrian detections based on sign pattern. IEEE Trans. Ind. Electron. 41(4), 384–391 (1994)

    Article  Google Scholar 

  40. Murray, D., Basu, A.: Motion tracking with an active camera. IEEE Trans. Pattern Anal. Mach. Intell. 16(5), 449–459 (1994)

    Article  Google Scholar 

  41. Nelson, B., Khosla, P.K.: Increasing the tracking region of an eye-in-hand system by singluarity and joint limit avoidance. In: Proceedings of the IEEE International Conference on Robotics and Automation, vol. 3, pp. 418–423 (1993)

  42. Nelson, B., Khosla, P.K.: Integrating sensor placement and visual tracking strategies. In: Proceedings of the 1994 IEEE International Conference on Robotics and Automation, vol. 2, pp. 1351–1356 (1994)

  43. Nelson, B., Khosla, P.K.: The resolvability ellipsoid for visual servoing. In: Proceedings of the 1994 IEEE Conference on Computer Vision and Pattern Recognition, pp. 829–832 (1994)

  44. Olague, G., Mohr, R.: Optimal 3D sensor placement to obtain accurate 3D point positions. In: Proceedings of the Fourteenth International Conference on Pattern Recognition, vol. 1, pp. 16–20, August (1998)

  45. Oliver, N., Rosario, B., Pentland, A.: A Bayesian computer vision system for modeling human interactions. IEEE Trans. Pattern Anal. Mach. Intell. 22(8), 831–843 (2000)

    Article  Google Scholar 

  46. O’Rourke, J.: Art Gallery Theorems and Algorithms. Oxford University Press, New York (1987)

    MATH  Google Scholar 

  47. Peixoto, P., Batista, J., Araujo, H., de Almeida, A.: Combination of several vision sensors for interpretation of human actions. In: Proceedings of the Conference on Experimental Robotics (2000)

  48. Pers, J., Vuckovic, G., Dezman, B., Kovacic, S.: Human activities at different levels of detail. In: Proceedings of the Czech Pattern Recognition Society Computer Vision Winter Workshop, pp. 27–32, Feb. (2003)

  49. Polana, R., Nelson, R.: Nonparametric recognition of nonrigid motion. Technical report, University of Rochester (1994)

  50. Rosales, R., Sclaroff, S.: 3D trajectory recovery for tracking multiple objects and trajectory guided recognition of actions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, June (1999)

  51. Sharma, R., Hutchinson, S.: Motion perceptibility and its application to active vision-based servo control. IEEE Trans. Robot. Autom. 13(4), 607–617 (1997)

    Article  Google Scholar 

  52. Starner, T., Pentland, A.: Real-time american sign language recognition from video using hidden markov models. In: Proceedings of the IEEE International Symposium on Computer Vision (1995)

  53. Stauffer, C., Grimson, W.: Learning patterns of activity using real-time tracking. IEEE Trans. Pattern Anal. Mach. Intell. 22(8), 747–757, August (2000)

    Article  Google Scholar 

  54. Stauffer, C., Tieu, K.: Automated multi-camera planar tracking correspondence modeling. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, vol. 1, June (2003)

  55. Stillman, S., Tanawongsuwan, R., Essa, I.: A system for tracking and recognizing multiple people with multiple cameras. In: Proceedings of the International Conference on Audio and Video Based Biometric Person Authentication, March (1999)

  56. Tarabanis, K.A., Tsai, R.Y., Kaul, A.: Computing occlusion-free viewpoints. IEEE Trans. Pattern Anal. Mach. Intell. 18(3), 273–292, Mar. (1996)

    Article  Google Scholar 

  57. Thornton, I., Pinto, J., Shiffrar, M.: The visual perception of human locomotion. Cogn. Neuropsychol. 15(6), 535–552 (1998)

    Article  Google Scholar 

  58. Tordoff, B.J., Murray, D.W.: Resolution vs. tracking error: zoom as a gain controller. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition, June (2003)

  59. Ukita, N., Matsuyama, T.: Incremental observable-area modeling for cooperative tracking. In: Proceedings of the International Conference on Pattern Recognition, Sept. (2000)

  60. Viola, P., Jones, M., Snow, D.: Detecting pedestrians using patterns of motion and appearance. Int. J. Comput. Vis. 63, 153–161 (2005)

    Article  Google Scholar 

  61. Wren, C., Azarbayejani, A., Darrell, T., Pentland, A.: Pfinder: real-time tracking of the human body. IEEE Trans. Pattern Anal. Mach. Intell. 19(7), 780–785, July (1997)

    Article  Google Scholar 

  62. Yao, Y., Allen, P.: Computing robust viewpoints with multi-constraints using tree annealing. IEEE Trans. Pattern Anal. Mach. 2, 993–998 (1995)

    Google Scholar 

  63. Zhou, X., Collins, R., Kanade, T., Metes, P.: A master-slave system to acquire biometric imagery of humans at distance. In: Proceedings of the ACM International Workshop on Video Surveillance (2003)

  64. Zobel, M., Denzler, J., Niemann, H.: Binocular 3-D object tracking with varying focal lengths. In: Proceedings of the IASTED International Conference Signal Processing, Pattern Recognition, and Applications, July (2002)

Download references

Author information

Authors and Affiliations

  1. University of Minnesota, CSE, 200 Union Street SE, Minneapolis, MN, 55455, USA

    Loren Fiore, Duc Fehr, Robot Bodor, Andrew Drenner, Guruprasad Somasundaram & Nikolaos Papanikolopoulos

Authors
  1. Loren Fiore
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Duc Fehr
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Robot Bodor
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Andrew Drenner
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Guruprasad Somasundaram
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Nikolaos Papanikolopoulos
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nikolaos Papanikolopoulos.

Additional information

This work has been supported by the NSF through grants #IIS-0219863, #CNS-0224363, #CNS-0324864, #IIP-0443945, #CNS-0420836, #IIP-0726109, and #CNS-0708344.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Fiore, L., Fehr, D., Bodor, R. et al. Multi-Camera Human Activity Monitoring. J Intell Robot Syst 52, 5–43 (2008). https://doi.org/10.1007/s10846-007-9201-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10846-007-9201-6

Keywords

Search

Navigation