Abstract
We describe a real-time computer vision and machine learning system for modeling and recognizing human behaviors in a visual surveillance task. The system is particularly concerned with detecting when interactions between people occur, and classifying the type of interaction. Examples of interesting interaction behaviors include following another person, altering one's path to meet another, and so forth. Our system combines top-down with bottom-up information in a closed feedback loop, with both components employing a statistical Bayesian approach. We propose and compare two different state-based learning architectures, namely HMMs and CHMMs, for modeling behaviors and interactions. The CHMM model is shown to work much more efficiently and accurately.
Finally, to deal with the problem of limited training data, a synthetic ‘Alife-style’ training system is used to develop flexible prior models for recognizing human interactions. We demonstrate the ability to use these a priori models to accurately classify real human behaviors and interactions with no additional tuning or training.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
R.K. Bajcsy. Active perception vs. passive perception. In CVWS85, pages 55–62, 1985.
A. Bobick and R. Bolles. The representation space paradigm of concurrent evolving object descriptions. PAMI, pages 146–156, February 1992.
A.F. Bobick. Computers seeing action. In Proceedings of BMVC, volume 1, pages 13–22, 1996.
Matthew Brand. Coupled hidden markov models for modeling interacting processes. Submitted to Neural Computation, November 1996.
Matthew Brand, Nuria Oliver, and Alex Pentland. Coupled hidden markov models for complex action recognition. In In Proceedings of IEEE CVPR97, 1996.
W.L. Buntine. Operations for learning with graphical models. Journal of Artificial Intelligence Research, 1994.
W.L. Buntine. A guide to the literature on learning probabilistic networks from data. IEEE Transactions on Knowledge and Data Engineering, 1996.
Hilary Buxton and Shaogang Gong. Advanced visual surveillance using bayesian networks. In International Conference on Computer Vision, Cambridge, Massachusetts, June 1995.
C. Castel, L. Chaudron, and C. Tessier. What is going on? a high level interpretation of sequences of images. In Proceedings of the workshop on conceptual descriptions from images, ECCV, pages 13–27, 1996.
T. Darrell and A. Pentland. Active gesture recognition using partially observable markov decision processes. In ICPR96, page C9E.5, 1996.
J.H. Fernyhough, A.G. Cohn, and D.C. Hogg. Building qualitative event models automatically from visual input. In ICCV98, pages 350–355, 1998.
Zoubin Ghahramani and Michael I. Jordan. Factorial hidden Markov models. In David S. Touretzky, Michael C. Mozer, and M.E. Hasselmo, editors, NIPS, volume 8, Cambridge, MA, 1996. MITP.
David Heckerman. A tutorial on learning with bayesian networks. Technical Report MSR-TR-95-06, Microsoft Research, Redmond, Washington, 1995. Revised June 96.
T. Huang, D. Koller, J. Malik, G. Ogasawara, B. Rao, S. Russel, and J. Weber. Automatic symbolic traffic scene analysis using belief networks. pages 966–972. Proceedings 12th National Conference in AI, 1994.
R. Kauth, A. Pentland, and G. Thomas. Blob: An unsupervised clustering approach to spatial preprocessing of mss imagery. In 11th Int’l Symp. on Remote Sensing of the Environment, Ann Harbor MI, 1977.
B. Moghaddam and A. Pentland. Probabilistic visual learning for object detection. In ICCV95, pages 786–793, 1995.
H.H. Nagel. From image sequences towards conceptual descriptions. IVC, 6(2):59–74, May 1988.
N. Oliver, F. Berard, and A. Pentland. Lafter: Lips and face tracking. In IEEE International Conference on Computer Vision and Pattern Recognition (CVPR97), S.Juan, Puerto Rico, June 1997.
N. Oliver, B. Rosario, and A. Pentland. Graphical models for recognizing human interactions. In To appear in Proceedings of NIPS98, Denver, Colorado, USA, November 1998.
A. Pentland. Classification by clustering. In IEEE Symp. on Machine Processing and Remotely Sensed Data, Purdue, IN, 1976.
A. Pentland and A. Liu. Modeling and prediction of human behavior. In DARPA97, page 201 206, 1997.
Lawrence R. Rabiner. A tutorial on hidden markov models and selected applications in speech recognition. PIEEE, 77(2):257–285, 1989.
Lawrence K. Saul and Michael I. Jordan. Boltzmann chains and hidden Markov models. In Gary Tesauro, David S. Touretzky, and T.K. Leen, editors, NIPS, volume 7, Cambridge, MA, 1995. MITP.
Padhraic Smyth, David Heckerman, and Michael Jordan. Probabilistic independence networks for hidden Markov probability models. AI memo 1565, MIT, Cambridge, MA, Feb 1996.
C. Williams and G. E. Hinton. Mean field networks that learn to discriminate temporally distorted strings. In Proceedings, connectionist models summer school, pages 18–22, San Mateo, CA, 1990. Morgan Kaufmann.
C. Wren, A. Azarbayejani, T. Darrell, and A. Pentland. Pfinder: Real-time tracking of the human body. InPhotonics East, SPIE, volume 2615, 1995. Bellingham, WA.
C.R. Wren, A. Azarbayejani, T. Darrell, and A. Pentland. Pfinder: Real-time tracking of the human body. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(7):780–785, July 1997.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 1999 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Oliver, N., Rosario, B., Pentland, A. (1999). A Bayesian Computer Vision System for Modeling Human Interactions. In: Computer Vision Systems. ICVS 1999. Lecture Notes in Computer Science, vol 1542. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-49256-9_16
Download citation
DOI: https://doi.org/10.1007/3-540-49256-9_16
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-65459-9
Online ISBN: 978-3-540-49256-6
eBook Packages: Springer Book Archive