Abstract
We propose Action-Reaction Learning as an approach for analyzing and synthesizing human behaviour. This paradigm uncovers causal mappings between past and future events or between an action and its reaction by observing time sequences. We apply this method to analyze human interaction and to subsequently synthesize human behaviour. Using a time series of perceptual measurements, a system automatically discovers correlations between past gestures from one human participant (action) and a subsequent gesture (reaction) from another participant. A probabilistic model is trained from data of the human interaction using a novel estimation technique, Conditional Expectation Maximization (CEM). The estimation uses general bounding and maximization to monotonically find the maximum conditional likelihood solution. The learning system drives a graphical interactive character which probabilistically predicts a likely response to a user’s behaviour and performs it interactively. Thus, after analyzing human interaction in a pair of participants, the system is able to replace one of them and interact with a single remaining user.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
A. Azarbayejani and A. Pentland. Real-time self-calibrating stereo person tracking using 3-d shape estimation from blob features. In International Conference on Pattern Recognition (ICPR), 1996.
C. Bishop. Neural Networks for Pattern Recognition. Oxford Press, 1996.
C. Bregler. Learning and recognizing human dynamics in video sequences. In IEEE Conf. on Computer Vision and Pattern Recognition, 1997.
R.A. Brooks. From earwigs to humans. Robotics and Autonomous Systems, 20(2–4), 1997.
A. Blake and A. Yuille. Active vision. MIT Press, 1992.
K. Dautenhahn and G. Hayes, editors. Agents in Interaction-Acquiring Competence through Imitation. Inter. Conf. on Autonomous Agents, 1998.
A.P. Dempster, N.M. Laird, and D.B. Rubin. Maximum likelihood from incomplete data via the em algorithm. Journal of the Royal Statistical Society, B39, 1977.
S. Elliott and J. R. Anderson. The effect of memory decay on predictions from changing categories. Journal of Experimental Psychology: Learning, Memory and Cognition, 1995.
N. Gershenfeld and A. Weigend. Time Series Prediction: Forecasting the Future and Understanding the Past. Addison-Wesley, 1993.
M. Isaard and A. Blake. A mixed-state condensation tracker with automatic model-switching. In International Conference on Computer Vision 6, 1998.
N. Johnson, A. Galata, and D. Hogg. The acquisition and use of interaction behaviour models. In IEEE Conf. on Computer Vision and Pattern Recognition, 1998.
M.I. Jordan and R.A. Jacobs. Hierarchical mixtures of experts and the em algorithm. Neural Computation, 6:181–214, 1994.
T. Jebara and A. Pentland. Maximum conditional likelihood via bound maximization and the cem algorithm. In Neural Information Processing Systems (NIPS) 11, 1998.
T. Jebara, K. Russel, and A. Pentland. Mixtures of eigenfeatures for real-time structure from texture. In Proceedings of the International Conference on Computer Vision, 1998.
K.S. Lashley. The problem of serial order in behavior. In L.A. Jefress, editor, Cerebral Mechanisms in Behavior, pages 112–136, New York, 1951. The Hixon Symposium, John Wiley.
E. W. Large, H. I. Christensen, and R. Bajcsy. Scaling dynamic planning and control: Cooperation through competition. In IEEE International Conference on Robotics and Automation, 1997.
M.J. Mataric. Behavior-based robotics as a tool for synthesis of artificial behavior and analysis of natural behavior. Trends in Cognitive Science, 2(3), 1998.
P. Maes, T. Darrel, B. Blumberg, and A. Pentland. The alive system: Wireless, full-body interaction with autonomous agents. Multimedia and Multisensory Virtual Worlds, ACM Multimedia Systems, 1996.
N. Oliver, A. Pentland, F. Berard, and J. Coutaz. Lafter: Lips and face tracker. In Computer Vision and Pattern Recognition Conference’ 97, 1997.
P. Pirjanian and H.I. Christensen. Behavior coordination using multiple-objective decision making. In SPIE Conf. on Intelligent Systems and Advanced Manufacturing, 1997.
A. Pentland and A. Liu. Modeling and prediction of human behavior. In IEEE Intelligent Vehicles 95, 1995.
A.C. Popat. Conjoint probabilistic subband modeling (phd. thesis). Technical Report 461, M.I.T. Media Laboratory, 1997.
E.L. Thorndike. Animal intelligence. an experimental study of the associative process in animals. Psychological Review, Monograph Supplements, 2(4):109, 1898.
D. Terzopoulos, X. Tu, and Grzeszczukm R. Artificial fishes: Autonomous locomotion, perception, behavior, and learning in a simulated physical world. Artificial Life, 1(4):327–351, 1994.
E. Uchibe, M. Asada, and K. Hosoda. State space construction for behaviour acquisition in multi agent environments with vision and action. In Proceedings of the International Conference on Computer Vision, 1998.
E.A. Wan. Time series prediction by using a connectionist network with internal delay lines. In A.S. Weigend and N.A. Gershenfeld, editors, Time Series Prediction, 1993.
J.B. Watson. Psychology as the behaviorist views it. Psychological Review, 20:158–117, 1913.
A. Wilson and A. Bobick. Recognition and interpretation of parametric gesture. In International Conference on Computer Vision, 1998.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 1999 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Jebara, T., Pentland, A. (1999). Action Reaction Learning: Automatic Visual Analysis and Synthesis of Interactive Behaviour. In: Computer Vision Systems. ICVS 1999. Lecture Notes in Computer Science, vol 1542. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-49256-9_17
Download citation
DOI: https://doi.org/10.1007/3-540-49256-9_17
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-65459-9
Online ISBN: 978-3-540-49256-6
eBook Packages: Springer Book Archive