Abstract
Mutual gaze arises from the interaction of the gaze behavior of two individuals. It is an important part of all face-to-face social interactions, including verbal exchanges. In order for humanoid robots to interact more naturally with people, they need internal models that allow them to produce realistic social gaze behavior. The approach taken in this work is to collect data from human conversational pairs with the goal of learning a controller for robot gaze directly from human data. In a small initial data collection experiment, mutual gaze between pairs of people is detected and recorded in real time during conversational interaction. A Markov model representation of human gaze data is produced in order to demonstrate how this data could be used to create a controller. We also discuss how an algebraic analysis of the state transition structure of such models may reveal interesting properties of human gaze interaction. Results are also presented from a second, larger experiment in which mutual gaze is detected offline using recorded video data for greater accuracy. Trends in behavior linking gaze and speech in this data set are also discussed.
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
This script was developed by Frank Kuegler and can be found at http://www.ling.uni-potsdam.de/~kuegler/docs/praattut/mark_pauses.script.
References
Applied Science Laboratories (2009) Mobile eye gaze tracking system. http://asleyetracking.com/
Argyle M (1988) Bodily communication, 2nd edn. Routledge, London
Baron-Cohen S, Campbell R, Karmiloff-Smith A, Grant J, Walker J (1995) Are children with autism blind to the mentalistic significance of the eyes? Br J Dev Psychol 13:379–398
Baron-Cohen S, Wheelwright S, Jolliffe T (1997) Is there a “language of the eyes”? Evidence from normal adults, and adults with autism or Asperger syndrome. Vis Cogn 4:311–331
Baron-Cohen S, Wheelwright S, Hill J, Raste Y, Plumb I (2001) The ‘reading the mind in the eyes’ test revised version: a study with normal adults, and adults with Asperger syndrome or high-functioning autism. J Child Psychol Psychiatry 42:241–252
Boersma P, Weenink D (2011) Praat: doing phonetics by computer [computer program], version 5.3.03. Retrieved from: http://www.praat.org/
Call J, Tomasello M (2003) Social cognition. In: Maestripieri D (ed) Primate psychology. Harvard University Press, Cambridge, pp 234–253
Cook M, Smith JM (1975) The role of gaze in impression formation. Br J Soc Clin Psychol 14(1):19–25
Dovidio JF, Ellyson SL (1982) Decoding visual dominance: attributions of power based on relative percentages of looking while speaking and looking while listening. Soc Psychol Q 45(2):106–113. http://www.jstor.org/stable/3033933
Egri-Nagy A, Nehaniv CL (2005) Algebraic hierarchical decomposition of finite state automata: comparison of implementations for Krohn-Rhodes theory. In: Implementation and application of automata (CIAA) 2004, revised selected papers. Lecture notes in computer science, vol 3317. Springer, Berlin, pp 315–316
Egri-Nagy A, Nehaniv CL (2008) Hierarchical coordinate systems for understanding complexity and its evolution, with applications to genetic regulatory networks. Artif Life (special issue on evolution of complexity) 14(3):299–312
Egri-Nagy A, Nehaniv CL (2011) SgpDec: hierarchical composition and decomposition of permutation groups and transformation semigroups. http://sgpdec.sourceforge.net/
Farroni T (2003) Infants perceiving and acting on the eyes: tests of an evolutionary hypothesis. J Exp Child Psychol 85(3):199–212. doi:10.1016/S0022-0965(03)00022-5
Gosling SD, Rentfrow PJ, Swann WB Jr (2003) A very brief measure of the big five personality domains. J Res Pers 37:504–528
Kleinke C (1986) Gaze and eye contact: a research review. Psychol Bull 100(1):78–100. doi:10.1037/0033-2909.100.1.78
Kobayashi H, Kohshima S (1997) Unique morphology of the human eye. Nature 387:767–768
Kobayashi H, Kohshima S (2001) Unique morphology of the human eye and its adaptive meaning: comparative studies on external morphology of the primate eye. J Hum Evol 40:419–435
Krohn K, Rhodes J (1965) Algebraic theory of machines. I. Prime decomposition theorem for finite semigroups and machines. Trans Am Math Soc 116:450–464
Mazur A, Rosa E, Faupel M, Heller J, Leen R, Thurman B (1980) Physiological aspects of communication via mutual gaze. Am J Sociol 86(1):50–74
Metta G, Fitzpatrick P, Natale L (2006) YARP: yet another robot platform. Int J Adv Robot Syst (special issue on software development and integration in robotics) 3(1)
Meyn SP, Tweedie RL (1993) Markov chains and stochastic stability. Springer, London
Mills DL (1994) Improved algorithms for synchronizing computer network clocks. Comput Commun Rev 24:317–327. doi:10.1145/190809.190343
Moore C, Corkum V (1998) Infant gaze following based on eye direction. Br J Dev Psychol 16(4):495–503
Mutlu B, Forlizzi J, Hodgins J (2006) A storytelling robot: modeling and evaluation of human-like gaze behavior. In: Humanoids, pp 518–523. doi:10.1109/ICHR.2006.321322
Novick D, Hansen B, Ward K (1996) Coordinating turn-taking with gaze. In: Proceedings of the fourth international conference on spoken language, ICSLP 96, vol 3, pp 1888–1891
Raidt S, Bailly G, Elisei F (2007) Analyzing and modeling gaze during face-to-face interaction. In: 7th international conference on intelligent virtual agents, IVA’2007, Paris, France, 17–19 September 2007, pp 100–101
Rhodes J (2009) Applications of automata theory and algebra via the mathematical theory of complexity to finite-state physics, biology, philosophy, and games. World Scientific, Singapore
Ristic J, Kingstone A (2005) Taking control of reflexive social attention. Cognition 94(3):B55–B65
Saito DN, Tanabe HC, Izuma K, Hayashi MJ, Morito Y, Komeda H, Uchiyama H, Kosaka H, Okazawa H, Fujibayashi Y, Sadato N (2010) Stay tuned: inter-individual neural synchronization during mutual gaze and joint attention. Front Integr Neurosci 4:127
Saunders J, Nehaniv CL, Lyon C (2010) Robot learning of lexical semantics from sensorimotor interaction and the unrestricted speech of human tutors. In: 2nd international symposium on new frontiers in HRI. AISB
Seeing Machines, Inc. (2011) faceAPI. http://seeingmachines.com/
Tomasello M, Hare B, Lehmann H, Call J (2007) Reliance on head versus eyes in the gaze following of great apes and human infants: the cooperative eye hypothesis. J Hum Evol 52:314–320
Trevarthen C, Aitken KJ (2001) Infant intersubjectivity: research, theory, and clinical applications. J Child Psychol Psychiatry Allied Discipl 42(1):3–48. doi:10.1017/S0021963001006552
Vollmer AL, Lohan KS, Fischer K, Nagai Y, Pitsch K, Fritsch J, Rohlfing KJ, Wredek B (2009) People modify their tutoring behavior in robot-directed interaction for action learning. In: Proceedings of the 2009 IEEE 8th international conference on development and learning, DEVLRN’09. IEEE Comput. Soc., Washington, pp 1–6. doi:10.1109/DEVLRN.2009.5175516
Wang N, Gratch J (2010) Don’t just stare at me! In: Proceedings of the 28th international conference on human factors in computing systems, CHI’10. ACM, New York, pp 1241–1250
Yoshikawa Y, Shinozawa K, Ishiguro H, Hagita N, Miyamoto T (2006) The effects of responsive eye movement and blinking behavior in a communication robot. In: IROS, pp 4564–4569. doi:10.1109/IROS.2006.282160
Yu C, Scheutz M, Schermerhorn P (2010) Investigating multimodal real-time patterns of joint attention in an HRI word learning task. In: 5th ACM/IEEE international conference on human-robot interaction, HRI’10. ACM, New York, pp 309–316. doi:10.1145/1734454.1734561
Acknowledgements
This research was conducted within the EU Integrated Project ITALK (Integration and Transfer of Action and Language in Robots) funded by the European Commission under contract number FP7-214668.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag London
About this chapter
Cite this chapter
Broz, F., Lehmann, H., Nehaniv, C.L., Dautenhahn, K. (2013). Automated Analysis of Mutual Gaze in Human Conversational Pairs. In: Nakano, Y., Conati, C., Bader, T. (eds) Eye Gaze in Intelligent User Interfaces. Springer, London. https://doi.org/10.1007/978-1-4471-4784-8_4
Download citation
DOI: https://doi.org/10.1007/978-1-4471-4784-8_4
Publisher Name: Springer, London
Print ISBN: 978-1-4471-4783-1
Online ISBN: 978-1-4471-4784-8
eBook Packages: Computer ScienceComputer Science (R0)