Skip to main content
SpringerLink
Log in

A Reference Architecture for Social Head Gaze Generation in Social Robotics

  • Published:
International Journal of Social Robotics Aims and scope Submit manuscript

Abstract

This article outlines a reference architecture for social head gaze generation in social robots. The architecture discussed here is grounded in human communication, based on behavioral robotics theory, and captures the commonalities, essence, and experience of 32 previous social robotics implementations of social head gaze. No such architecture currently exists, but such an architecture is needed to: (1) serve as a template for creating or re-engineering systems, (2) provide analyses and understanding of different systems, and (3) provide a common lexicon and taxonomy that facilitates communication across various communities. A constructed reference architecture and the Software Architecture Analysis Method (SAAM) are used to evaluate, improve, and re-engineer two existing head gaze system architectures (Human–Robot Collaboration architecture and Robot Behavior Toolkit architecture). SAAM shows that no existing architecture incorporated the summation of functionalities found in the 32 studies. SAAM suggests several architectural improvements so that the two existing architectures can better support adaptation to a new environment and extension of capability. The resulting reference architecture guides the implementation of social head gaze in a rescue robot for the purpose of victim management in urban search and rescue (US&R). Using the proposed reference architecture will benefit social robotics because it will simplify the principled implementations of head gaze generation and allow for comparisons between such implementations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Admoni H, Dragan A, Srinivasa SS, Scassellati B (2014) Deliberate delays during robot-to-human handovers improve compliance with gaze communication. In: Proceedings of the 2014 ACM/IEEE international conference on human-robot interaction, HRI ’14, pp 49–56. ACM, New York

  2. Admoni H, Hayes B, Feil-Seifer D, Ullman D, Scassellati B (2013) Are you looking at me? Perception of robot attention is mediated by gaze type and group size. In: Proceedings of the 8th ACM/IEEE international conference on human-robot interaction, pp 389–396. Tokyo

  3. Al Moubayed S (2012) Bringing the avatar to life: studies and developments in facial communication for virtual agents and robots

  4. Andrist S, Tan XZ, Gleicher M, Mutlu B (2014) Conversational gaze aversion for humanlike robots. In: Proceedings of the 2014 ACM/IEEE international conference on human-robot interaction, pp 25–32. ACM, New York

  5. Argyle M, Cook M (1976) Gaze and mutual gaze. Cambridge University Press, Cambridge

    Google Scholar 

  6. Arkin RC (1998) Behavior-based robotics. The MIT Press, Cambridge

    Google Scholar 

  7. Bennewitz M, Faber F, Joho D, Schreiber M, Behnke S (2005) Towards a humanoid museum guide robot that interacts with multiple persons. In: Proceedings of the IEEE/RSJ international conference on humanoid robots

  8. Bethel C, Murphy R (2010) Non-facial and non-verbal affective expression for appearance-constrained robots used in victim management. Paladyn J Behav Robot 1:219–230

    Google Scholar 

  9. Bradley M (1994) Measuring emotion: the self-assessment manikin and the semantic differential. J Behav Ther Exp Psychiatry 25(1):49–59

    Article  Google Scholar 

  10. Breazeal C, Kidd CD, Thomaz AL, Hoffman G, Berlin M (2003) Effects of nonverbal communication on efficiency and robustness in human-robot teamwork. In: Proceedings of IROS

  11. Cassell J, Torres O, Prevost S (1998) Turn taking vs. discourse structure: how best to model multimodal conversation. In: Machine conversations. Kluwer, The Hague

  12. Chovil N (1991) Discourse-oriented facial displays in conversation. Res Lang Soc Interact 25(1–4):163–194

    Article  Google Scholar 

  13. Clements PC (2002) Software architecture in practice. Ph.D. thesis, Carnegie Mellon University

  14. Cloutier R, Muller G, Verma D, Nilchiani R, Hole E, Bone M (2010) The concept of reference architectures. Syst Eng 13(1):14–27

    Google Scholar 

  15. Dautenhahn K (1999) Robots as social actors: Aurora and the case of autism. In: Proceedings of the third cognitive technology conference

  16. Dobrica L, Niemela E (2002) A survey on software architecture analysis methods. IEEE Trans Softw Eng 28(7):638–653

    Article  Google Scholar 

  17. Duncan S (1972) Some signals and rules for taking speaking turns in conversations. J Personal Soc Psychol 23(2):283–292

    Article  Google Scholar 

  18. Exline RV, Winters LC (1965) Affective relations and mutual glances in dyads. In: Affect, cognition, and personality 1(5). Springer, New York

  19. Fincannon T, Barnes L, Murphy R, Riddle D (2004) Evidence of the need for social intelligence in rescue robots. In: Proceedings of the international conference on intelligent robots and systems (IROS), vol 2, pp 1089–1095

  20. Griffin ZM (2001) Gaze durations during speech reflect word selection and phonological encoding. Cognition 82(1):B1–B14

    Article  Google Scholar 

  21. Hadar U, Steiner T, Grant E, Rose FC (1983) Kinematics of head movements accompanying speech during conversation. Hum Mov Sci 2(1):35–46

    Article  Google Scholar 

  22. Halliday M (1967) Intonation and grammar in British English. Janua linguarum: Series practica. Mouton, The Hague

    Book  Google Scholar 

  23. Hassan AE, Holt RC (2000) A reference architecture for web servers. In: Proceedings of the 7th working conference on reverse engineering, pp 150–159. IEEE, Washington, DC

  24. Heerink M, Kröse B, Evers V, Wielinga B (2010) Relating conversational expressiveness to social presence and acceptance of an assistive social robot. Virtual Real 14(1):77–84

    Article  Google Scholar 

  25. Henkel Z, Bethel C, Murphy R, Srinivasan V (2014) Evaluation of proxemic scaling functions for social robotics. IEEE Trans Hum-Mach Syst 44(3):374–385

    Article  Google Scholar 

  26. Holroyd A, Rich C, Sidner CL, Ponsler B (2011) Generating connection events for human-robot collaboration. In: 20th IEEE international workshop on robot and human interactive communication, RO-MAN, pp 24–246

  27. Holthaus P, Pitsch K, Wachsmuth S (2011) How can I help? Int J Soc Robot 3(4):383–393

    Article  Google Scholar 

  28. Huang C, Mutlu B (2012) Robot behavior toolkit: generating effective social behaviors for robots. In: Proceedings of the 7th ACM/IEEE conference on human-robot interaction

  29. Huang CM, Mutlu B (2013) The repertoire of robot behavior: designing social behaviors to support human-robot joint activity. J Hum-Robot Interact 2(2):80–102

    Article  Google Scholar 

  30. Huang CM, Mutlu B (2014) Learning-based modeling of multimodal behaviors for humanlike robots. In: Proceedings of the 2014 ACM/IEEE international conference on human-robot interaction, HRI ’14, pp 57–64. ACM, New York

  31. Imai M, Kanda T, Ono T, Ishiguro H, Mase K (2002) Robot mediated round table: analysis of the effect of robot’s gaze. In: Proceedings of 11th IEEE international workshop on robot and human interactive communication, pp 411–416

  32. Imai M, Ono T, Ishiguro H (2001) Physical relation and expression: joint attention for human-robot interaction. In: Proceedings of 10th IEEE international workshop on robot and human interactive communication, pp 512–517

  33. Ishi CT, Liu C, Ishiguro H, Hagita N (2010) Head motions during dialogue speech and nod timing control in humanoid robots. In: Proceedings of the 5th ACM/IEEE international conference on human-robot interaction, pp 293–300. IEEE Press, Piscataway

  34. Jamone L, Brandao M, Natale L, Hashimoto K, Sandini G, Takanishi A (2014) Autonomous online generation of a motor representation of the workspace for intelligent whole-body reaching. Robot Auton Syst 62(4):556–567

    Article  Google Scholar 

  35. Kazman R, Bass L, Webb M, Abowd G (1994) Saam: a method for analyzing the properties of software architectures. In: Proceedings of the 16th international conference on software engineering, pp 81–90. IEEE Computer Society Press, Los Alamitos

  36. Kendon A (1967) Some functions of gaze-direction in social interaction. Acta Psychol 26:22–63

    Article  Google Scholar 

  37. Kozima H, Nakagawa C, Yasuda Y (2005) Interactive robots for communication-care: a case-study in autism therapy. In: 14th IEEE international workshop on robot and human interactive communication (RO-MAN), pp 341–346

  38. Kuno Y, Sadazuka K, Kawashima M, Yamazaki K, Yamazaki A, Kuzuoka H (2007) Museum guide robot based on sociological interaction analysis. In: Proceedings of the SIGCHI conference on human factors in computing systems, pp 1191–1194. ACM, New York

  39. Liu C, Ishi CT, Ishiguro H, Hagita N (2012) Generation of nodding, head tilting and eye gazing for human-robot dialogue interaction. In: Proceedings of the 7th annual ACM/IEEE international conference on human-robot interaction, pp 285–292. ACM, New York

  40. MacDorman K, Minato T, Shimada M, Itakura S, Cowley S, Ishiguro H (2005) Assessing human likeness by eye contact in an android testbed. In: Proceedings of the XXVII annual meeting of the Cognitive Science Society, pp 21–23

  41. Machines S (2009) Faceapi. http://www.seeingmachines.com/product/faceapi

  42. Matsusaka Y, Fujie S, Kobayashi T (2001) Modeling of conversational strategy for the robot participating in the group conversation. In: Interspeech’01, pp 2173–2176

  43. Meyer AS, Sleiderink AM, Levelt WJ (1998) Viewing and naming objects: eye movements during noun phrase production. Cognition 66(2):B25–B33

    Article  Google Scholar 

  44. Minato T, Shimada M, Ishiguro H, Itakura S (2004) Development of an android robot for human-robot interaction. Innovations in applied artificial intelligence, pp 424–434

  45. Mitsunaga N, Smith C, Kanda T, Ishiguro H, Hagita N (2008) Adapting robot behavior for human-robot interaction. IEEE Trans Robot 24(4):911–916

    Article  Google Scholar 

  46. Moon A, Troniak DM, Gleeson B, Pan MK, Zeng M, Blumer BA, MacLean K, Croft EA (2014) Meet me where i’m gazing: how shared attention gaze affects human-robot handover timing. In: Proceedings of the 2014 ACM/IEEE international conference on human-robot interaction, HRI ’14, pp 334–341. ACM, New York

  47. Munhall KG, Jones JA, Callan DE, Kuratate T, Vatikiotis-Bateson E (2004) Visual prosody and speech intelligibility head movement improves auditory speech perception. Psychol Sci 15(2):133–137

    Article  Google Scholar 

  48. Murphy RR (2000) Introduction to AI robotics. The MIT Press, Cambridge

    Google Scholar 

  49. Mutlu B, Forlizzi J, Hodgins J (2006) A storytelling robot: modeling and evaluation of human-like gaze behavior. In: Proceedings of the international conference on humanoid robots. IEEE, Piscataway

  50. Mutlu B, Shiwa TKT, Ishiguro H, Hagita N (2009) Footing in human-robot conversations: how robots might shape participant roles using gaze cues. In: Proceedings of the 4th ACM/IEEE international conference on human robot interaction, pp 61–68. ACM, New York

  51. Mutlu B, Yamaoka F, Kanda T, Ishiguro H, Hagita N (2009) Nonverbal leakage in robots: communication of intentions through seemingly unintentional behavior. In: HRI ’09: Proceedings of the 4th ACM/IEEE international conference on human robot interaction, pp 69–76. ACM, New York

  52. Nass C, Steuer J, Tauber ER (1994) Computers are social actors. In: Proceedings of the SIGCHI conference on human factors in computing systems, pp 72–78. ACM, New York

  53. Pitsch K, Vollmer AL, Muhlig M (2013) Robot feedback shapes the tutor’s presentation how a robot’s online gaze strategies lead to micro-adaptation of the human’s conduct. Interact Stud 14(2):268–296

    Article  Google Scholar 

  54. Rich C, Ponsleur B, Holroyd A, Sidner CL (2010) Recognizing engagement in human-robot interaction. In: Proceeding of the 5th ACM/IEEE international conference on human-robot interaction, pp 375–382. ACM, New York

  55. Ruhland K, Andrist S, Badler J, Peters C, Badler N, Gleicher M, Mutlu B, McDonnell R (2014) Look me in the eyes: a survey of eye and gaze animation for virtual agents and artificial systems. In: Eurographics 2014—State of the Art Reports, pp 69–91. The Eurographics Association, Aire-la-Ville

  56. Sacks H, Schegloff EA, Jefferson G (1974) A simplest systematics for the organization of turn-taking for conversation. Language 50(4):696–735

    Article  Google Scholar 

  57. Sakamoto D, Kanda T, Ono T, Kamashima M, Imai M, Ishiguro H (2004) Cooperative embodied communication emerged by interactive humanoid robots. In: 13th IEEE international workshop on robot and human interactive communication, RO-MAN, pp 443–448

  58. Sauppé A, Mutlu B (2014) Robot deictics: how gesture and context shape referential communication. In: Proceedings of the 2014 ACM/IEEE international conference on human-robot interaction, HRI ’14, pp 342–349. ACM, New York

  59. Sciutti A, Bisio A, Nori F, Metta G, Fadiga L, Sandini G (2013) Robots can be perceived as goal-oriented agents. Interact Stud 14(3):329–350

    Article  Google Scholar 

  60. Seib V (2010) Ros object recognition stack. http://wiki.ros.org/object_recognition

  61. Shimada M, Yoshikawa Y, Asada M, Saiwaki N, Ishiguro H (2011) Effects of observing eye contact between a robot and another person. Int J Soc Robot 3:143–154

    Article  Google Scholar 

  62. Sidner CL, Lee C, Kidd CD, Lesh N, Rich C (2005) Explorations in engagement for humans and robots. Artif Intell 166:10–1016

    Article  Google Scholar 

  63. Sirkin D, Ju W, Cutkosky M (2012) Communicating meaning and role in distributed design collaboration: how crowdsourced users help inform the design of telepresence robotics. In: Design thinking research, pp 173–187. Springer, New York

  64. Srinivasan V, Bethel C, Murphy R (2014) Evaluation of head gaze loosely synchronized with real-time synthetic speech for social robots. IEEE Trans Hum-Mach Syst 44(6):767–778

    Article  Google Scholar 

  65. Srinivasan V, Murphy R, Henkel Z, Groom V, Nass C (2011) A toolkit for exploring the role of voice in human-robot interaction. In: Proceedings of the 6th international conference on human-robot interaction, HRI ’11, pp 255–256. ACM, New York

  66. Staudte M, Crocker M (2009) The effect of robot gaze on processing robot utterances. In: Proceedings of the 31th annual conference of the Cognitive Science Society. Cognitive Science Society, Amsterdam

  67. Staudte M, Crocker MW (2009) Visual attention in spoken human-robot interaction. In: HRI ’09: Proceedings of the 4th ACM/IEEE international conference on human robot interaction, pp 77–84. ACM, New York

  68. Trovato G, Zecca M, Sessa S, Jamone L, Ham J, Hashimoto K, Takanishi A (2013) Cross-cultural study on human-robot greeting interaction: acceptance and discomfort by Egyptians and Japanese. Paladyn J Behav Robot 4(2):83–93

  69. Weiss DM (1998) Commonality analysis: a systematic process for defining families. In: Development and evolution of software architectures for product families, pp 214–222. Springer, Heidelberg

  70. Yamazaki A, Yamazaki K, Kuno Y, Burdelski M, Kawashima M, Kuzuoka H (2008) Precision timing in human-robot interaction: coordination of head movement and utterance. In: Proceeding of the twenty-sixth annual SIGCHI conference on human factors in computing systems, pp 131–140. ACM, New York

Download references

Acknowledgments

This work was supported in part by NSF IIS-0905485 “The Social Medium is the Message”, RESPOND-R mobile, distributed test instrument created through NSF Grant CNS-0923203, and Microsoft External Research.

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Texas A&M University, College Station, TX, 77843, USA

    Vasant Srinivasan & Robin R. Murphy

  2. Department of Computer Science and Engineering, Mississippi State University, Mississippi State, MS, 39762, USA

    Cindy L. Bethel

Authors
  1. Vasant Srinivasan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Robin R. Murphy
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Cindy L. Bethel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vasant Srinivasan.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Srinivasan, V., Murphy, R.R. & Bethel, C.L. A Reference Architecture for Social Head Gaze Generation in Social Robotics. Int J of Soc Robotics 7, 601–616 (2015). https://doi.org/10.1007/s12369-015-0315-x

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12369-015-0315-x

Keywords

Search

Navigation