Skip to main content
SpringerLink
Log in

Cognitive Human–Robot Interaction

  • Chapter
  • First Online:
Springer Handbook of Robotics

Part of the book series: Springer Handbooks ((SHB))

Abstract

A key research challenge in robotics is to design robotic systems with the cognitive capabilities necessary to support human–robot interaction. These systems will need to have appropriate representations of the world; the task at hand; the capabilities, expectations, and actions of their human counterparts; and how their own actions might affect the world, their task, and their human partners. Cognitive human–robot interaction is a research area that considers human(s), robot(s), and their joint actions as a cognitive system and seeks to create models, algorithms, and design guidelines to enable the design of such systems. Core research activities in this area include the development of representations and actions that allow robots to participate in joint activities with people; a deeper understanding of human expectations and cognitive responses to robot actions; and, models of joint activity for human–robot interaction. This chapter surveys these research activities by drawing on research questions and advances from a wide range of fields including computer science, cognitive science, linguistics, and robotics.

figure a

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 269.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 349.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Abbreviations

2-D:

two-dimensional

BML:

behavior mark-up language

fMRI:

functional magnetic resonance imaging

FOA:

focus of attention

FOV:

field of view

HCI:

human–computer interaction

HRI/OS:

HRI operating system

HRI:

human–robot interaction

IAA:

interaction agent

IU:

interaction unit

MDP:

Markov decision process

OAA:

open agent architecture

OOF:

out of field

PaMini:

pattern-based mixed-initiative

POMDP:

partially observable Markov decision process

RSS:

Robotics Science and Systems

SRA:

spatial reasoning agent

XML:

extensible markup language

References

  1. T. Fong, C. Kunz, L.M. Hiatt, M. Bugajska: The human-robot interaction operating system, Proc. 1st ACM SIGCHI/SIGART Conf. HRI, Salt Lake City (2006) pp. 41–48

    Google Scholar 

  2. J.G. Trafton, A.C. Schultz, N.L. Cassimatis, L.M. Hiatt, D. Perzanowski, D.P. Brock, M.D. Bugajska, W. Adams: Communicating and collaborating with robotic agents. In: Cognition and Multi-Agent Interaction, ed. by R. Sun (Cambridge Univ. Press, New York 2006) pp. 252–278

    Google Scholar 

  3. C.-M. Huang, B. Mutlu: Robot behavior toolkit: Generating effective social behaviors for robots, Proc. 7th ACM/IEEE Intl. Conf. HRI, Boston (2012) pp. 25–32

    Google Scholar 

  4. A.H. Vera, H.A. Simon: Situated action: A symbolic interpretation, Cogn. Sci. 17(1), 7–48 (1993)

    Article  Google Scholar 

  5. T. Winograd, F. Flores: Understanding Computers and Cognition: A New Foundation for Design (Ablex Publ., New York 1986)

    MATH  Google Scholar 

  6. L.A. Suchman: Plans and Situated Actions: The Problem of Human-Machine Communication (Cambridge Univ. Press, Cambridge 1987)

    Google Scholar 

  7. A.N. Leont'ev: The problem of activity in psychology, J. Russ. East Eur. Psychol. 13(2), 4–33 (1974)

    Google Scholar 

  8. E. Hutchins: The social organization of distributed cognition. In: Perspectives on Socially Shared Cognition, ed. by L.B. Resnick, J.M. Levine, S.D. Teasley (American Psychological Association, Wachington, DC 1991)

    Google Scholar 

  9. B. Gates: A robot in every home, Sci. Am. 296, 58–65 (2007)

    Article  Google Scholar 

  10. C. Pantofaru, L. Takayama, T. Foote, B. Soto: Exploring the role of robots in home organization, Proc. 7th Annu. ACM/IEEE Intl. Conf. HRI, Boston (2012) pp. 327–334

    Google Scholar 

  11. K.M. Tsui, M. Desai, H.A. Yanco, C. Uhlik: Exploring use cases for telepresence robots, ACM/IEEE 6th Int. Conf. HRI, Lausanne (2011) pp. 11–18

    Google Scholar 

  12. F. Tanaka, A. Cicourel, J.R. Movellan: Socialization between toddlers and robots at an early childhood education center, Proc. Natl. Acad. Sci. USA 104(46), 17954–17958 (2007)

    Article  Google Scholar 

  13. T. Kanda, R. Sato, N. Saiwaki, H. Ishiguro: A two-month field trial in an elementary school for long-term human-robot interaction, IEEE Trans. Robotics 23(5), 962–971 (2007)

    Article  Google Scholar 

  14. B. Reeves, C. Nass: The Media Equation: How People Treat Computers, Television and New Media Like Real People and Places (Cambridge University Press, Cambridge 1996)

    Google Scholar 

  15. S. Turkle, O. Daste, C. Breazeal, B. Scassellati: Encounters with Kismet and Cog: Children respond to relational artifacts, Proc. IEEE-RAS/RSJ Int. Conf. Humanoid Robots, Los Angeles (2004) pp. 1–20

    Google Scholar 

  16. S. Turkle: Alone Together: Why We Expect More from Technology and Less from Each Other (Basic Books, New York, 2011)

    Google Scholar 

  17. C. Nass, Y. Moon: Machines and mindlessness: Social responses to computers, J. Soc. Issues 56(1), 81–103 (2000)

    Article  Google Scholar 

  18. S. Turkle: Evocative Objects: Things We Think With (MIT Press, Cambridge 2011)

    Google Scholar 

  19. K. Wada, T. Shibata, Y. Kawaguchi: Long-term robot therapy in a health service facility for the aged – A case study for 5 years, Proc. 11th IEEE Int. Conf. Rehabil. Robotics, Kyoto (2009) pp. 930–933

    Google Scholar 

  20. B.R. Duffy: Anthropomorphism and the social robot, Robotics Auton. Syst. 42(3-4), 177–190 (2003)

    Article  MATH  Google Scholar 

  21. S. Kiesler, A. Powers, S.R. Fussell, C. Torrey: Anthropomorphic interactions with a software agent and a robot, Soc. Cogn. 26(2), 168–180 (2008)

    Article  Google Scholar 

  22. S.R. Fussell, S. Kiesler, L.D. Setlock, V. Yew: How people anthropomorphize robots, Proc. 3rd ACM/IEEE Int. Conf. HRI, Amsterdam (2008) pp. 145–152

    Google Scholar 

  23. D.S. Syrdal, K. Dautenhahn, S.N. Woods, M.L. Walters, K.L. Koay: Looking good? Appearance preferences and robot personality inferences at zero acquaintance, AAAI Spring Symp.: Multidiscip. Collab. Socially Assist. Robotics, Stanford (2007) pp. 86–92

    Google Scholar 

  24. K.F. MacDorman, T. Minato, M. Shimada, S. Itakura, S. Cowley, H. Ishiguro: Assessing human likeness by eye contact in an android testbed, Proc. XXVII Annu. Meet. Conf. Cogn. Sci. Soc., Stresa (2005) pp. 1373–1378

    Google Scholar 

  25. F. Hegel, S. Krach, T. Kircher, B. Wrede, G. Sagerer: Understanding social robots: A user study on anthropomorphism, Proc. 17th IEEE Int. Symp. Robot Hum. Interact. Commun., Munich (2008) pp. 574–579

    Google Scholar 

  26. M. Mori: The uncanny valley, Energy 7(4), 33–35 (1970)

    Google Scholar 

  27. C. Bartneck, T. Kanda, H. Ishiguro, N. Hagita: My robotic Doppelganger – A critical look at the Uncanny Valley Theory, IEEE 18th Intl. Symp. Robot Hum. Interact. Commun., Toyama (2009) pp. 269–276

    Google Scholar 

  28. M.L. Walters, D.S. Syrdal, K. Dautenhahn, R. te Boekhorst, K.L. Koay: Avoiding the uncanny valley: Robot appearance, personality and consistency of behavior in an attention-seeking home scenario for a robot companion, Auton. Robots 24(2), 159–178 (2008)

    Article  Google Scholar 

  29. A.P. Saygin, T. Chaminade, H. Ishiguro, J. Driver, C. Frith: The thing that should not be: Predictive coding and the uncanny valley in perceiving human and humanoid robot actions, Soc. Cogn. Affect. Neurosci. 7(4), 413–422 (2012)

    Article  Google Scholar 

  30. W. Mitchell, K.A. Szerszen Sr., A.S. Lu, P.W. Schermerhorn, M. Scheutz, K.F. MacDorman: A mismatch in the human realism of face and voice produces an uncanny valley, i-Perception 2, 10–12 (2011)

    Article  Google Scholar 

  31. C. Breazeal: Designing Sociable Robots (MIT Press, Cambridge 2002)

    MATH  Google Scholar 

  32. N. Matsumoto, H. Fujii, M. Okada: Minimal design for human-agent communication, Artif. Life Robotics 10(1), 49–54 (2006)

    Article  Google Scholar 

  33. H. Kozima, H. Yano: A Robot that Learns to Communicate with Human Caregivers, Proc. 1st Int. Workshop Epigenetic Robotics, Lund (2001) pp. 47–52

    Google Scholar 

  34. A. Powers, A.D.I. Kramer, S. Lim, J. Kuo, S-l. Lee, S. Kiesler: Eliciting information from people with a gendered humanoid robot, IEEE 14th Int. Workshop Robot Hum. Interact. Commun., Nashville (2005) pp. 158–163

    Google Scholar 

  35. K.M. Lee, N. Park, H. Song: Can a robot be perceived as a developing creature?: Effects of a robot's long-term cognitive developments on its social presence and people's social responses toward it, Human Commun. Res. 31(4), 538–563 (2005)

    Google Scholar 

  36. J. Goetz, S. Kiesler, A. Powers: Matching robot appearance and behavior to tasks to improve human–robot cooperation, Proc. 12th IEEE Int. Workshop Robot Hum. Interact. Commun., Silicon Valley (2003) pp. 55–60

    Google Scholar 

  37. M.K. Lee, S. Kiesler, J. Forlizzi, S. Srinivasa, P. Rybski: Gracefully mitigating breakdowns in robotic services, Proc. 6th ACM/IEEE Int. Conf. HRI, Lausanne (2010) pp. 203–210

    Google Scholar 

  38. B. Shore: Culture in Mind: Cognition, Culture, and the Problem of Meaning (Oxford Univ. Press, Oxford 1996)

    Google Scholar 

  39. V. Evers, H. Maldonado, T. Brodecki, P. Hinds: Relational vs. group self-construal: Untangling the role of national culture in HRI, Proc. 3rd ACM/IEEE Int. Conf. HRI, Amsterdam (2008)

    Google Scholar 

  40. L. Wang, P.-L.P. Rau, V. Evers, B.K. Robinson, P. Hinds: When in Rome: The role of culture and context in adherence to robot recommendations, Proc. 5th ACM/IEEE Int. Conf. HRI, Osaka (2010) pp. 359–366

    Google Scholar 

  41. S. Sabanovic: Robots in society, society in robots – Mutual shaping of society and technology as a framework for social robot design, Int. J. Soc. Robotics 2(4), 439–450 (2010)

    Article  Google Scholar 

  42. G. Shaw-Garlock: Looking forward to sociable robots, Int. J. Soc. Robotics 1(3), 249–260 (2009)

    Article  Google Scholar 

  43. P.H. Kahn, A.L. Reichert, H.E. Gary, T. Kanda, H. Ishiguro, S. Shen, J.H. Ruckert, B. Gill: The new ontological category hypothesis in human–robot interaction, Proc. 6th ACM/IEEE Int. Conf. HRI, Lausanne (2011) pp. 159–160

    Google Scholar 

  44. P.H. Kahn, N.G. Freier, B. Friedman, R.L. Severson, E.N. Feldman: Social and moral relationships with robotic others?, IEEE 13th Int. Workshop Robot Hum. Interact. Commun., Kurashiki (2004) pp. 545–550

    Google Scholar 

  45. S. Turkle: A Nascent Robotics Culture: New Complicities for Companionship (AAAI, Boston 2006)

    Google Scholar 

  46. B. Scassellati: How developmental psychology and robotics complement each other, NSF/DARPA Workshop Dev. Learn. (MIT Press, CSAIL, Cambridge 2006)

    Google Scholar 

  47. H. Ishiguro: Android science – toward a new cross-interdisciplinary framework, ICCS/CogSci Workshop Toward Soc. Mech. Android Sci., Stresa (2005) pp. 1–6

    Google Scholar 

  48. K.F. MacDorman, H. Ishiguro: The uncanny advantage of using androids in cognitive and social science research, Interact. Stud. 7(3), 297–337 (2006)

    Article  Google Scholar 

  49. M. Stanley, J. Sabini: On maintaining social norms: A field experiment in the subway. In: Advances in Environmental Psychology: The Urban Environment, ed. by A. Baum, J.E. Singer, S. Valins (Erlbaum Associates, Hillsdale 1978) pp. 31–40

    Google Scholar 

  50. H. Kozima, M.P. Michalowski, C. Nakagawa: Keepon: A playful robot for research, therapy, and entertainment, Int. J. Soc. Robotics 1(1), 3–18 (2009)

    Article  Google Scholar 

  51. K.F. MacDorman: Introduction to the special issue on android science, Connect. Sci. 18(4), 313–317 (2006)

    Article  Google Scholar 

  52. J.J. Gibson: The Ecological Approach to Visual Perception (Houghton Mifflin, Boston 1979)

    Google Scholar 

  53. E.S. Reed: Encountering the World: Toward an Ecological Psychology (Oxford Univ. Press, Oxford 1996)

    Google Scholar 

  54. Y. Yamaji, T. Miyake, Y. Yoshiike, P.R.S. De Silva, M. Okada: STB: Human-dependent sociable trash box, Proc. 5th ACM/IEEE Int. Conf. HRI, Osaka (2010) pp. 197–198

    Google Scholar 

  55. H. Ishiguro: Android science: Conscious and subconscious recognition, Connect. Sci. 18(4), 319–332 (2006)

    Article  Google Scholar 

  56. S. Nishio, H. Ishiguro, N. Hagita: Geminoid: Teleoperated android of an existing person. In: Humanoid Robots, New Developments, ed. by A.C. De Pina Filho (InTech, Vienna 2007) pp. 343–352

    Google Scholar 

  57. S. Nishio, H. Ishiguro, N. Hagita: Can a teleoperated robot represent personal presence? – A case study with children, Psychologia 50(4), 330–342 (2007)

    Article  Google Scholar 

  58. M. Shimada, K. Yamauchi, T. Minato, H. Ishiguro, S. Itakura: Studying the influence of the chameleon effect on humans using an android, IEEE/RSJ Int. Conf. Intell. Robots Syst. (IROS), Nice (2008)

    Google Scholar 

  59. J. Wainer, D.J. Feil-Seifer, D.A. Shell, M.J. Mataric: Embodiment and human-robot interaction: A taskbased perspective, Proc. 2nd ACM/IEEE Int. Conf. HRI, Washington (2007) pp. 872–877

    Google Scholar 

  60. P. Schermerhorn, M. Scheutz: Disentangling the effects of robot affect, embodiment, and autonomy on human team members in a mixed-initiative task, Proc. 4th Int. Conf. Adv. Comput.–Hum. Interact., Gosier (2011) pp. 235–241

    Google Scholar 

  61. W.A. Bainbridge, J.W. Hart, E.S. Kim, B. Scassellati: The benefits of interactions with physically present robots over video-displayed agents, Int. J. Soc. Robotics 1(2), 41–52 (2010)

    Google Scholar 

  62. B. Mutlu: Designing embodied cues for dialog with robots, AI Magazine 32(4), 17–30 (2011)

    Article  Google Scholar 

  63. E.T. Hall: The Hidden Dimension (Anchor Books, New York 1966)

    Google Scholar 

  64. L. Takayama, C. Pantofaru: Influences on proxemic behaviors in human–robot interaction, IEEE/RSJ Int. Conf. Intell. Robots Syst. (IROS), St. Louis (2009) pp. 5495–5502

    Google Scholar 

  65. D.S. Syrdal, K.L. Koay, M.L. Walters, K. Dautenhahn: A personalized robot companion? The role of individual differences on spatial preferences in HRI scenarios, Proc. 16th IEEE Int. Symp. Robot Hum. Interact. Commun., Jeju Island (2007) pp. 1143–1148

    Google Scholar 

  66. J. Mumm, B. Mutlu: Human-robot proxemics: Physical and psychological distancing in human-robot interaction, Proc. 6th ACM/IEEE Int. Conf. HRI, Lausanne (2011) pp. 331–338

    Google Scholar 

  67. M.L. Walters, M.A. Oskoei, D.S. Syrdal, K. Dautenhahn: A long-term human–robot proxemic study, Proc. 20th IEEE Int. Symp. Robot Hum. Interact. Commun., Atlanta (2011) pp. 137–142

    Google Scholar 

  68. C. Yu, M. Scheutz, P. Schermerhorn: Investigating multimodal real-time patterns of joint attention in an HRI word learning task, Proc. 5th ACM/IEEE Int. Conf. HRI, Osaka (2010) pp. 309–316

    Google Scholar 

  69. T. Yonezawa, H. Yamazoe, A. Utsumi, S. Abe: Gaze-communicative behavior of stuffed-toy robot with joint attention and eye contact based on ambient gaze-tracking, Proc. 9th Int. Conf. Multimodal Interfaces, Nagoya (2007) pp. 140–145

    Chapter  Google Scholar 

  70. Y. Yoshikawa, K. Shinozawa, H. Ishiguro, N. Hagita, T. Miyamoto: Responsive robot gaze to interaction partner, Robotics Sci. Syst., Philadelphia (2006)

    Google Scholar 

  71. B. Mutlu, T. Shiwa, T. Kanda, H. Ishiguro, N. Hagita: Footing in human-robot conversations: How robots might shape participant roles using gaze cues, Proc. 4th ACM/IEEE Int. Conf. HRI, San Diego, California (2009) pp. 61–68

    Google Scholar 

  72. M. Staudte, M.W. Crocker: Visual attention in spoken human-robot interaction, Proc. 4th ACM/IEEE Int. Conf. HRI, San Diego (2009) pp. 77–84

    Google Scholar 

  73. B. Mutlu, J. Forlizzi, J.K. Hodgins: A storytelling robot: Modeling and evaluation of human-like gaze behavior, IEEE-RAS Conf. Humanoid Robots, Genoa (2006) pp. 518–523

    Google Scholar 

  74. C. Yu, P. Schermerhorn, M. Scheutz: Adaptive eye gaze patterns in interactions with human and artificial agents, ACM Trans. Interact. Intell. Syst. 1(2), 1–25 (2012)

    Article  Google Scholar 

  75. H. Admoni, C. Bank, J. Tan, M. Toneva, B. Scassellati: Robot gaze does not reflexively cue human attention, Proc. 33rd Annu. Conf. Cogn. Sci. Soc., Boston (2011) pp. 1983–1988

    Google Scholar 

  76. W.S. Condon: Cultural microrhythms. In: Interaction Rhythms: Periodicity in Communicative Behavior, ed. by M. Davis (Human Sciences Press, New York 1982) pp. 53–76

    Google Scholar 

  77. E. Goffman: Some context for content analysis: A view of the origins of structural studies of face-to-face interaction. In: Conducting Interaction: Patterns of Behavior in Focused Encounters, ed. by A. Kendon (Cambridge Univ. Press, Cambridge 1990) pp. 15–49

    Google Scholar 

  78. C. Trevarthen: Can a robot hear music? Can a robot dance? Can a robot tell what it knows or intends to do? Can it feel pride or shame in company? – Questions of the nature of human vitality, Proc. 2nd Int. Workshop Epigenet. Robotics, Edinburgh (2002)

    Google Scholar 

  79. M. Michalowski, S. Sabanovic, H. Kozima: A dancing robot for rhythmic social interaction, Proc. 2nd ACM/IEEE Int. Conf. HRI, Washington DC (2007) pp. 89–96

    Google Scholar 

  80. M.P. Michalowski, R. Simmons, H. Kozima: Rhythmic attention in child-robot dance play, Proc. 18th IEEE Int. Symp. Robot Hum. Interact. Commun., Toyama (2009) pp. 816–821

    Google Scholar 

  81. E. Avrunin, J. Hart, A. Douglas, B. Scassellati: Effects related to synchrony and repertoire in perceptions of robot dance, Proc. 6th ACM/IEEE Int. Conf. HRI, Lausanne (2011) pp. 93–100

    Google Scholar 

  82. G. Hoffman, C. Breazeal: Anticipatory perceptual simulation for human-robot joint practice: Theory and application study, Proc. 23rd AAAI Conf. Artif. Intell., Chicago (2008) pp. 1357–1362

    Google Scholar 

  83. G. Hoffman, G. Weinberg: Interactive improvisation with a robotic marimba player, Auton. Robots 31(2-3), 133–153 (2011)

    Article  Google Scholar 

  84. G. Deàk, I. Fasel, J. Movellan: The emergence of shared attention: Using robots to test developmental theories, Proc. 1st Int. Workshop Epigenet. Robotics, Lund (2001) pp. 95–104

    Google Scholar 

  85. K. Dautenhahn: Roles and functions of robots in human society: Implications from research in autism therapy, Robotica 21(4), 443–452 (2003)

    Article  Google Scholar 

  86. H. Kozima, C. Nakagawa, Y. Yasuda: Wowing together: What facilitates social interactions in children with autistic spectrum disorders, Proc. 6th Int. Workshop Epigenet. Robotics Model. Cogn. Dev. Robotics Syst., Paris (2006) p. 177

    Google Scholar 

  87. B. Scassellati: How social robots will help us to diagnose, treat, and understand autism, Proc. 12th Int. Symp. Robotics Res., San Francisco, ed. by S. Thrun, R.A. Brooks, H. Durrant-Whyte (Springer, Berlin, Heidelberg 2005) pp. 552–563

    Google Scholar 

  88. D.J. Feil-Seifer, M.J. Mataric: B3IA: An architecture for autonomous robot-assisted behavior intervention for children with autism spectrum disorders, Proc. 17th IEEE Int. Workshop Robot Hum. Interact. Commun., Munich (2008) pp. 328–333

    Google Scholar 

  89. H. Kozima, C. Nakagawa, Y. Yasuda: Interactive robots for communication-care: A case-study in autism therapy, Proc. 14th IEEE Int. Workshop Robot Hum. Interact. Commun., Nashville (2005) pp. 341–346

    Google Scholar 

  90. H. Kozima, Y. Yasuda, C. Nakagawa: Social interaction facilitated by a minimally-designed robot: Findings from longitudinal therapeutic practices for autistic children, Proc. 16th IEEE Int. Symp. Robot Hum. interact. Commun., Jeju Island (2007) pp. 599–604

    Google Scholar 

  91. H.A. Simon: The Sciences of the Artificial (MIT Press, Cambridge 1969)

    Google Scholar 

  92. B. Adams, C.L. Breazeal, R.A. Brooks, B. Scassellati: Humanoid robots: A new kind of tool, IEEE Intell. Syst. Appl. 15(4), 25–31 (2000)

    Article  Google Scholar 

  93. L.W. Barsalou, C. Breazeal, L.B. Smith: Cognition as coordinated non-cognition, Cogn. Process. 8(2), 79–91 (2007)

    Article  Google Scholar 

  94. A.L. Thomaz, M. Berlin, C. Breazeal: An embodied computational model of social referencing, Proc. 14th IEEE Int. Workshop Robot Hum. Interact. Commun., Nashville (2005) pp. 591–598

    Google Scholar 

  95. G. Hoffman, C. Breazeal: Robotic partners? Bodies and minds: An embodied approach to fluid human-robot collaboration, Proc. 5th Int. Workshop Cogn. Robotics, Boston (2006) pp. 95–102

    Google Scholar 

  96. G. Hoffman: Effects of anticipatory action on human-robot teamwork efficiency, fluency, and perception of team, Proc. 2nd ACM/IEEE Int. Conf. HRI, Washington D.C. (2007) pp. 1–8

    Google Scholar 

  97. Y. Demiris, A. Meltzoff: The robot in the crib: A developmental analysis of imitation skills in infants and robots, Infant Child Dev. 17(1), 43–53 (2008)

    Article  Google Scholar 

  98. C. Nehaniv, K. Dautenhahn (Eds.): Imitation and Social Learning in Robots, Humans and Animals: Behavioural, Social and Communicative Dimensions (Cambridge Univ. Press, Cambridge 2009)

    Google Scholar 

  99. B. Scassellati: Imitation and mechanisms of joint attention: A developmental structure for building social skills on a humanoid robot, Lect. Notes Comput. Sci. 1562, 176–195 (1999)

    Article  Google Scholar 

  100. Y. Nagai, K. Hosoda, A. Morita, M. Asada: A constructive model for the development of joint attention, Connect. Sci. 15(4), 211–229 (2003)

    Article  Google Scholar 

  101. F. Kaplan, V. Hafner: The challenges of joint attention, Proc. 4th Int. Workshop Epigenet. Robotics, Lund (2004) pp. 67–74

    Google Scholar 

  102. C. Crick, M. Munz, B. Scassellati: Synchronization in social tasks: Robotic drumming, Proc. 15th IEEE Int. Workshop Robot Hum. Interact. Commun., Hatfield (2006) pp. 97–102

    Google Scholar 

  103. P. Bakker, Y. Kuniyoshi: Robot see, robot do: An overview of robot imitation, AISB-96 Workshop Learn. Robots Animals, Brighton (1996) pp. 3–11

    Google Scholar 

  104. R.S. Jackendoff: On beyond zebra: The relation of linguistic and visual information, Cognition 26, 89–114 (1987)

    Article  Google Scholar 

  105. B. Landau, R.S. Jackendoff: What and where in spatial language and spatial cognition, Behav. Brain Sci. 16, 217–265 (1993)

    Article  Google Scholar 

  106. L. Talmy: The fundamental system of spatial schemas in language. In: From Perception to Meaning: Image Schemas in Cognitive Linguistics, ed. by B. Hamp (Mouton de Gruyter, Berlin 2005)

    Google Scholar 

  107. T.P. Regier: The Acquisition of Lexical Semantics for Spatial Terms: A Connectionist Model of Perceptual Categorization, Ph.D. Thesis (University of California at Berkeley, Berkeley 1992)

    Google Scholar 

  108. J.D. Kelleher, F.J. Costello: Applying computational models of spatial prepositions to visually situated dialog, Comput. Linguist. 35(2), 271–306 (2008)

    Article  Google Scholar 

  109. T.P. Regier, L.A. Carlson: Grounding spatial language in perception: An empirical and computational investigation, J. Exp. Psychol. 130(2), 273–298 (2001)

    Article  Google Scholar 

  110. G. Bugmann, E. Klein, S. Lauria, T. Kyriacou: Corpus-based robotics: A route instruction example, Proc. 8th Conf. Intell. Auton. Syst. (IAS-8), Amsterdam (2004) pp. 96–103

    Google Scholar 

  111. M. Levit, D. Roy: Interpretation of spatial language in a map navigation task, IEEE Trans. Syst. Man Cybern. B 37(3), 667–679 (2007)

    Article  Google Scholar 

  112. M. MacMahon, B. Stankiewicz, B. Kuipers: Walk the talk: Connecting language, knowledge, and action in route instructions, Proc. Natl. Conf. Artif. Intell., Boston (2006) pp. 1475–1482

    Google Scholar 

  113. H. Kress-Gazit, G.E. Fainekos: Translating structured English to robot controllers, Adv. Robotics 22, 1343–1359 (2008)

    Article  Google Scholar 

  114. C. Matuszek, D. Fox, K. Koscher: Following directions using statistical machine translation, Proc. 5th ACM/IEEE Int. Conf. HRI, Nara (2010) pp. 251–258

    Google Scholar 

  115. A. Vogel, D. Jurafsky: Learning to follow navigational directions, Proc. 48th Annu. Meet. Assoc. Comput. Linguist., Uppsala (2010) pp. 806–814

    Google Scholar 

  116. S. Harnad: The symbol grounding problem, Physica D 43, 335–346 (1990)

    Article  MathSciNet  Google Scholar 

  117. T. Winograd: Procedures as a Representation for Data in a Computer Program for Understanding Natural Language, MIT Tech. Rep. TMAC-TR-84 (MIT, Cambridge 1971)

    Google Scholar 

  118. K.Y. Hsiao, N. Mavridis, D. Roy: Coupling perception and simulation: Steps towards conversational robotics, Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst. (IROS), Las Vegas (2003) pp. 928–933

    Google Scholar 

  119. D. Roy, K.Y. Hsiao, N. Mavridis: Conversational Robots: Building blocks for grounding word meanings, Proc. HLT-NAACL 2003 Workshop Learn. Word Mean. Non-Linguist. Data, Stroudsburg (2003) pp. 70–77

    Google Scholar 

  120. D. Roy: Semiotic schemas: A framework for grounding language in action and perception, Artif. Intell. 167(1-2), 170–205 (2005)

    Article  Google Scholar 

  121. J. Dzifcak, M. Scheutz, C. Baral, P. Schermerhorn: What to do and how to do it: Translating natural language directives into temporal and dynamic logic representation for goal management and action execution, IEEE Int. Conf. Robotics Autom. (ICRA), Kobe (2009) pp. 4163–4168

    Google Scholar 

  122. Y. Sugita, J. Tani: Learning semantic combinatoriality from the interaction between linguistic and behavioral processes, Adapt. Behav. – Animals Animat. Softw. Agents Robots Adapt. Syst. 13(1), 33–52 (2005)

    Google Scholar 

  123. J. Modayil, B. Kuipers: Autonomous development of a grounded object ontology by a learning robot, Proc. 22nd AAAI Conf. Artif. Intell., Vancouver (2007) pp. 1095–1101

    Google Scholar 

  124. D. Marocco, A. Cangelosi, K. Fischer, T. Belpaeme: Grounding action words in the sensorimotor interaction with the world: Experiments with a simulated iCub humanoid robot, Front. Neurorobotics 4, 1–15 (2010)

    Google Scholar 

  125. R. Ge, R.J. Mooney: A statistical semantic parser that integrates syntax and semantics, Proc. 9th Conf. Comput. Nat. Lang. Learn., Ann Arbor (2005) pp. 9–16

    Google Scholar 

  126. N. Shimizu, A. Haas: Learning to follow navigational route instructions, Proc. 21st Int. Jt. Conf. Artif. Intell., Pasadena (2009) pp. 1488–1493

    Google Scholar 

  127. S.R.K. Branavan, H. Chen, L.S. Zettlemoyer, R. Barzilay: Reinforcement learning for mapping instructions to actions, Proc. 47th Jt. Conf. Annu. Meet. Assoc. Comput. Linguist. 4th Int. Jt. Conf. Nat. Lang. Process. (AFNLP), Singapore (2009) pp. 82–90

    Google Scholar 

  128. S.R.K. Branavan, D. Silver, R. Barzilay: Learning to win by reading manuals in a Monte-Carlo framework, Proc. 49th Annu. Meet. Assoc. Comput. Linguist. Hum. Lang. Technol., Portland (2011)

    Google Scholar 

  129. T. Kollar, S. Tellex, D. Roy, N. Roy: Toward understanding natural language directions, Proc. 5th ACM/IEEE Int. Conf. HRI, Osaka (2010) pp. 259–266

    Google Scholar 

  130. D. Bailey: When Push Comes to Shove: A Computational Model of the Role of Motor Control in the Acquisition of Action Verbs, Ph.D. Thesis (Univ. of California, Berkeley 1997)

    Google Scholar 

  131. T. Kollar, S. Tellex, D. Roy, N. Roy: Grounding verbs of motion in natural language commands to robots, Proc. Int. Symp. Exp. Robotics, New Delhi (2010) pp. 31–47

    Google Scholar 

  132. S. Tellex, T. Kollar, S. Dickerson, M.R. Walter, A.G. Banerjee, S. Teller, N. Roy: Understanding natural language commands for robotic navigation and mobile manipulation, Proc. Natl. Conf. Artif. Intell., San Francisco (2011)

    Google Scholar 

  133. J.L. Burke, R.R. Murphy, M.D. Coovert, D.L. Riddle: Moonlight in Miami: Field study of human-robot interaction in the context of an urban search and rescue disaster response training exercise, Hum.–Comput. Interact. 19(1/2), 85–116 (2004)

    Article  Google Scholar 

  134. K. Stubbs, P.J. Hinds, D. Wettergreen: Autonomy and common ground in human-robot interaction: A field study, IEEE Intell. Syst. 22(2), 42–50 (2007)

    Article  Google Scholar 

  135. S. Kiesler: Fostering common ground in human-robot interaction, Proc. 14th IEEE Int. Workshop Robot Hum. Interact. Commun., Nashville (2005) pp. 729–734

    Google Scholar 

  136. T. Fong, C. Thorpe, C. Baur: Collaboration, dialogue, human-robot interaction, Robotics Res. 6, 255–266 (2003)

    Article  Google Scholar 

  137. M.E. Foster, T. By, M. Rickert, A. Knoll: Human-robot dialogue for joint construction tasks, Proc. 8th Int. Conf. Mulltimodal Interfaces, Banff (2006) pp. 68–71

    Google Scholar 

  138. S. Li, B. Wrede, G. Sagerer: A computational model of multi-modal grounding for human robot interaction, Proc. 7th SIGdial Workshop Discourse Dialogue, Sydney (2009) pp. 153–160

    Google Scholar 

  139. P.R. Cohen, H.J. Levesque: Teamwork, Nous 25(4), 487–512 (1991)

    Article  Google Scholar 

  140. J. Peltason, B. Wrede: Pamini: A framework for assembling mixed-initiative human-robot interaction from generic interaction patterns, Proc. 11th SIGdial Annu. Meet. Special Interest Group Discourse Dialogue, Tokyo (2010) pp. 229–232

    Google Scholar 

  141. J. Peltason, B. Wrede: The curious robot as a case-study for comparing dialog systems, AI Magazine 32(4), 85–99 (2011)

    Article  Google Scholar 

  142. M.F. Schober: Spatial perspective-taking in conversation, Cognition 47(1), 1–24 (1993)

    Article  Google Scholar 

  143. J.E. Hanna, M.K. Tanenhaus, J.C. Trueswell: The effects of common ground and perspective on domains of referential interpretation, J. Mem. Lang. 49(1), 43–61 (2003)

    Article  Google Scholar 

  144. J.G. Trafton, N.L. Cassimatis, M.D. Bugajska, D.P. Brock, F.E. Mintz, A.C. Schultz: Enabling effective human–robot interaction using perspective-taking in robots, IEEE Trans. Syst. Man Cybern. A 35(4), 460–470 (2005)

    Article  Google Scholar 

  145. M. Berlin, J. Gray, A.L. Thomaz, C. Breazeal: Perspective taking: An organizing principle for learning in human–robot interaction, Proc. 21st Natl. Conf. Artif. Intell., Boston (2006) p. 1444

    Google Scholar 

  146. R. Moratz, K. Fischer, T. Tenbrink: Cognitive modeling of spatial reference for human-robot interaction, Int. J. Artif. Intell. Tools 10(04), 589–611 (2001)

    Article  Google Scholar 

  147. R. Ros, S. Lemaignan, E.A. Sisbot, R. Alami, J. Steinwender, K. Hamann, F. Warneken: Which one? Grounding the referent based on efficient human-robot interaction, Proc. 19th IEEE Int. Symp. Robot Hum. Interact. Commun., Viareggio (2010) pp. 570–575

    Google Scholar 

  148. A. Holroyd, C. Rich, C.L. Sidner, B. Ponsler: Generating connection events for human-robot collaboration, Proc. 20th IEEE Int. Symp. Robot Hum. Interact. Commun., Atlanta (2011) pp. 241–246

    Google Scholar 

  149. G. Butterworth, L. Grover: Joint visual attention, manual pointing, and preverbal communication in human infancy. In: Attention and Performance, Vol. 13: Motor Representation and Control, ed. by M. Jeannerod (Lawrence Erlbaum Assoc., Mahwah 1990) pp. 605–624

    Google Scholar 

  150. C. Rich, P. Ponsler, A. Holroyd, C.L. Sidner: Recognizing engagement in human-robot interaction, Proc. 5th ACM/IEEE Int. Conf. HRI, Osaka (2010) pp. 375–382

    Google Scholar 

  151. V. Gallese, A. Goldman: Mirror neurons and the simulation theory of mind-reading, Trends Cogn. Sci. 2(12), 493–501 (1998)

    Article  Google Scholar 

  152. E. Bicho, W. Erlhagen, L. Louro, E. Costa e Silva: Neuro-cognitive mechanisms of decision making in joint action: A human–robot interaction study, Hum. Mov. Sci. 30(5), 846–868 (2011)

    Article  Google Scholar 

  153. J. Gray, C. Breazeal, M. Berlin, A. Brooks, J. Lieberman: Action parsing and goal inference using self as simulator, Proc. 14th IEEE Int. Workshop Robot Hum. Interact. Commun., Nashville (2005) pp. 202–209

    Google Scholar 

  154. M.N. Nicolescu, M.J. Mataric: Linking perception and action in a control architecture for human-robot domains, Proc. 36th Annu. Hawaii Int. Conf. Syst. Sci., Big Island (2003) pp. 10–20

    Google Scholar 

  155. C. Breazeal, G. Hoffman, A. Lockerd: Teaching and working with robots as a collaboration, Proc. 3rd Int. Jt. Conf. Auton. Agents Multiagent Syst., New York, Vol. 3 (2004) pp. 1030–1037

    Google Scholar 

  156. R. Alami, A. Clodic, V. Montreuil, E.A. Sisbot, R. Chatila: Task planning for human–robot interaction, Proc. 2005 Jt. Conf. Smart Obj. Ambient Intell. Innov. Context-Aware Serv. Usages Technol., Grenoble (2005) pp. 81–85

    Google Scholar 

  157. R. Alami, A. Clodic, V. Montreuil, E.A. Sisbot, R. Chatila: Toward human-aware robot task planning, AAAI Spring Symp.: To Boldly Go where No Human-Robot Team Has Gone Before, Palo Alto (2006) pp. 39–46

    Google Scholar 

  158. R. Bellman: Dynamic Programming (Princeton Univ. Press, Princeton 1957)

    MATH  Google Scholar 

  159. N. Roy, J. Pineau, S. Thrun: Spoken dialog management for robots, Proc. Assoc. Comput. Linguist., Hong Kong (2000) pp. 93–100

    Google Scholar 

  160. J. Hoey, P. Poupart, C. Boutilier, A. Mihailidis: POMDP models for assistive technology, Proc. AAAI Fall Symp. Caring Mach., AI in Eldercare (2005)

    Google Scholar 

  161. J. Williams, S. Young: Scaling up POMDPs for dialogue management: The summary POMDP method, Proc. IEEE Autom. Speech Recognit. Underst. Workshop, Cancun (2005)

    Google Scholar 

  162. D. Litman, S. Singh, M. Kearns, M. Walker: NJFun: A reinforcement learning spoken dialogue system, Proc. ANLP/NAACL 2000 Workshop Conversat. Syst., Seattle (2000) pp. 17–20

    Chapter  Google Scholar 

  163. F. Broz, I. Nourbakhsh, R. Simmons: Planning for human-robot interaction using time-state aggregated POMDPs, Proc. 23rd Conf. Artif. Intell., Chicago (2008) pp. 1339–1344

    Google Scholar 

  164. F. Doshi, N. Roy: The permutable POMDP: Fast solutions to POMDPs for preference elicitation, Proc. 7th Int. Conf. Auton. Agents Multiagent Syst., Estoril (2008) pp. 493–500

    Google Scholar 

  165. R. Wilcox, S. Nikolaidis, J. Shah: Optimization of temporal dynamics for adaptive human-robot interaction in assembly manufacturing, Proc. Robotics Sci. Syst., Sydney (2012) p. 441

    Google Scholar 

  166. T. Prommer, H. Holzapfel, A. Waibel: Rapid simulation-driven reinforcement learning of multimodal dialog strategies in human–robot interaction, 9th Int. Conf. Spoken Lang. Process., Pittsburgh (2006)

    Google Scholar 

  167. J.M. Porta, N. Vlassis, M. Spaan, P. Poupart: Point-based value iteration for continuous POMDP, J. Mach. Learn. Res. 7, 2329–2367 (2006)

    MathSciNet  MATH  Google Scholar 

  168. F. Doshi, N. Roy: Efficient model learning for dialog management, Proc. 2nd ACM/IEEE Int. Conf. HRI, Arlington (2007) pp. 65–72

    Google Scholar 

  169. F. Doshi, N. Roy: Spoken language interaction with model uncertainty: An adaptive human-robot interaction system, Connect. Sci. 20(4), 299–319 (2008)

    Article  Google Scholar 

  170. M. Cakmak, A.L. Thomaz: Designing robot learners that ask good questions, Proc. 7th Annu. ACM/IEEE Int. Conf. HRI, Boston (2012) pp. 17–24

    Google Scholar 

  171. M.A. Goodrich, D.R. Olsen: Seven principles of efficient human robot interaction, IEEE Int. Conf. Syst. Man Cybern., Washington D.C. (2003) pp. 3942–3948

    Google Scholar 

  172. J.W. Crandall, M.A. Goodrich, D.R. Olsen, C.W. Nielsen: Validating human-robot interaction schemes in multitasking environments, IEEE Trans. Syst. Man Cybern. A 35(4), 438–449 (2005)

    Article  Google Scholar 

  173. R. Sun (Ed.): Cognition and Multiagent Interaction: From Cognitive Modeling to Social Simulation (Cambridge Univ. Press, Cambridge 2005)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Sciences, University of Wisconsin–Madison, 1210 West Dayton Street, WI 53706, Madison, USA

    Bilge Mutlu

  2. Department of Aeronautics and Astronautics, Massachusetts Institute of Technology, 77 Massachusetts Avenue 33-315, MA 02139, Cambridge, USA

    Nicholas Roy

  3. School of Informatics and Computing, Indiana University Bloomington, 919 East 10th Street, IN 47408, Bloomington, USA

    Selma Šabanović

Authors
  1. Bilge Mutlu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nicholas Roy
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Selma Šabanović
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bilge Mutlu .

Editor information

Editors and Affiliations

  1. Department of Electrical Engineering, University of Naples Federico II, Via Claudio 21, 80125, Naples, Italy

    Bruno Siciliano

  2. Department of Computer Sciences, Artificial Intelligence Laboratory, Stanford University, 450 Serra Mall, CA 94305, Stanford, USA

    Oussama Khatib

Video-References

Video-References

:

Gaze and gesture cues for robots available from http://handbookofrobotics.org/view-chapter/71/videodetails/128

:

Robotic secrets revealed, Episode 1 available from http://handbookofrobotics.org/view-chapter/71/videodetails/129

:

Robotic secrets revealed, Episode 2: The trouble begins available from http://handbookofrobotics.org/view-chapter/71/videodetails/130

:

Human-robot jazz improvization available from http://handbookofrobotics.org/view-chapter/71/videodetails/236

:

Designing robot learners that ask good questions available from http://handbookofrobotics.org/view-chapter/71/videodetails/237

:

Active keyframe-based learning from demonstration available from http://handbookofrobotics.org/view-chapter/71/videodetails/238

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Mutlu, B., Roy, N., Šabanović, S. (2016). Cognitive Human–Robot Interaction. In: Siciliano, B., Khatib, O. (eds) Springer Handbook of Robotics. Springer Handbooks. Springer, Cham. https://doi.org/10.1007/978-3-319-32552-1_71

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-32552-1_71

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-32550-7

  • Online ISBN: 978-3-319-32552-1

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation