Skip to main content
SpringerLink
Log in

A Model for Generating Socially-Appropriate Deictic Behaviors Towards People

  • Published:
International Journal of Social Robotics Aims and scope Submit manuscript

Abstract

Pointing behaviors are essential in enabling social robots to communicate about a particular object, person, or space. Yet, pointing to a person can be considered rude in many cultures, and as robots collaborate with humans in increasingly diverse environments, they will need to effectively refer to people in a socially-appropriate way. We confirmed in an empirical study that although people would point precisely to an object to indicate where it is, they were reluctant to do so when pointing to another person. We propose a model for selecting utterances and pointing behaviors towards people in terms of a balance between understandability and social appropriateness. Calibrating our proposed model based on empirical human behavior, we developed a system able to autonomously select among six deictic behaviors and execute them on a humanoid robot. We evaluated the system in an experiment in a shopping mall, and the results show that the robot’s deictic behavior was perceived by both the listener and the referent as more polite, more natural, and better overall when using our model, as compared with a model considering understandability alone.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. This paper is an extended version of our conference paper [1] with integrated technical details, additional discussions, expanded explanations, and supplementary analysis of the experiment.

  2. ATRacker is a product of ATR Promotions: http://www.atr-p.com/products/HumanTracker.html.

References

  1. Liu P, Glas DF, Kanda T, Ishiguro H, Hagita N (2013) It’s not polite to point: generating socially-appropriate deictic behaviors towards people. In: 8th ACM/IEEE international conference on human–robot interaction, pp 267–274

  2. Bennewitz M, Faber F, Joho D, Schreiber M, Behnke S (2005) Towards a humanoid museum guide robot that interacts with multiple persons. In: 2005 5th IEEE-RAS international conference on humanoid robots, pp 418–423

  3. Shiomi M, Kanda T, Ishiguro H, Hagita N (2007) Interactive humanoid robots for a science museum. IEEE Intell Syst 22:25–32

    Article  Google Scholar 

  4. Nieuwenhuisen M, Behnke S (2013) Human-like interaction skills for the mobile communication robot robotinho. Int J Soc Robot 5:549–561

    Article  Google Scholar 

  5. Kanda T, Sato R, Saiwaki N, Ishiguro H (2007) A two-month field trial in an elementary school for long-term human–robot interaction. IEEE Trans Robot 23:962–971

    Article  Google Scholar 

  6. Berns K, Mehdi SA (2010) Use of an autonomous mobile robot for elderly care. In: Advanced technologies for enhancing quality of life (AT-EQUAL), pp 121–126

  7. Sabelli AM, Kanda T, Hagita N (2011) A conversational robot in an elderly care center: an ethnographic study. In: 2011 6th ACM/IEEE international conference on Human–robot interaction (HRI), pp 37–44

  8. Semwal VB, Katiyar SA, Chakraborty R, Nandi G (2015) Biologically-inspired push recovery capable bipedal locomotion modeling through hybrid automata. Robot Auton Syst 70:181–190

    Article  Google Scholar 

  9. Sugiyama O, Kanda T, Imai M, Ishiguro H, Hagita N, Anzai Y (2006) Humanlike conversation with gestures and verbal cues based on a three-layer attention-drawing model. Connect Sci 18:379–402

    Article  Google Scholar 

  10. Schmidt J, Hofemann N, Haasch A, Fritsch J, Sagerer G (2008) Interacting with a mobile robot: evaluating gestural object references. In: IEEE/RSJ international conference on intelligent robots and systems (IROS), pp 3804–3809

  11. Sakurai S, Sato E, Yamaguchi T (2007) Recognizing pointing behavior using image processing for human–robot interaction. In: 2007 IEEE/ASME international conference on advanced intelligent mechatronics, pp 1–6

  12. Holladay RM, Dragan AD, Srinivasa SS (2014) Legible robot pointing. In: 2014 RO-MAN: the 23rd IEEE international symposium on robot and human interactive communication, pp 217–223

  13. Salem M, Kopp S, Wachsmuth I, Rohlfing K, Joublin F (2012) Generation and evaluation of communicative robot gesture. Int J Soc Robot 4:201–217

    Article  Google Scholar 

  14. Spexard T, Li S, Wrede B, Fritsch J, Sagerer G, Booij O et al (2006) BIRON, where are you? Enabling a robot to learn new places in a real home environment by integrating spoken dialog and visual localization. In: 2006 IEEE/RSJ international conference on intelligent robots and systems, pp 934–940

  15. Hato Y, Satake S, Kanda T, Imai M, Hagita N (2010) Pointing to space: modeling of deictic interaction referring to regions. In: 2010 5th ACM/IEEE international conference on human–robot interaction (HRI), pp 301–308

  16. Kendon A (2004) Gesture: visible action as utterance. Cambridge University Press, Cambridge

    Book  Google Scholar 

  17. Van Der Sluis I, Krahmer E (2001) Generating referring expressions in a multimodal context an empirically oriented approach. Lang Comput 37:158–176

    Google Scholar 

  18. Haywood SL, Pickering MJ, Branigan HP (2005) Do speakers avoid ambiguities during dialogue? Psychol Sci 16:362–366

    Article  Google Scholar 

  19. Paraboni I, van Deemter K, Masthoff J (2007) Generating referring expressions: making referents easy to identify. Comput Linguist 33:229–254

    Article  Google Scholar 

  20. Vaish A, Kumari P (2014) A comparative study on machine learning algorithms in emotion state recognition using ECG. In: Proceedings of the second international conference on soft computing for problem solving (SocProS 2012), 28–30 December 2012, pp 1467–1476

  21. Kumari P, Vaish A (2015) Information-theoretic measures on intrinsic mode function for the individual identification using EEG sensors. IEEE Sens J 15:4950–4960

  22. Bangerter A., Chevalley, E (2007) Pointing and describing in referential communication: When are pointing gestures used to communicate? In: MOG 2007 workshop on multimodal output generation, p 17

  23. Brooks AG, Breazeal C (2006) Working with robots and objects: Revisiting deictic reference for achieving spatial common ground. In: Proceedings of the 1st ACM SIGCHI/SIGART conference on human–robot interaction, pp 297–304

  24. Schultz AC, Trafton JG (2005) Towards collaboration with robots in shared space: spatial perspective and frames of reference. Interactions 12:22–24

    Article  Google Scholar 

  25. Glas DF, Miyashita T, Ishiguro H, Hagita N (2012) Human–robot interaction in social robotics. In: Kanda T, Ishiguro H (eds) Laser-based tracking of human position and orientation using parametric shape modeling. CRC Press, Boca Raton, pp 158–236

    Google Scholar 

  26. Burgoon JK, Buller DB, Hale JL, Turck MA (1984) Relational messages associated with nonverbal behaviors. Hum Commun Res 10:351–378

    Article  Google Scholar 

  27. Trout DL, Rosenfeld HM (1980) The effect of postural lean and body congruence on the judgment of psychotherapeutic rapport. J Nonverbal Behav 4:176–190

    Article  Google Scholar 

  28. Edelmann RJ (1982) The effect of embarrassed reactions upon others. Aust J Psychol 34:359–367

    Article  Google Scholar 

  29. Wolfe JM (1994) Guided search 2.0 A revised model of visual search. Psychon Bull Rev 1:202–238

    Article  Google Scholar 

  30. Sternberg S (1966) High-speed scanning in human memory. Science 153:652–654

    Article  Google Scholar 

  31. Treisman AM, Gelade G (1980) A feature-integration theory of attention. Cogn Psychol 12:97–136

    Article  Google Scholar 

  32. Kranstedt, A, Lücking A, Pfeiffer T, Rieser H, Wachsmuth I (2006) Deixis: how to determine demonstrated objects using a pointing cone. In: Gesture in human–computer interaction and simulation. Springer, Heidelberg, pp 300–311

  33. Kühnlein P, Stegmann J (2003) Empirical issues in deictic gesture: referring to objects in simple identification tasks. Report 2003/3, SFB, vol 360

  34. Wolfe JM, Horowitz TS, Kenner N, Hyle M, Vasan N (2004) How fast can you change your mind? The speed of top-down guidance in visual search. Vis Res 44:1411–1426

    Article  Google Scholar 

  35. Wang Q, Cavanagh P, Green M (1994) Familiarity and pop-out in visual search. Percept Psychophys 56:495–500

    Article  Google Scholar 

  36. Glas DF, Miyashita T, Ishiguro H, Hagita N (2009) Laser-based tracking of human position and orientation using parametric shape modeling. Adv Robot 23:405–428

    Article  Google Scholar 

  37. Müller C (2004) Forms and uses of the Palm Up Open Hand: a case of a gesture family. In: Posner R, Müller C (eds) The semantics and pragmatics of everyday gestures. Weidler Buchverlag, Berlin, pp 234–256

    Google Scholar 

  38. Ishii S (1973) Characteristics of Japanese nonverbal communicative behavior. J Commun Assoc Pacific 2:43–60

    Google Scholar 

  39. Richie D (1998) A lateral view: essays on culture and style in contemporary Japan. Stone Bridge Press, Berkeley

    Google Scholar 

Download references

Acknowledgments

We would like to thank Satoshi Koizumi for facilitating the smooth operation of the experiments. This study was funded in part by the Ministry of Internal Affairs and Communications of Japan and in part by JSPS KAKENHI Grant Number 25240042.

Author information

Authors and Affiliations

  1. Advanced Telecommunications Research Institute International, Kyoto, Japan

    Phoebe Liu, Dylan F. Glas, Takayuki Kanda, Hiroshi Ishiguro & Norihiro Hagita

Authors
  1. Phoebe Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dylan F. Glas
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Takayuki Kanda
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hiroshi Ishiguro
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Norihiro Hagita
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Phoebe Liu.

Ethics declarations

Conflict of interest

The authors declare that they have no conflicts of interest.

Ethical Approval

This research was conducted in compliance with the standards and regulations of our company’s ethical review board, which requires every experiment we conduct to be subject to a review and approval procedure according to strict ethical guidelines.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, P., Glas, D.F., Kanda, T. et al. A Model for Generating Socially-Appropriate Deictic Behaviors Towards People. Int J of Soc Robotics 9, 33–49 (2017). https://doi.org/10.1007/s12369-016-0348-9

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12369-016-0348-9

Keywords

Search

Navigation