Skip to main content
SpringerLink
Log in

Personal Greetings: Personalizing Robot Utterances Based on Novelty of Observed Behavior

  • Published:
International Journal of Social Robotics Aims and scope Submit manuscript

Abstract

One challenge in creating conversational service robots is how to reproduce the kind of individual recognition and attention that a human can provide. We believe that interactions can be made to seem more warm and humanlike by using sensors to observe a person’s behavior or appearance over time, and programming the robot to comment when it observes a novel feature, such as a new hairstyle, or a consistent behavior, such as visiting every afternoon. To create a system capable of recognizing such novelty and typicality, we collected one month of training data from customers in a shopping mall and recorded features of people’s visits, such as time of day and group size. We then trained SVM classifiers to identify each feature as novel, typical, or neither, based on the inputs of a human coder, and we trained an additional classifier to choose an appropriate topic for a personalized greeting. An utterance generator was developed to generate text for the robot to speak, based on the selected topic and sensor data. A cross-validation analysis showed that the trained classifiers could accurately reproduce human novelty judgments with 88% accuracy and topic selection with 95% accuracy. We then deployed a teleoperated robot using this system to greet customers in a shopping mall for three weeks, and we present example interactions and results from interviews showing that customers appreciated the robot’s personalized greetings and felt a sense of familiarity with the robot.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. ATRacker is a product of ATR Promotions: http://www.atr-p.com/HumanTracker.html.

  2. OKAO Vision is a product of OMRON Corporation: http://www.omron.com/r_d/coretech/vision/okao.html.

References

  1. Friedman B, Kahn PH, Hagman J (2003) Hardware companions?-What online AIBO discussion forums reveal about the Human-Robotic relationship. Paper presented at the ACM conference on human factors in computing systems (CHI2003)

  2. Kahn PH, Kanda T, Ishiguro H, Freier NG, Severson RL, Gill BT, Ruckert JH, Shen S (2012) “Robovie, you’ll have to go into the closet now”: children’s social and moral relationships with a humanoid robot. Dev Psychol 48(2):303

    Article  Google Scholar 

  3. Wada K, Shibata T (2007) Living with seal robots-its sociopsychological and physiological influences on the elderly at a care house. IEEE Trans Robot 23(5):972–980

    Article  Google Scholar 

  4. Sabelli AM, Kanda T, Hagita N (2011) A conversational Robot in an Elderly Care Center: an Ethnographic Study. Paper presented at the ACM/IEEE international conference on human-robot interaction (HRI2011)

  5. Eyssel F, Reich N (2013) Loneliness makes the heart grow fonder (of robots)—on the effects of loneliness on psychological anthropomorphism. In: 8th ACM/IEEE international conference on human-robot interaction (HRI), pp 121–122

  6. Bickmore TW, Picard RW (2005) Establishing and maintaining long-term human-computer relationships. ACM Trans Comput Hum Int 12(2):293–327

    Article  Google Scholar 

  7. Gockley R, Bruce A, Forlizzi J, Michalowski M, Mundell A, Rosenthal S, Sellner B, Simmons R, Snipes K, Schultz AC, Jue W (2005) Designing robots for long-term social interaction. In: Intelligent robots and systems (IROS 2005). 2005 IEEE/RSJ international conference on, 2-6 Aug. 2005, pp 1338-1343. doi:10.1109/iros.2005.1545303

  8. Leite I, Castellano G, Pereira A, Martinho C, Paiva A (2012) Long-term interactions with empathic robots: evaluating perceived support in children. Soc Robot 298–307

  9. Kidd CD (2008) Designing for long-term human-robot interaction and application to weight loss. Massachusetts Institute of Technology, Cambridge, MA

    Google Scholar 

  10. Pan Y, Okada H, Uchiyama T, Suzuki K (2015) On the reaction to Robot’s speech in a hotel public space. Int J Soc Robot 7(5):911–920. doi:10.1007/s12369-015-0320-0

    Article  Google Scholar 

  11. Riek LD, Paul PC, Robinson P (2009) When my robot smiles at me: enabling human-robot rapport via real-time head gesture mimicry. J Multimodal User Interfaces 3(1–2):99–108

    Google Scholar 

  12. Sakamoto D, Kanda T, Ono T, Kamashima M, Imai M, Ishiguro H (2004) Cooperative embodied communication emerged by interactive humanoid robot. Int J Hum Comput Stud 62:247–265

    Article  Google Scholar 

  13. Sabelli AM, Kanda T, Hagita N (2011) A conversational robot in an elderly care center: an ethnographic study. In: 6th ACM/IEEE international conference on human-robot interaction (HRI), 8-11 March 2011, pp 37–44

  14. Kanda T, Sato R, Saiwaki N, Ishiguro H (2007) A two-month field trial in an elementary school for long-term human-robot interaction. IEEE Trans Robot 23(5):962–971. doi:10.1109/tro.2007.904904

    Article  Google Scholar 

  15. Kanda T, Shiomi M, Miyashita Z, Ishiguro H, Hagita N (2010) A communication robot in a shopping mall. Trans Robot 26(5):897–913. doi:10.1109/tro.2010.2062550

    Article  Google Scholar 

  16. Chandola V, Banerjee A, Kumar V (2009) Anomaly detection: a survey. ACM Comput Surv 41(3):1–58. doi:10.1145/1541880.1541882

    Article  Google Scholar 

  17. Huber PJ (1981) Robust statistics, vol 1., Wiley series in probability and mathematical statisticsWiley, New York

    Book  MATH  Google Scholar 

  18. Marsland S, Nehmzow U, Shapiro J (2005) On-line novelty detection for autonomous mobile robots. Robot Auton Syst 51(2):191–206

    Article  Google Scholar 

  19. Andry P, Gaussier P, Moga S, Banquet JP, Nadel J (2001) Learning and communication via imitation: an autonomous robot perspective. IEEE Trans Syst Man Cybern Part A 31(5):431–442. doi:10.1109/3468.952717

    Article  Google Scholar 

  20. Bonaccorsi M, Fiorini L, Cavallo F, Saffiotti A, Dario P (2016) A cloud robotics solution to improve social assistive robots for active and healthy aging. Int J Soc Robot 8(3):393–408. doi:10.1007/s12369-016-0351-1

    Article  Google Scholar 

  21. Glas DF, Miyashita T, Ishiguro H, Hagita N (2009) Laser-based tracking of human position and orientation using parametric shape modeling. Adv Robot 23(4):405–428. doi:10.1163/156855309x408754

    Article  Google Scholar 

  22. Lao S, Kawade M (2005) Vision-based face understanding technologies and their applications. In: Li SZ, Lai J, et al. (eds) Advances in biometric person authentication. Springer, New York, pp 339–348

  23. Yücel Z, Zanlungo F, Ikeda T, Miyashita T, Hagita N (2013) Deciphering the crowd: modeling and identification of pedestrian group motion. Sensors 13(1):875–897

    Article  Google Scholar 

  24. Chang CC, Lin CJ (2001) LIBSVM: a library for support vector machines

  25. Shiwa T, Kanda T, Imai M, Ishiguro H, Hagita N How quickly should communication robots respond? In: Proceedings of the 3rd ACM/IEEE international conference on human robot interaction, Amsterdam, The Netherlands, 2008. ACM, pp 153-160. doi:10.1145/1349822.1349843

  26. Brščić D, Kanda T, Ikeda T, Miyashita T (2013) Person tracking in large public spaces using 3-D range sensors. IEEE Trans Hum Mach Syst 43(6):522–534. doi:10.1109/thms.2013.2283945

    Article  Google Scholar 

Download references

Acknowledgements

We would like to thank Dr. Satoshi Koizumi, Tony Han, Benoit Toulmé, Peace Cho, and the management of the APiTA Town Keihanna shopping mall for their help in the organization and execution of the data collection.

Author information

Authors and Affiliations

  1. ATR, Keihanna Science City, Kyoto, Japan

    Dylan F. Glas, Kanae Wada, Masahiro Shiomi, Takayuki Kanda & Norihiro Hagita

  2. Osaka University, Toyonaka, Osaka, Japan

    Hiroshi Ishiguro

Authors
  1. Dylan F. Glas
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kanae Wada
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Masahiro Shiomi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Takayuki Kanda
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hiroshi Ishiguro
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Norihiro Hagita
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dylan F. Glas.

Ethics declarations

Conflict of interest

The authors declare that they have no conflicts of interest.

Ethical Approval

This research was conducted in compliance with the standards and regulations of our company’s ethical review board, which requires every experiment we conduct to be subject to a review and approval procedure according to strict ethical guidelines.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Glas, D.F., Wada, K., Shiomi, M. et al. Personal Greetings: Personalizing Robot Utterances Based on Novelty of Observed Behavior. Int J of Soc Robotics 9, 181–198 (2017). https://doi.org/10.1007/s12369-016-0385-4

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12369-016-0385-4

Keywords

Search

Navigation