Skip to main content
SpringerLink
Log in

Capturing Expertise: Developing Interaction Content for a Robot Through Teleoperation by Domain Experts

  • Published:
International Journal of Social Robotics Aims and scope Submit manuscript

Abstract

The development of humanlike service robots which interact socially raises a new question: how can we create good interaction content for such robots? Domain experts specializing in the target service have the knowledge for making such content. Yet, while they can easily engage in good face-to-face interactions, we found it difficult for them to prepare conversational content for a robot in written form. Instead, we propose involving experts as teleoperators in a short-cycle iterative development process in which the expert develops content, teleoperates a robot using that content, and then revises the content based on that interaction. We propose a software system and design guidelines to enable such an iterative design process. To validate these solutions, we conducted a comparison experiment in the field, with a teleoperated robot acting as a guide at a tourist information center in Nara, Japan. The results showed that our system and guidelines enabled domain experts with no robotics background to create better interaction content and conduct better interactions than domain experts without our system.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

References

  1. Pineau J, Montemerlo M, Pollack M, Roy N, Thrun S (2003) Towards robotic assistants in nursing homes: challenges and results. Robot Autonom Syst 42:271–281

    Article  MATH  Google Scholar 

  2. Shiomi M, Kanda T, Glas DF, Satake S, Ishiguro H, Hagita N (2009) Field trial of networked social robots in a shopping mall. In: Proceedings of the IEEE/RSJ international conference on intelligent robots and systems (IROS2009), pp 2846–2853

  3. Gross H-M, Boehme H-J, Schroeter C, Mueller S, Koenig A, Martin C, Merten M, Bley A (2008) ShopBot: progress in developing an interactive mobile shopping assistant for everyday use. IEEE international conference on systems, man, and cybernetics, pp 3471–3478

  4. Gockley R, Bruce A, Forlizzi J, Michalowski M, Mundell A, Rosenthal S, Sellner B, Simmons R, Snipes K, Schultz AC, Wang J (2005) Designing robots for long-term social interaction. In: IEEE/RSJ international conference on intelligent robots and systems (IROS2005), pp 1338–1343

  5. Weiss A, Igelsbock J, Tscheligi M, Bauer A, Kuhnlenz K, Wollherr D, Buss M (2010) Robots asking for directions: the willingness of passers-by to support robots. In: Proceedings of the ACM/IEEE international conference on human-robot interaction (HRI2010), pp 23–30

  6. Polanyi M (1962) Personal knowledge: towards a post-critical philosophy. University of Chicago Press, Chicago

    Google Scholar 

  7. Eraut M (2000) Non-formal learning and tacit knowledge in professional work. Br J Educat Psychol 70:113–136

    Article  Google Scholar 

  8. Kahn P, Freier N, Kandau T, Ishiguro H, Ruckert J, Severson R, Kane S (2008) Design patterns for sociality in human-robot interaction. In: ACM/IEEE international conference on human-robot interaction, pp 97–104

  9. Kanda T, Shiomi M, Miyashita Z, Ishiguro H, Hagita N (2010) A communication robot in a shopping mall. IEEE Trans Robot 26:897–913

    Article  Google Scholar 

  10. Glas DF, Kanda T, Ishiguro H, Hagita N (2012) Teleoperation of multiple social robots. IEEE Trans Syst Man Cybernet A Syst Humans 42(3):530–544

    Article  Google Scholar 

  11. McTear MF (2002) Spoken dialogue technology: enabling the conversational user interface. ACM Comput Surv (CSUR) 34:90–169

    Article  Google Scholar 

  12. Glas DF, Satake S, Kanda T, Hagita N (2011) An interaction design framework for social robots. In: Robotics: science and systems conference

  13. McTear MF (1998) Modelling spoken dialogues with state transition diagrams: experiences with the CSLU toolkit. In: International conference on spoken language processing (ICSLP1998), pp 1223–1226

  14. Bohus D, Raux A, Harris TK, Eskenazi M, Rudnicky AI (2007) Olympus: an open-source framework for conversational spoken language interface research. In: HLT-NAACL 2007 workshop on bridging the gap: academic and industrial research in dialog technology, 2007, pp 32–39

  15. Krenn B, Sieber G (2008) Functional mark-up for behaviour planning: theory and practice. In: Proceedings of the AAMAS 2008 workshop FML: functional markup language. Why conversational agents do what they do

  16. Nishimura Y, Minotsu S, Dohi H, Ishizuka M, Nakano M, Funakoshi K, Takeuchi J, Hasegawa Y, Tsujino H (2007) A markup language for describing interactive humanoid robot presentations. In: International conference on intelligent user interfaces (IUI 2007), pp 333–336

  17. Chernova S, Orkin J, Breazeal C (2010) Crowdsourcing HRI through online multiplayer games. In: The AAAI fall symposium dialog with robots, pp 14–19

  18. Wallis P, Mitchard H, Das J, ODea D (2001) Dialogue modelling for a conversational agent. In: Stumptner M, Corbett D, Brooks M (eds) AI 2001: advances in artificial intelligence, vol 2256. Lecture notes in artificial intelligenceSpringer, Berlin

  19. Boose JH, Bradshaw JM (1987) Expertise transfer and complex problems: using AQUINAS as a knowledge-acquisition workbench for knowledge-based systems. Int J Man Mach Stud 26(1):3–28

  20. Kuo I-H, Jayawardena C, Broadbent E, MacDonald BA (2011) Multidisciplinary design approach for implementation of interactive services: communication initiation and user identification for healthcare service robots. Int J Soc Robot 3:443–456

    Article  Google Scholar 

  21. Lohse M, Siepmann F (2014) A modeling framework for user-driven iterative design of autonomous systems. Int J Soc Robot 6(1):121–139

    Article  Google Scholar 

  22. Ross D, Lim J, Lin R-S, Yang M-H (2008) Incremental learning for robust visual tracking. Int J Comput Vis 77(1–3):125–141

    Article  Google Scholar 

  23. Zang P, Tian R, Thomaz AL, Isbell CL (2010) Batch versus interactive learning by demonstration. In: Proceedings of the IEEE 9th international conference on development and learning (ICDL), pp 219–224

  24. Petelin JB, Nelson ME, Goodman J (2007) Deployment and early experience with remote-presence patient care in a community hospital. Surgic Endosc 21:53–56

    Article  Google Scholar 

  25. Tsui KM, Desai M, Yanco HA, Uhlik C (2011) Exploring use cases for telepresence robots. In: ACM/IEEE international conference on human-robot interaction (HRI2011)

  26. Yun S-S, Kim M, Choi M-T (2013) Easy interface and control of tele-education robots. Int J Soc Robot 5:335–343

    Article  Google Scholar 

  27. Takayama L, Marder-Eppstein E, Harris H, Beer JM (2011) Assisted driving of a mobile remote presence system: system design and controlled user evaluation. In Proceedings of the IEEE international conference on robotics and automation (ICRA2011), pp 1883–1889

  28. Zheng K, Glas DF, Kanda T, Ishiguro H, Hagita N (2011) How many social robots can one operator control? Proceedings of the ACM/IEEE 6th annual conference on human-robot interaction. Lausanne, Switzerland, pp 379–386

    Google Scholar 

  29. Kanda T, Shiomi M, Miyashita Z, Ishiguro H, Hagita N (2009) An affective guide robot in a shopping mall. In: ACM/IEEE international conference on human-robot interaction (HRI2009), pp 173–180

  30. Castro-Gonzlez Á, Malfaz M, Salichs MA (2011) Learning the selection of actions for an autonomous social robot by reinforcement learning based on motivations. Int J Soc Robot 3:427–441

    Article  Google Scholar 

  31. Hollingsed TK, Ward Nigel G (2007) A combined method for discovering short-term affect-based response rules for spoken tutorial dialog. In: Proceedings of the ISCA ITRW workshop on speech and language technology in education (SLaTE)

  32. Nigel W, Anais GR, Karen W, David GN (2005) Some usability issues and research priorities in spoken dialog applications. Technical report UTEP-CS-05-23, University of Texas at El Paso

  33. Jung J, Kanda T, Kim M-S (2013) Guidelines for contextual motion design of a humanoid robot. Int J Soc Robot 5(2):153–169

    Article  Google Scholar 

  34. Sacks HH, Schegloff EA, Jefferson G (1974) A simplest systematics for the organization of turn-taking for conversation. Language 50(4):696–735

    Article  Google Scholar 

  35. Chao C, Thomaz AL (2010) Turn taking for human-robot interaction. Paper presented at the AAAI fall symposium: dialog with robots

  36. Nagaoka C, Komori M, Nakamura T (2005) Influence of response latencies on impression evaluation of speakers in dialogues: differences of cues used for evaluation by degree of social skill. Tech Rep IEICE 104(745):57–60

    Google Scholar 

  37. Jaffe J, Feldstein S (1970) Rhythms of dialogue. Academic, New York

    Google Scholar 

  38. Shiwa T, Kanda T, Imai M, Ishiguro H, Hagita N (2008) How quickly should communication robots respond? In: Proceedings of the ACM/IEEE international conference on human-robot interaction (HRI2008), pp 153–160

  39. Glas DF, Kanda T, Ishiguro H, Hagita N (2012) Temporal awareness in teleoperation of conversational robots. IEEE Trans Syst Man Cybernet A Syst Humans 42(4):905–919

    Article  Google Scholar 

  40. Ward N, Tsukahara W (2000) Prosodic features which cue back-channel responses in English and Japanese. J Pragmat 32(8):1177–1207

    Article  Google Scholar 

  41. Kanda T, Ishiguro H, Imai M, Ono T (2004) Development and evaluation of interactive humanoid robots. Proc IEEE 92(11):1839–1850

    Article  Google Scholar 

  42. Miller GA (1956) The magical number seven, plus or minus two: some limits on our capacity for processing information. Psychological Rev 63(2):81

    Article  Google Scholar 

  43. Dahlbck N, Jnsson A, Ahrenberg L (1993) Wizard of Oz studies: why and how. In: International conference on intelligent user interfaces, pp 193–200

  44. Villano M, Crowell CR, Wier K, Tang K, Thomas B, Shea N, Schmitt LM, Diehl JJ (2011) DOMER: a wizard of oz interface for using interactive robots to scaffold social skills for children with autism spectrum disorders. In: ACM/IEEE 6th international conference on human-robot interaction, Lausanne, Switzerland

  45. Kawai H, Toda T, Ni J, Tsuzaki M, Tokuda K (2004) XIMERA: a new TTS from ATR based on corpus-based technologies. In: ISCA speech synthesis workshop, pp 179–184

Download references

Acknowledgments

This research was supported by the Ministry of Internal Affairs and Communications of Japan. We wish to thank the staff at the Nara City Tourist information center, and the members of the NPO “Suzaku” for their helpful participation. We also wish to thank Satoshi Koizumi and Masaya Shimoyama for their help.

Author information

Authors and Affiliations

  1. ATR Intelligent Robotics and Communication Laboratories, 2-2-2 Hikaridai, Keihanna Science City, Kyoto, Japan

    Kanae Wada, Dylan F. Glas, Masahiro Shiomi, Takayuki Kanda & Norihiro Hagita

  2. Faculty of Science and Engineering, Osaka University, 1-3, Machikaneyama, Toyonaka, Osaka, Japan

    Hiroshi Ishiguro

Authors
  1. Kanae Wada
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dylan F. Glas
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Masahiro Shiomi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Takayuki Kanda
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hiroshi Ishiguro
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Norihiro Hagita
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dylan F. Glas.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wada, K., Glas, D.F., Shiomi, M. et al. Capturing Expertise: Developing Interaction Content for a Robot Through Teleoperation by Domain Experts. Int J of Soc Robotics 7, 653–672 (2015). https://doi.org/10.1007/s12369-015-0288-9

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12369-015-0288-9

Keywords

Search

Navigation