Skip to main content
Springer Nature Link
Log in

Building a Model of the Environment from a Route Perspective for Human–Robot Interaction

  • Published:
International Journal of Social Robotics Aims and scope Submit manuscript

Abstract

We built a model of the environment for human–robot interaction by learning from humans cognitive processes. Our method, which differs from previous map building techniques in terms of perspectives, is based on a route perspective that is a mental tour of the environment. The main contribution of this work is the theory and computational implementation of the concept of route with its respective visual memory. The concept of route is modeled as a three layered model composed of memory layer, survey layer and route layer. By imitating the human concept of a route, the route layer is modeled as a directional path segmented by action taking associated with visual memory taken while traveling the path. We developed a system that generates human understandable route directions which was evaluated towards two methods: one copied from the explanation of a human expert and one generated with a model without the route perspective layer. Finally, experimental results demonstrate the usefulness of the route perspective layer, since it performed better than the model without the route layer and similarly to the human expert.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

References

  1. Morales Y, Satake S, Kanda T, Hagita N (2011) Modeling environments from a route perspective. Presented at the ACM/IEEE international conference on human robot interaction (HRI2011), Lausanne Switzerland

  2. Burgard W, Cremers AB, Fox D, Hahnel D, Lakemeyer G, Schulz D et al (1998) The interactive museum tour-guide robot. Presented at the national conference on artificial intelligence (AAAI1998)

  3. Siegwart R, Arras KO, Bouabdallah S, Burnier D, Froidevaux G, Greppin X et al (2003) Robox at expo. 02: a large scale installation of personal robots. Robot Auton Syst 42:203–222

    Article  MATH  Google Scholar 

  4. Kulyukin V, Gharpure C, Nicholson J (2005) RoboCart: toward robot-assisted navigation of grocery stores by the visually impaired. Presented at the IEEE/RSJ international conference on intelligent robots and systems (IROS2005)

  5. Gross H-M, Boehme H-J, Schroeter C, Mueller S, Koenig A, Martin C, et al. (2008) ShopBot: progress in developing an interactive mobile shopping assistant for everyday use. Presented at the IEEE international conference on systems, man, and cybernetics (SMC2008)

  6. Mühlbauer Q, Sosnowski S, Xu T, Zhang T, Kühnlenz K, Buss M (2009) Navigation through Urban environments by visual perception and interaction. Presented at the IEEE international conference on robotics and automation (ICRA2009)

  7. Alami R, Warnier M, Guitton J, Lemaignan S, Sisbot E (2011) When the robot considers the human. In: Proceedings of the 15th international symposium on robotics research

  8. Trafton JG, Cassimatis NL, Bugajska MD, Brock DP, Mintz FE, Schultz AC (2005) Enabling effective human–Robot interaction using perspective-taking in robots. IEEE Trans Syst Man Cybern. Part A 35:460–470

    Article  Google Scholar 

  9. Berlin M, Gray J, Thomaz AL, Breazeal C (2006) Perspective taking: an organizing principle for learning in human–Robot interaction. Presented at the national conference on artificial intelligence (AAAI2006)

  10. Pandey AK, Alami R (2010) Visuo-spatial reasoning for human–robot interaction. Presented at the 12th international symposium on experimental robotics (ISER-2010)

  11. Okuno Y, Kanda T, Imai M, Ishiguro H, Hagita N (2009) Providing route directions: design of robot’s utterance, gesture, and timing. Presented at the ACM/IEEE international conference on human–robot interaction (HRI2009)

  12. Ono T, Imai M, Ishiguro H (2001) A model of embodied communications with gestures between humans and robots. Presented at the annual meeting of the cognitive science society (CogSci2001)

  13. Beeson P, Jong NK, Kuipers B (2005) Towards autonomous topological place detection using the extended voronoi graph. Presented at the IEEE international conference on robotics and automation (ICRA2005)

  14. Topp EA, Christensen HI (2006) Topological modelling for human augmented mapping. Presented at the IEEE/RSJ international conference on intelligent robots and systems (IROS2006)

  15. Werner S, Krieg-Brückner B, Herrmann T (2000) Modelling navigational knowledge by route graphs in spatial cognition II (Number 1849 in LNAI), Freksa C, Habel C, Wender K (eds) ed: Springer, pp. 295–317

  16. Lynch K (1960) The image of the city. The MIT Press, Cambridge

    Google Scholar 

  17. Golledge RG, Stimson RJ (1996) Spatial behavior: a geographic perspective. The Guilford Press, New York

    Google Scholar 

  18. Tversky B (1993) Cognitive maps, cognitive collages, and spatial mental models. Spatial information theory a theoretical basis for GIS, pp 14–24

  19. Fox D, Burgard W, Thrun S (1999) Markov localization for mobile robots in dynamic environments. J Artif Intell Res 11:391–427

    MATH  Google Scholar 

  20. Brunskill E, Kollar T, Roy N (2007) Topological mapping using spectral clustering and classification. Presented at the IEEE/RSJ international conference on intelligent robots and systems (IROS2007)

  21. Striegnitz K, Tepper P, Lovett A, Cassell J (2005) Knowledge representation for generating locating gestures in route directions. Presented at the Workshop on spatial language and dialogue (5th workshop on language and space)

  22. Kopp S, Tepper PA, Ferriman K, Striegnitz K, Cassell J (2008) Trading spaces: how humans and humanoids use speech and gesture to give directions. In: T Nishida (ed) Conversational informatics: an engineering approach, pp. 133–160

  23. Durrant-Whyte H, Bailey T (2006) Simultaneous localization and mapping: part i. IEEE Robot Autom Mag 13:99–110

    Article  Google Scholar 

  24. Bailey T, Durrant-Whyte H (2006) Simultaneous localization and mapping (slam): part ii. IEEE Robot Autom Mag 13:108–117

    Article  Google Scholar 

  25. Thrun S, Burgard W, Fox D (2005) Probabilistic robotics (intelligent robotics and autonomous agents). The MIT Press, Cambridge

    Google Scholar 

  26. Montemerlo M, Thrun S, Koller D, Wegbreit B (2002) FastSLAM: a factored solution to the simultaneous localization and mapping problem. Presented at the proceedings of the AAAI national conference on artificial intelligence, Edmonton

  27. Olson E, Leonard J, Teller S (2006) Fast iterative alignment of pose graphs with poor initial estimates. Presented at the IEEE international conference on robotics and automation (ICRA 2006)

  28. Bosse M, Newman PM, Leonard JJ, Teller S (2004) Slam in large scale cyclic environments using the atlas framework. Int J Robot Res 23:1113–1139

    Article  Google Scholar 

  29. Konolige K (2004) Large-scale map-making. Presented at the national conference on artificial intelligence (AAAI’04)

  30. Paul R, Newman P (2010) Fab-map 3d: topological mapping with spatial and visual appearance. Presented at the proceedings IEEE international conference on robotics and automation (ICRA’10), Anchorage, Alaska

  31. Elfes A (1989) Using occupancy grids for mobile robot perception and navigation. Computer 22:46–57

    Article  Google Scholar 

  32. Grisetti G, Stachniss C, Burgard W (2007) Improved techniques for grid mapping with rao-blackwellized particle filters. IEEE Trans Robot 23:34–46

    Article  Google Scholar 

  33. Konolige K, Bowman J, Chen J, Mihelich P, Calonder M, Lepetit V et al (2010) View-based maps. Int J Robot Res 29:941–957

    Article  Google Scholar 

  34. Kuipers B (2000) The spatial semantic hierarchy. Artif Intell 119:191–233

    Article  MATH  MathSciNet  Google Scholar 

  35. Beeson P, Modayil J, Kuipers B (2010) Factoring the mapping problem: mobile robot map-building in the hybrid spatial semantic hierarchy. Int J Robot Res 29:428–459

    Article  Google Scholar 

  36. Zender H, Mozos ÓM, Jensfelt P, Kruijff G-JM, Burgard W (2008) Conceptual spatial representations for indoor mobile robots. Robot Auton Syst 56:493–502

    Article  Google Scholar 

  37. Kollar T, Tellex S, Roy D, Roy N (2010) Toward understanding natural language directions. Presented at the ACM/IEEE international conference on human–robot interaction (HRI2010)

  38. Matuszek C, Fox D, Koscher K (2010) Following directions using statistical machine translation. Presented at the ACM/IEEE international conference on human–robot interaction (HRI2010)

  39. Yeap WK (1988) Towards a computational theory of cognitive maps. Artif Intell 34:297–360

    Article  Google Scholar 

  40. Mozos ÓM, Triebel R, Jensfelt P, Rottmann A, Burgard W (2007) Supervised semantic labeling of places using information extracted from sensor data. Robot Auton Syst 55:391–402

    Article  Google Scholar 

  41. Vasudevan S, Gachter S, Nguyen V, Siegwart R (2007) Cognitive maps for mobile robots-an object based approach. Robot Auton Syst 55:359–371

    Article  Google Scholar 

  42. Yeap WK, Jefferies ME (1999) Computing a representation of the local environment. Artif Intell 107:265–301

    Article  MATH  Google Scholar 

  43. Yeap WK, Jefferies ME (2000) On early cognitive mapping. Spat Cogn Comput 2:85–116

    Google Scholar 

  44. Peltason J, Siepmann FHK, Spexard TP, Wrede B, Hanheide M, Topp EA (2009) Mixed-initiative in human augmented mapping. Presented at the IEEE international conference on robotics and automation (ICRA2009)

  45. Hato Y, Satake S, Kanda T, Imai M, Hagita N (2010) Pointing to space: modeling of deictic interaction referring to regions. Presented at the ACM/IEEE international conference on human–robot interaction (HRI2010)

  46. Kendon A (2004) Gesture: visible action as utterance. Cambridge University Press, New York

    Book  Google Scholar 

  47. Kita S (2003) Interplay of gaze, hand, torso orientation and language in pointing. In: Kita S (ed) Pointing: where language, culture, and cognition meet, pp. 307–328

  48. Daniel M-P, Tom A, Manghi E, Denis M (2003) Testing the value of route directions through navigational performance. Spat Cogn Comput 3:269–289

    Google Scholar 

  49. Vanetti EJ, Allen GL (1988) Communicating environmental knowledge: the impact of verbal and spatial abilities on the production and comprehension of route directions. Environ Behav 20:667–682

    Article  Google Scholar 

  50. Levit M, Roy D (2007) Interpretation of spatial language in a map navigation task. IEEE Trans Syst Man Cybern Part B 37:667–679

    Article  Google Scholar 

  51. Shimizu N, Haas A (2009) Learning to follow navigational route instructions. Presented at the international Joint conference on artificial intelligence (IJCAI2009)

  52. Vogel A, Jurafsky D (2010) Learning to follow navigational directions, presented at the the 48th annual meeting of the association for computational linguistics stroudsburg, PA

  53. Look G (2008) Cognitively-inspired direction giving, Ph.D. Thesis

  54. Ross RJ, Mandel C, Bateman JA, Hui S, Frese U, Towards stratified spatial modeling for communication & navigation, from sensors to human spatial concepts, p. 41

  55. Shi H, Tenbrink T (2005) Telling rolland where to go: HRI dialogues on route navigation. In: Proceedings of WoSLaD workshop on spatial language and dialogue

  56. Mandel C, Frese U, Rofer T (2006) Robot navigation based on the mapping of coarse qualitative route descriptions to route graphs, in IEEE/RSJ international conference on intelligent robots and systems (IROS), pp. 205–210

  57. Nothegger C, Winter S, Raubal M (2004) Selection of salient features for route directions. Spat Cogn Comput 4:113–136

    Google Scholar 

  58. Montello DR (1997) The perception and cognition of environmental distance: direct sources of information. Presented at the proceedings of the international conference on spatial information theory: a theoretical basis for GIS

  59. Richter K-F, Klippel A (2004) A model for context-specific route directions. Lecture notes in computer science: SPATIAL COGNITION IV. REASONING, ACTION. INTERACTION 3343/2005, 58–78

  60. Klippel A, Hansen S, Davies J, Winter S (2005) A high-level cognitive framework for route directions. Presented at the the national biennial conference of the spatial science institute

  61. Denis M (1997) The description of routes: a cognitive approach to the production of spatial discourse. Cah de Psychol Cogn 16:409–458

    Google Scholar 

  62. Nuechter A (2009) 3D robotic mapping. Springer Verlag, Heidelberg

    MATH  Google Scholar 

  63. Moravec HP (1989) Certainty grids for sensor fusion in mobile robots, Sensor devices and systems for robotics, pp. 243–276

  64. Besl P, McKay N (1992) A method for registration of 3-D shapes. IEEE Trans Pattern Anal Mach Intell 14:239–256

    Article  Google Scholar 

  65. Nuchter A, Lingemann K, Hertzberg J, Surmann H (2007) 6d slam-3d mapping outdoor environments. J Field Robot 24:699–722

    Article  Google Scholar 

  66. Remolina E, Kuipers B (2004) Towards a general theory of topological maps. Artif Intell 152:47–104

    Article  MATH  MathSciNet  Google Scholar 

  67. Nagatani K, Choset H (1999) Toward robust sensor based exploration by constructing reduced generalized voronoi graph. Presented at the IEEE/RSJ international conference on intelligent robots and systems

  68. Morris AC, Silver D, Ferguson D, Thayer S (2005) Towards topological exploration of abandoned mines. Presented at the the IEEE international conference on robotics and automation (ICRA2005), Barcelona

  69. Dellaert F, Fox D, Burgard W, Thrun S (1999) Monte carlo localization for mobile robots, in robotics and automation, 1999. Proceedings. 1999 IEEE international conference on, pp. 1322–1328.

  70. Kuipers B (1978) Modeling spatial knowledge. Cogn Sci 2:129–153

  71. Smith R (2007) An overview of the tesseract OCR engine. Presented at the Ninth international conference on document analysis and recognition (ICDAR 2007)

  72. Levenshtein VI (1966) Binary codes capable of correcting deletions, insertions, and reversals. Sov Phys Dokl 10:707–710

    MathSciNet  Google Scholar 

Download references

Acknowledgments

This research was supported by the Ministry of Internal Affairs and Communications of Japan.

Author information

Authors and Affiliations

  1. Intelligent Robotics and Communication Laboratories, Advanced Telecommunications Research Institute International, Kyoto, Japan

    Yoichi Morales, Satoru Satake, Takayuki Kanda & Norihiro Hagita

Authors
  1. Yoichi Morales
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Satoru Satake
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Takayuki Kanda
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Norihiro Hagita
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yoichi Morales.

Additional information

This paper is an extended version of conference paper [1], with integrated technical details, additional discussions between our proposal and existing cognitive map literature; we added more participants for our experiments and added explanations in the experimental and conclusion sections.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Morales, Y., Satake, S., Kanda, T. et al. Building a Model of the Environment from a Route Perspective for Human–Robot Interaction. Int J of Soc Robotics 7, 165–181 (2015). https://doi.org/10.1007/s12369-014-0265-8

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12369-014-0265-8

Keywords