Skip to main content
SpringerLink
Log in

Reinforcement Learning Aided Robot-Assisted Navigation: A Utility and RRT Two-Stage Approach

  • Published:
International Journal of Social Robotics Aims and scope Submit manuscript

Abstract

This work proposes a robot-assisted navigation approach based on user intent adjustment, in the context of robotic walkers. Walkers are prescribed to users with gait disorders so that they can support their body weight on the upper limbs, however, the manipulation of such devices can be cumbersome for some users. Common problems for the users are lack of dexterous upper limb control and visual impairments. These problems can render walkers’ users helpless, making them unable to operate these devices effectively and efficiently. We present a new approach to robot-assisted navigation using a utility decision and safety analysis procedure with user intent adjustments learned by reinforcement learning (RL) and supported on a rapidly-exploring random tree inspired algorithm. The proposed approach offers full control of the assistive platform to the user until obstacles are detected. In dangerous scenarios, corrections are computed in order that the assistive platform avoids collisions and follows social norms, effectively guiding the user through the environment while enforcing safer routes. The experimental validation was carried out in a virtual environment and in a real world scenario using a robotic walker built in our lab (ISR-AIWALKER). Experimental results have shown that the proposed approach provides a reliable solution to the robot-assisted navigation of a robotic walker, in particular the use of utility theory to evaluate candidate motions together with a RL model increases the safety of the user’s navigation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

Notes

  1. Kinect One driver Package—https://github.com/code-iai/iai_kinect2.

References

  1. Balash Y, Hadar-Frumer M, Herman T, Peretz C, Giladi N, Hausdorff J (2007) The effects of reducing fear of falling on locomotion in older adults with a higher level gait disorder. J Neural Trans 114(10):1309–1314

    Article  Google Scholar 

  2. Bateni H, Maki BE (2005) Assistive devices for balance and mobility: benefits, demands, and adverse consequences. Arch Phys Med Rehabil 86(1):134–145

    Article  Google Scholar 

  3. Burns B, Brock O (2007) Single-query motion planning with utility-guided random trees. In: 2007 IEEE international conference on robotics and automation, pp 3307–3312

  4. Chuy Jr O, Hirata Y, Wang Z, Kosuge K (2005) Motion control algorithms for a new intelligent robotic walker in emulating ambulatory device function. In: 2005 IEEE international conference mechatronics and automation

  5. Constantinescu R, Leonard C, Deeley C, Kurlan R (2007) Assistive devices for gait in Parkinson’s disease. Parkinsonism Relat Disord 13(3):133–138

    Article  Google Scholar 

  6. Duan Y, Cui B, Yang H (2008) Robot navigation based on fuzzy RL algorithm. In: International symposium on neural networks. Springer, Berlin, pp 391–399

  7. Dubowsky S, Genot F, Godding S, Kozono H, Skwersky A, Yu H, Yu LS (2000) PAMM: a robotic aid to the elderly for mobility assistance and monitoring: a helping-hand for the elderly. In: IEEE International conference on robotics and automation

  8. Ferrari F, Divan S, Guerrero C, Zenatti F, Guidolin R, Palopoli L, Fontanelli D (2019) Human-robot interaction analysis for a smart walker for elderly: the acanto interactive guidance system. Int J Soc Robot. https://doi.org/10.1007/s12369-019-00572-5

    Article  Google Scholar 

  9. Garrote L, Paulo J, Perdiz J, Peixoto P, Nunes UJ (2018) Robot-assisted navigation for a robotic walker with aided user intent. In: 2018 27th IEEE international symposium on robot and human interactive communication (RO-MAN), pp 348–355

  10. Garrote L, Rosa J, Paulo J, Premebida C, Peixoto P, Nunes U (2017) 3D point cloud downsampling for 2D indoor scene modelling in mobile robotics. In: 2017 IEEE international conference on autonomous robot systems and competitions (ICARSC)

  11. Geravand M, Werner C, Hauer K, Peer A (2016) An integrated decision making approach for adaptive shared control of mobility assistance robots. Int J Soc Robot 8(5):631–648

    Article  Google Scholar 

  12. Khriji L, Touati F, Benhmed K, Al-Yahmedi A (2011) Mobile robot navigation based on Q-learning technique. Int J Adv Robot Syst 8(1):4

    Article  Google Scholar 

  13. Kirby R, Simmons R, Forlizzi J (2009) COMPANION: a constraint-optimizing method for person-acceptable navigation. In: IEEE international symposium on robot and human interactive communication (RO-MAN)

  14. Kober J, Bagnell JA, Peters J (2013) Reinforcement learning in robotics: a survey. Int J Robot Res 32(11):1238–1274

    Article  Google Scholar 

  15. Lacey G, Namara SM, Dawson-Howe KM (1998) Personal adaptive mobility aid for the infirm and elderly blind. In: Mittal VO, Yanco HA, Aronis J, Simpson R (eds) Assistive technology and artificial intelligence: applications in robotics, user interfaces and natural language processing. Springer, Berlin, Heidelberg, pp 211–220. https://doi.org/10.1007/BFb0055980

    Chapter  Google Scholar 

  16. Lavalle SM (1998) Rapidly-exploring random trees: a new tool for path planning. Report No. TR 98-11, Computer Science Department, Iowa State University

  17. Lee G, Jung EJ, Ohnuma T, Chong NY, Yi BJ (2011) JAIST robotic walker control based on a two-layered Kalman filter. In: 2011 IEEE international conference on robotics and automation (ICRA)

  18. Lopes A, Rodrigues J, Perdigao J, Pires G, Nunes U (2016) A new hybrid motion planner: applied in a brain-actuated robotic wheelchair. IEEE Robot Autom Mag 23(4):82–93

    Article  Google Scholar 

  19. Lutz W, Sanderson W, Scherbov S (2008) The coming acceleration of global population ageing. Nature 451(7179):716–719

    Article  Google Scholar 

  20. Makarenko, AA, Williams SB, Bourgault F, Durrant-Whyte HF (2002) An experiment in integrated exploration. In: IEEE/RSJ international conference on intelligent robots and systems, 2002, vol 1, pp 534–539

  21. McLachlan S, Arblaster J, Liu D, Miro JV, Chenoweth L (2005) A multi-stage shared control method for an intelligent mobility assistant. In: 9th international conference on rehabilitation robotics, 2005. ICORR 2005, pp 426–429

  22. Morris A, Donamukkala R, Kapuria A, Steinfeld A, Matthews JT, Dunbar-Jacob J, Thrun, S (2003) A robotic walker that provides guidance. In: 2003 IEEE international conference on robotics and automation, vol 1, pp 25–30

  23. Mou WH, Chang MF, Liao CK, Hsu YH, Tseng SH, Fu LC (2012) Context-aware assisted interactive robotic walker for Parkinson’s disease patients. In: 2012 IEEE/RSJ international conference on intelligent robots and systems (IROS)

  24. Moustris GP, Geravand M, Tzafestas C, Peer A (2016) User-adaptive shared control in a mobility assistance robot based on human-centered intention reading and decision making scheme. In: IEEE ICRA workshop: human–robot interfaces for enhanced physical interactions

  25. Paulo J, Peixoto P, Nunes U (2017) ISR-AIWALKER: robotic walker for intuitive and safe mobility assistance and gait analysis. IEEE Trans Hum Mach Syst 47:1110–1122

    Article  Google Scholar 

  26. Pérez-Higueras N, Caballero F, Merino L (2018) Teaching robot navigation behaviors to optimal RRT planners. Int J Soc Robot 10(2):235–249

    Article  Google Scholar 

  27. Ramrattan RS, Wolfs RC, Panda-Jonas S, Jonas JB, Bakker D, Pols HA, Hofman A, de Jong PT (2001) Prevalence and causes of visual field loss in the elderly and associations with impairment in daily functioning: the Rotterdam study. Arch Ophthalmol 119(12):1788–1794

    Article  Google Scholar 

  28. Russell S, Norvig P (2009) Artificial intelligence: a modern approach, 3rd edn. Prentice Hall Press, Upper Saddle River

    MATH  Google Scholar 

  29. Simpson R, Boninger ML (2008) Clinical evaluation of guido robotic walker. J Rehabil Res Dev 45(9):1281

    Article  Google Scholar 

  30. Simpson R, LoPresti E, Cooper R (2008) How many people would benefit from a smart wheelchair? J Rehabil Res Dev 45(1):53–72

    Article  Google Scholar 

  31. Truong XT, Ngo TD (2016) Dynamic social zone based mobile robot navigation for human comfortable safety in social environments. Int J Soc Robot 8(5):663–684

    Article  Google Scholar 

  32. Wachaja A, Agarwal P, Zink M, Adame MR, Möller K, Burgard W (2017) Navigating blind people with walking impairments using a smart walker. Auton Robots 41(3):555–573

    Article  Google Scholar 

  33. Xu W, Huang J, Yan Q (2015) Multi-sensor based human motion intention recognition algorithm for walking-aid robot. In: 2015 IEEE international conference on robotics and biomimetics (ROBIO), pp 2041–2046

  34. Yu KT, Lam CP, Chang MF, Mou WH, Tseng SH, Fu LC (2010) An interactive robotic walker for assisting elderly mobility in senior care unit. In: 2010 IEEE workshop on advanced robotics and its social impacts (ARSO)

Download references

Acknowledgements

This work was supported by the Portuguese Foundation for Science and Technology (FCT) under the Ph.D. Grant with reference SFRH/BD/88459/2012 with funds from QREN–POPH and the European Social Fund from the European Union. It was also partially supported by the Project funded by FCT with reference FCT/COMPETE 2020/P2020 through Grants UID/EEA/00048/2013 and MATIS - CENTRO-01-0145-FEDER-000014, Portugal.

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, Institute of Systems and Robotics, University of Coimbra, Coimbra, Portugal

    Luís Garrote, João Paulo & Urbano J. Nunes

Authors
  1. Luís Garrote
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. João Paulo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Urbano J. Nunes
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Luís Garrote.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Garrote, L., Paulo, J. & Nunes, U.J. Reinforcement Learning Aided Robot-Assisted Navigation: A Utility and RRT Two-Stage Approach. Int J of Soc Robotics 12, 689–707 (2020). https://doi.org/10.1007/s12369-019-00585-0

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12369-019-00585-0

Keywords

Search

Navigation