Skip to main content
SpringerLink
Log in

Developing a way-finding system on mobile robot assisting visually impaired people in an indoor environment

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

A way-finding system in an indoor environment consists of several components: localization, representation, path planning, and interaction. For each component, numerous relevant techniques have been proposed. However, deploying feasible techniques, particularly in real scenarios, remains challenging. In this paper, we describe a functional way-finding system deployed on a mobile robot to assist visual impairments (VI). The proposed system deploys state-of-the-art techniques that are adapted to the practical issues at hand. First, we adapt an outdoor visual odometry technique to indoor use by covering manual markers or stickers on ground-planes. The main purpose is to build reliable travel routes in the environment. Second, we propose a procedure to define and optimize the landmark/representative scenes of the environment. This technique handles the repetitive and ambiguous structures of the environment. In order to interact with VI people, we deploy a convenient interface on a smart phone. Three different indoor scenarios and thirteen subjects are conducted in our evaluations. Our experimental results show that VI people, particularly VI pupils, can find the right way to requested targets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Similar content being viewed by others

Notes

  1. http://www.whiteboxrobotics.com/Product/spec.html

  2. https://code.google.com/p/openfabmap/

  3. http://www.axis.com/global/sv/products/axis-207

  4. Nguyen Dinh Chieu Blind School, Hanoi

  5. 5nd floor of Ta Quang Buu Library, HUST

  6. 10th International Research Institute MICA, HUST

References

  1. Alcantarilla FP (2011) Vision based localization: from humanoid robots to visually impaired people. Ph.D Thesis

  2. Bailey T, Durrant-Whyte H (2006) Simultaneous localization and mapping (slam): part ii. IEEE Robot Autom Mag 13(3):108–117

    Article  Google Scholar 

  3. Bigham J, Jayant C, Miller A (2010) White: Vizwiz::locateit - enabling blind people to locate objects in their environment. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp 65–72

  4. Chow C, Liu C (1968) Approximating discrete probability distributions with dependence trees. IEEE Trans Inf Theory 14(3):462–467

    Article  MATH  Google Scholar 

  5. Cummins M, Newman P (2008) Fab-map: Probabilistic localization and mapping in the space of apperance. Int J Robot Res 27(6):647–665

    Article  Google Scholar 

  6. Dakopoulos D, Bourbakis NG (2010) Wearable obstacle avoidance electronic travel aids for blind: a survey. IEEE Trans Syst, Man, Cybernet Part C: Appl Rev 40 (1):25–35

    Article  Google Scholar 

  7. Endres H, Feiten W, Lawitzky G (1998) Field test of a navigation system: Autonomous cleaning in supermarkets. In: the Proceeding of International Conference on Robotics and Automation. IEEE

  8. Fallah N, Apostolopoulos I, Bekris K, Folmer E (2013) Indoor human navigation systems - a survey. Interact Comput 25(1):21–33

    Google Scholar 

  9. Fraundorfer F, Scaramuzza D (2012) Visual odometry : Part ii: Matching, robustness, optimization, and applications. IEEE Robot Autom Mag 19(2):78–90

    Article  Google Scholar 

  10. Hamme D, Veelaert P (2011) Robust visual odometry using uncertainty models. Proc Adv Concepts Intell Vis Syst 6915:1–12

    Google Scholar 

  11. Hart PE, Nilsson NJ, Raphael B (1968) A formal basis for the heuristic determination of minimum cost paths. IEEE Trans Syst Sci Cybern 4(2):100–107

    Article  Google Scholar 

  12. Helal A, Moore SE, Ramachandran B (2001) Drishti: An integrated navigation system for visually impaired and disabled. In: Proceedings. Fifth International Symposium on Wearable Computers, 2001. IEEE, pp 149–156

  13. King S, Weiman C (1990) Helpmate autonomous mobile robot navigation system. In: the Proceeding of the SPIE Conference on Mobile Robots, pp 190–198

  14. Korf RE (1985) Iterative-deepening-a: an optimal admissible tree search. In: Proceedings of the 9th international joint conference on Artificial intelligence-Volume 2. Morgan Kaufmann Publishers Inc, pp 1034–1036

  15. Kulyukin V, Gharpure C (2006) Nicholson: Robot assisted way-finding for the visually impaired in structured indoor environments. Auton Robot 21:29–41

    Article  Google Scholar 

  16. Kulyukin V, Gharpure C, Nicholson J, Pavithran S (2004) Rfid in robot-assisted indoor navigation for the visually impaired. In: Proceedings. 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2004. (IROS 2004), vol 2. IEEE, pp 1979–1984

  17. Lacey G, Dawson-Howe K (1998) The application of robotics to a mobility aid for the elderly blind. Robot Auton Syst 23:245–252

    Article  Google Scholar 

  18. LaMarca A, Brunette W, Koizumi D (2002) Making sensor networks practical with robots. In: the Proceeding of International Conference on Pervasive Computing. IEEE

  19. Lehel P, Hemayed E, Farag A (1999) Robot assisted way-finding for the visually impaired in structured indoor environments. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol 2

  20. Liu JJ, Phillips C, Daniilidis K (2010) Video-based localization without 3d mapping for the visually impaired. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). IEEE, pp 23–30

  21. Loomis JM, Golledge RD (2001) Klatzky: Gps-based navigation systems for the visually impaired. Fundamentals of wearable computers and augmented reality, pp 429–446

  22. Marion AH, Micheal AJ (2008) Assistive Technology for Visually Impaired and Blind People. Springer

  23. Murali VN, Coughlan JM (2013) Smartphone-based crosswalk detection and localization for visually impaired pedestrians. In: 2013 IEEE International Conference on Multimedia and Expo Workshops (ICMEW). IEEE, pp 1–7

  24. Newman P, Ho K (2005) Slam-loop closing with visually salient features. In: Proceedings of the 2005 IEEE International Conference on Robotics and Automation, 2005. ICRA 2005. IEEE, pp 635–642

  25. Nister D, Stewenius H (2006) Scalable recognition with a vocabulary tree. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol 2. IEEE, pp 2161–2168

  26. Pradeep V, Medioni G, Weiland J (2010) Robot vision for the visually impaired. In: Computer Vision and Pattern Recognition Workshops (CVPRW), 2010 IEEE Computer Society Conference on. IEEE, pp 15–22

  27. Pradeep V, Medioni G, Weiland J (2010) Robot vision for the visually impaired. In: Computer Vision and Pattern Recognition Workshops (CVPRW), 2010 IEEE Computer Society Conference on. IEEE, pp 15–22

  28. Schindler G, Brown M, Szeliski R (2007) City-scale location recognition. In: IEEE Conference on Computer Vision and Pattern Recognition, 2007. CVPR ’07, pp 1–7

  29. Sivic J, Zisserman A (2003) Video google: a text retrieval approach to object matching in videos. In: Proceedings. Ninth IEEE International Conference on Computer Vision, 2003, vol 2, pp 1470–1477

  30. Sunderhauf N, Protzel P (2011) Brief-gist-closing the loop by simple means. In: 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, pp 1234–1241

  31. Winlock T, Christiansen E, Belongie S (2010) Toward real-time grocery detection for the visually impaired. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). IEEE, pp 49–56

Download references

Acknowledgments

This research is funded by Vietnam National Foundation for Science and Technology Development (NAFOSTED) under grant number FWO.102.2013.08.

Author information

Authors and Affiliations

  1. International Research Institute MICA, HUST - CNRS/UMI - 2954 - INP Grenoble, Hanoi University of Science and Technology, HaNoi, Vietnam

    Quoc-Hung Nguyen, Hai Vu & Thanh-Hai Tran

  2. Thai Nguyen Medical College, Thai Nguyen, Vietnam

    Quoc-Hung Nguyen

  3. Hung Yen University of Technology and Education, Hung Yen, Vietnam

    Quang-Hoan Nguyen

Authors
  1. Quoc-Hung Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hai Vu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Thanh-Hai Tran
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Quang-Hoan Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Quoc-Hung Nguyen.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nguyen, QH., Vu, H., Tran, TH. et al. Developing a way-finding system on mobile robot assisting visually impaired people in an indoor environment. Multimed Tools Appl 76, 2645–2669 (2017). https://doi.org/10.1007/s11042-015-3204-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-015-3204-2

Keywords

Search

Navigation