Skip to main content
Springer Nature Link
Log in

Empowering Individuals with Visual Impairments: A Deep Learning-Based Smartphone Navigation Assistant

  • Conference paper
  • First Online:
Proceedings of the 9th International Conference on Advanced Intelligent Systems and Informatics 2023 (AISI 2023)

Part of the book series: Lecture Notes on Data Engineering and Communications Technologies ((LNDECT,volume 184))

  • 749 Accesses

Abstract

Long white canes and other technological aids are frequently used by visually impaired people to identify and avoid obstacles ahead and hazards. Due to their general knowledge of everything’s location, visually impaired individuals can move freely in their homes without much assistance. However, they encounter more challenges and risk harm when they roam the streets. To help visually impaired individuals navigate the streets independently and safely, this paper proposes a deep learning-based smartphone navigation assistant system. The backend and frontend are the two main components. On the front end, the images are captured by utilizing the mobile camera. The backend is fed with these captured images. A You Only Look Once (YOLOv8) deep learning architecture is used in the backend, followed by a rule-based model. Finally, a set of pre-recorded audio messages that contain navigational guidance is returned to the user. The deep-learning architecture is trained and fine-tuned on a dataset gathered from five different sources. The experimental results showed that the proposed model can be effectively used to help people who are blind. Additionally, the outcomes demonstrated that YOLOv8 achieved the best outcomes when compared to other deep-learning architectures. The proposed system achieved a 97% overall accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. WHO: Visual impairment and blindness, Fact Sheet n-282. https://www.who.int/en/news-room/fact-sheets/detail/blindness-and-visual-impairment. Accessed 17 Dec 2022

  2. Ackland, P., Resnikoff, S., Bourne, R.: World blindness and visual impairment: despite many successes, the problem is growing. Community Eye Health 30, 71–73 (2018). PMC5820628, PMID: 29483748

    Google Scholar 

  3. Kuriakose, B., Shrestha, R., Sandnes, F.E.: Tools and technologies for blind and visually impaired navigation support: a review. IETE Tech. Rev. 39(1), 3–18 (2022). https://doi.org/10.1080/02564602.2020.1819893

    Article  Google Scholar 

  4. Walle, H., De Runz, C., Serres, B., Venturini, G.: A survey on recent advances in AI and vision-based methods for helping and guiding visually impaired people. Appl. Sci. 12(5), 2308 (2022). https://doi.org/10.3390/app12052308

    Article  Google Scholar 

  5. Wang, J., Wang, S., Zhang, Y.: Artificial intelligence for visually impaired. Displays 77, 102391 (2023). https://doi.org/10.1016/j.displa.2023.102391

    Article  Google Scholar 

  6. Zhao, Z.-Q., Zheng, P., Xu, S.-T., Wu, X.: Object detection with deep learning: a review. IEEE Trans. Neural Netw. Learn. Syst. 30, 3212–3232 (2019). https://doi.org/10.1109/TNNLS.2018.2876865

    Article  Google Scholar 

  7. Wu, X., Sahoo, D., Hoi, S.C.: Recent advances in deep learning for object detection. Neurocomputing 396, 39–64 (2020). https://doi.org/10.1016/j.neucom.2020.01.085

    Article  Google Scholar 

  8. Srivastava, S., Divekar, A.V., Anilkumar, C., Naik, I., Kulkarni, V., Pattabiraman, V.: Comparative analysis of deep learning image detection algorithms. J. Big Data 8, 66 (2021). https://doi.org/10.1186/s40537-021-00434-w

    Article  Google Scholar 

  9. Kuriakose, B., Shrestha, R., Sandnes, F.E.: Tools and technologies for blind and visually impaired navigation support: a review. In: IETE Technical Review, VOL.39, No. 1, from pp. 3–18, September 2022. https://doi.org/10.1080/02564602.2020.1819893

  10. Ceccarini, C., Prandi, C.: Tourism for all: a mobile application to assist visually impaired users in enjoying tourist services. In: 2019 16th IEEE Annual Consumer Communications & Networking Conference (CCNC), Las Vegas, NV, USA, pp. 1–6 (2019). https://doi.org/10.1109/CCNC.2019.8651848

  11. Khan, M.A., Paul, P., Rashid, M., Hossain, M., Ahad, M.A.R.: An AI-based visual aid with integrated reading assistant for the completely blind. IEEE Trans. Hum. Mach. Syst. 50(6), 507–517 (2020). https://doi.org/10.1109/THMS.2020.3027534

    Article  Google Scholar 

  12. Chang, W.-J., Chen, L.-B., Sie, C.-Y., Yang, C.-H.: An artificial intelligence edge computing-based assistive system for visually impaired pedestrian safety at zebra crossings. IEEE Trans. Consum. Electron. 67(1), 3–11 (2021). https://doi.org/10.1109/TCE.2020.3037065

    Article  Google Scholar 

  13. Joshi, R.C., Yadav, S., Dutta, M., Travieso, C.: Efficient multi-object detection and smart navigation using artificial intelligence for visually impaired people. Entropy 22(9), 941 (2020). https://doi.org/10.3390/e22090941

    Article  MathSciNet  Google Scholar 

  14. Hossain, M.E., Qaiduzzaman, K.M., Rahman, M.: Sightless helper: an interactive mobile application for blind assistance and safe navigation. Lect. Notes Inst. Comput. Sci., Soc. Inf. Telecommun. Eng. 325, 581–592 (2020). https://doi.org/10.1007/978-3-030-52856-0_46

    Article  Google Scholar 

  15. Lin, B.-S., Lee, C.-C., Chiang, P.-Y.: Smart mobility for blind people using deep learning and Internet of Things. Sensors 17(6), 1371 (2017). https://doi.org/10.3390/s17061371

    Article  Google Scholar 

  16. Nasreen, J., Warsi, A., Shaikh, A.A., Muhammad, Y., Abdullah, M.: Object detection and narrator for visually impaired people. In: 2019 IEEE 6th International Conference on Engineering Technologies and Applied Sciences (ICETAS) (2019). https://doi.org/10.1109/ICETAS48360.2019.9117405

  17. Awad, M., El Haddad, J., Khneisser, E., Mahmoud, T., Yaacoub, E., Malli, M.: Intelligent eye: a mobile application for assisting blind people. In: 2018 IEEE Middle East and North Africa Communications Conference (MENACOMM) (2019). https://doi.org/10.1109/MENACOMM.2018.8371005

  18. Vaduva, I.-A.: Garbage classification (2021). https://www.kaggle.com/datasets/ionutandreivaduva/garbage-classification. Accessed 19 Feb 2023

  19. Abla, M.: Garbage classification (2020). https://www.kaggle.com/datasets/mostafaabla/garbage-classification. Accessed 9 Jan 2023

  20. Cai, Y., Tan, X.: Weakly supervised human body detection under arbitrary poses. In: International Conference on Image Processing. IEEE (2016). Accessed 9 Apr 2023

    Google Scholar 

  21. Rana, K.: Vehicle Dataset (2021). https://www.kaggle.com/datasets/krishrana/vehicle-dataset. Accessed 9 Apr 2023

  22. Prize, E.: Tree binary segmentation (2021). https://www.kaggle.com/datasets/earthshot/tree-binary-segmentation. Accessed 9 Apr 2023

  23. Jiang, X., Gao, T., Zhu, Z., Zhao, Y.: Real-time face mask detection method based on YOLOv3. Electronics 10(7), 837 (2021). https://doi.org/10.3390/electronics10070837

    Article  Google Scholar 

  24. Liu, R., Ren, Z.: Application of Yolo on mask detection task. In: 2021 IEEE 13th International Conference on Computer Research and Development (ICCRD), Beijing, China, pp. 130–136 (2021). https://doi.org/10.1109/ICCRD51685.2021.9386366

  25. Wahyutama, A.B., Hwang, M.: YOLO-based object detection for separate collection of recyclables and capacity monitoring of trash bins. Electronics 11(9), 1323 (2022). https://doi.org/10.3390/electronics11091323

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computer Science, Canadian International College (CIC), New Cairo, Cairo, Egypt

    Fatema A. Shawki, Mariem Mahfouz, Mohamed A. Abdelrazek & Gehad Ismail Sayed

Authors
  1. Fatema A. Shawki
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mariem Mahfouz
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mohamed A. Abdelrazek
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Gehad Ismail Sayed
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gehad Ismail Sayed .

Editor information

Editors and Affiliations

  1. Faculty of Computers and Information, Cairo University, Giza, Egypt

    AboulElla Hassanien

  2. Port Said University, Port Fouad City, Egypt

    Rawya Y. Rizk

  3. Department of Operations Research and Statistics, The University of Belgrade, Faculty of Organizational Sciences, Belgrade, Serbia

    Dragan Pamucar

  4. Faculty of Science, Helwan University, Cairo, Egypt

    Ashraf Darwish

  5. Shulin District, Fujian University of Technology, New Taipei, Taiwan

    Kuo-Chi Chang

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Shawki, F.A., Mahfouz, M., Abdelrazek, M.A., Sayed, G.I. (2023). Empowering Individuals with Visual Impairments: A Deep Learning-Based Smartphone Navigation Assistant. In: Hassanien, A., Rizk, R.Y., Pamucar, D., Darwish, A., Chang, KC. (eds) Proceedings of the 9th International Conference on Advanced Intelligent Systems and Informatics 2023. AISI 2023. Lecture Notes on Data Engineering and Communications Technologies, vol 184. Springer, Cham. https://doi.org/10.1007/978-3-031-43247-7_2

Download citation

Publish with us

Policies and ethics