Abstract
Long white canes and other technological aids are frequently used by visually impaired people to identify and avoid obstacles ahead and hazards. Due to their general knowledge of everything’s location, visually impaired individuals can move freely in their homes without much assistance. However, they encounter more challenges and risk harm when they roam the streets. To help visually impaired individuals navigate the streets independently and safely, this paper proposes a deep learning-based smartphone navigation assistant system. The backend and frontend are the two main components. On the front end, the images are captured by utilizing the mobile camera. The backend is fed with these captured images. A You Only Look Once (YOLOv8) deep learning architecture is used in the backend, followed by a rule-based model. Finally, a set of pre-recorded audio messages that contain navigational guidance is returned to the user. The deep-learning architecture is trained and fine-tuned on a dataset gathered from five different sources. The experimental results showed that the proposed model can be effectively used to help people who are blind. Additionally, the outcomes demonstrated that YOLOv8 achieved the best outcomes when compared to other deep-learning architectures. The proposed system achieved a 97% overall accuracy.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
WHO: Visual impairment and blindness, Fact Sheet n-282. https://www.who.int/en/news-room/fact-sheets/detail/blindness-and-visual-impairment. Accessed 17 Dec 2022
Ackland, P., Resnikoff, S., Bourne, R.: World blindness and visual impairment: despite many successes, the problem is growing. Community Eye Health 30, 71–73 (2018). PMC5820628, PMID: 29483748
Kuriakose, B., Shrestha, R., Sandnes, F.E.: Tools and technologies for blind and visually impaired navigation support: a review. IETE Tech. Rev. 39(1), 3–18 (2022). https://doi.org/10.1080/02564602.2020.1819893
Walle, H., De Runz, C., Serres, B., Venturini, G.: A survey on recent advances in AI and vision-based methods for helping and guiding visually impaired people. Appl. Sci. 12(5), 2308 (2022). https://doi.org/10.3390/app12052308
Wang, J., Wang, S., Zhang, Y.: Artificial intelligence for visually impaired. Displays 77, 102391 (2023). https://doi.org/10.1016/j.displa.2023.102391
Zhao, Z.-Q., Zheng, P., Xu, S.-T., Wu, X.: Object detection with deep learning: a review. IEEE Trans. Neural Netw. Learn. Syst. 30, 3212–3232 (2019). https://doi.org/10.1109/TNNLS.2018.2876865
Wu, X., Sahoo, D., Hoi, S.C.: Recent advances in deep learning for object detection. Neurocomputing 396, 39–64 (2020). https://doi.org/10.1016/j.neucom.2020.01.085
Srivastava, S., Divekar, A.V., Anilkumar, C., Naik, I., Kulkarni, V., Pattabiraman, V.: Comparative analysis of deep learning image detection algorithms. J. Big Data 8, 66 (2021). https://doi.org/10.1186/s40537-021-00434-w
Kuriakose, B., Shrestha, R., Sandnes, F.E.: Tools and technologies for blind and visually impaired navigation support: a review. In: IETE Technical Review, VOL.39, No. 1, from pp. 3–18, September 2022. https://doi.org/10.1080/02564602.2020.1819893
Ceccarini, C., Prandi, C.: Tourism for all: a mobile application to assist visually impaired users in enjoying tourist services. In: 2019 16th IEEE Annual Consumer Communications & Networking Conference (CCNC), Las Vegas, NV, USA, pp. 1–6 (2019). https://doi.org/10.1109/CCNC.2019.8651848
Khan, M.A., Paul, P., Rashid, M., Hossain, M., Ahad, M.A.R.: An AI-based visual aid with integrated reading assistant for the completely blind. IEEE Trans. Hum. Mach. Syst. 50(6), 507–517 (2020). https://doi.org/10.1109/THMS.2020.3027534
Chang, W.-J., Chen, L.-B., Sie, C.-Y., Yang, C.-H.: An artificial intelligence edge computing-based assistive system for visually impaired pedestrian safety at zebra crossings. IEEE Trans. Consum. Electron. 67(1), 3–11 (2021). https://doi.org/10.1109/TCE.2020.3037065
Joshi, R.C., Yadav, S., Dutta, M., Travieso, C.: Efficient multi-object detection and smart navigation using artificial intelligence for visually impaired people. Entropy 22(9), 941 (2020). https://doi.org/10.3390/e22090941
Hossain, M.E., Qaiduzzaman, K.M., Rahman, M.: Sightless helper: an interactive mobile application for blind assistance and safe navigation. Lect. Notes Inst. Comput. Sci., Soc. Inf. Telecommun. Eng. 325, 581–592 (2020). https://doi.org/10.1007/978-3-030-52856-0_46
Lin, B.-S., Lee, C.-C., Chiang, P.-Y.: Smart mobility for blind people using deep learning and Internet of Things. Sensors 17(6), 1371 (2017). https://doi.org/10.3390/s17061371
Nasreen, J., Warsi, A., Shaikh, A.A., Muhammad, Y., Abdullah, M.: Object detection and narrator for visually impaired people. In: 2019 IEEE 6th International Conference on Engineering Technologies and Applied Sciences (ICETAS) (2019). https://doi.org/10.1109/ICETAS48360.2019.9117405
Awad, M., El Haddad, J., Khneisser, E., Mahmoud, T., Yaacoub, E., Malli, M.: Intelligent eye: a mobile application for assisting blind people. In: 2018 IEEE Middle East and North Africa Communications Conference (MENACOMM) (2019). https://doi.org/10.1109/MENACOMM.2018.8371005
Vaduva, I.-A.: Garbage classification (2021). https://www.kaggle.com/datasets/ionutandreivaduva/garbage-classification. Accessed 19 Feb 2023
Abla, M.: Garbage classification (2020). https://www.kaggle.com/datasets/mostafaabla/garbage-classification. Accessed 9 Jan 2023
Cai, Y., Tan, X.: Weakly supervised human body detection under arbitrary poses. In: International Conference on Image Processing. IEEE (2016). Accessed 9 Apr 2023
Rana, K.: Vehicle Dataset (2021). https://www.kaggle.com/datasets/krishrana/vehicle-dataset. Accessed 9 Apr 2023
Prize, E.: Tree binary segmentation (2021). https://www.kaggle.com/datasets/earthshot/tree-binary-segmentation. Accessed 9 Apr 2023
Jiang, X., Gao, T., Zhu, Z., Zhao, Y.: Real-time face mask detection method based on YOLOv3. Electronics 10(7), 837 (2021). https://doi.org/10.3390/electronics10070837
Liu, R., Ren, Z.: Application of Yolo on mask detection task. In: 2021 IEEE 13th International Conference on Computer Research and Development (ICCRD), Beijing, China, pp. 130–136 (2021). https://doi.org/10.1109/ICCRD51685.2021.9386366
Wahyutama, A.B., Hwang, M.: YOLO-based object detection for separate collection of recyclables and capacity monitoring of trash bins. Electronics 11(9), 1323 (2022). https://doi.org/10.3390/electronics11091323
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Shawki, F.A., Mahfouz, M., Abdelrazek, M.A., Sayed, G.I. (2023). Empowering Individuals with Visual Impairments: A Deep Learning-Based Smartphone Navigation Assistant. In: Hassanien, A., Rizk, R.Y., Pamucar, D., Darwish, A., Chang, KC. (eds) Proceedings of the 9th International Conference on Advanced Intelligent Systems and Informatics 2023. AISI 2023. Lecture Notes on Data Engineering and Communications Technologies, vol 184. Springer, Cham. https://doi.org/10.1007/978-3-031-43247-7_2
Download citation
DOI: https://doi.org/10.1007/978-3-031-43247-7_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-43246-0
Online ISBN: 978-3-031-43247-7
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)