skip to main content
10.1145/3309074.3309114acmotherconferencesArticle/Chapter ViewAbstractPublication PagesiccspConference Proceedingsconference-collections
research-article

An assistive model for visually impaired people using YOLO and MTCNN

Published: 19 January 2019 Publication History

Abstract

Visually impaired people face difficulties in safe and independent movement which deprive them from regular professional and social activities in both indoors and outdoors. Similarly they have distressin identification of surrounding environment fundamentals. This paper presents a model to detect brightness and major colors in real-time image by using RGB method by means of an external camera and then identification of fundamental objects as well as facial recognition from personal dataset. For the Object identification and Facial Recognition, YOLO Algorithm and MTCNN Networking are used, respectively. The software support is achieved by using OpenCV libraries of Python as well as implementing machine learning process. The major processor used for our model, Raspberry Pi scans and detects the facial edges via Pi camera and objects in the image are captured and recognized using mobile camera. Image recognition results are transferred to the blind users by means of text-to-speech library. The device portability is achieved by using a battery. The object detection process achieved 6-7 FPS processing with an accuracy rate of 63-80%. The face identification process achieved 80-100% accuracy.

References

[1]
Global Data on Visual Impairments 2010. Available online: http://www.who.int/blindness/GLOBALDATAFINALforweb.pdf (Aaccessed on 23 April 2017).
[2]
He K., X. Zhang, S. Ren, J. Sun, "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "Fleet D., Pajdla T., Schiele B., Tuytelaars T. (eds) Computer Vision", Lecture Notes in Computer Science, vol 8691. Springer.
[3]
Prof. P Y Kumbhar, Mohd Attaullah, S. Dhere, S. Kumar Hipparagi, "Real Time Face Detection and Tracking Using OpenCV," Int. Journal For Research In Emerging Science and Technology, Vol.4, No. 4, Apr-2017.
[4]
W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, F.Cheng-Yang, C. Alexander Berg, "SSD: Single Shot MultiBox Detector," ver. 5, 2016,Cornell University Library.
[5]
J. Redmon, S. Divvala, R. Girshick, A. Farhadi; "You Only Look Once: Unified, Real-Time Object Detection," IEEE Conference on Computer Vision and Pattern Recognition; Published on: 12 December 2016.
[6]
Joseph Redmon, Ali Farhadi; YOLO9000: Better, Faster, Stronger; 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 09 November 2017.
[7]
{Online}. https://adeshpande3.github.io/A-Beginner%27s-Guide-To-Understanding-Convolutional-Neural-Networks-Part-2/ (Accessed on 19th July 2018).
[8]
M. Nakib, R. T. Khan, M. S. Hasan and J. Uddin, "Crime Scene Prediction by Detecting Threatening Objects Using Convolutional Neural Network," Int. Conf. on Computer, Communication, Chemical, Material and Electronic Engineering, pp. 1--4, 2018.
[9]
K. Zhang, Z. Zhang, Z. Li and Y. Qiao, "Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks," in IEEE Signal Processing Letters, vol. 23, no. 10, pp. 1499--1503, Oct. 2016.
[10]
S. Billotta, G. Bonanno, S. Garozzo, A. Grillo, D. Marano, G. Romeo, "You Only Look Once:Unified, Real-Time Object Detection," Nuclear Instruments and Methods in Physics Research, Section A: Accelerators, Spectrometers, Detectors and Assoc. Equip., vol. 794, pp. 185--192, 2015.
[11]
J. Redmon, "Darknet: Open source neural networks in c." {onloine}. http://pjreddie.com/darknet/, 2013--2016.
[12]
{Online}. https://pjreddie.com/darknet/yolo/ (Accessed on: 14th July 2018).
[13]
M.S. Akbar, P. Sarkar, A.M. Ashray, A.T. Mansoor, J. Uddin, "Face Recognition and RFID Verified Attendance System," Int. Conf. on Emerging Technologies in Computing 2018, London, UK.
[14]
{Online}. https://www.raspberrypi.org/products/raspberry-pi-3-model-b/ (Accessed on : 10th July 2018).

Cited By

View all
  • (2024)Vision Sense: Real-Time Object Detection And Audio Feedback System For Visually Impaired Individuals2024 2nd World Conference on Communication & Computing (WCONF)10.1109/WCONF61366.2024.10692302(1-6)Online publication date: 12-Jul-2024
  • (2024)Deep Learning Based Techniques to Develop & Enhance Assistive Gear for Visually Impaired2024 IEEE International Conference on Computer Vision and Machine Intelligence (CVMI)10.1109/CVMI61877.2024.10782209(1-6)Online publication date: 19-Oct-2024
  • (2024)Real-Time Obstacle Detection Using YOLOv8 on Raspberry Pi 4 for Visually Challenged PeopleSmart Trends in Computing and Communications10.1007/978-981-97-1320-2_19(221-235)Online publication date: 14-Jun-2024
  • Show More Cited By

Index Terms

  1. An assistive model for visually impaired people using YOLO and MTCNN

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Other conferences
    ICCSP '19: Proceedings of the 3rd International Conference on Cryptography, Security and Privacy
    January 2019
    303 pages
    ISBN:9781450366182
    DOI:10.1145/3309074
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 19 January 2019

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. OpenCV
    2. YOLO algorithm
    3. deep learning
    4. object identification
    5. visually impaired

    Qualifiers

    • Research-article

    Conference

    ICCSP 2019

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)25
    • Downloads (Last 6 weeks)4
    Reflects downloads up to 13 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Vision Sense: Real-Time Object Detection And Audio Feedback System For Visually Impaired Individuals2024 2nd World Conference on Communication & Computing (WCONF)10.1109/WCONF61366.2024.10692302(1-6)Online publication date: 12-Jul-2024
    • (2024)Deep Learning Based Techniques to Develop & Enhance Assistive Gear for Visually Impaired2024 IEEE International Conference on Computer Vision and Machine Intelligence (CVMI)10.1109/CVMI61877.2024.10782209(1-6)Online publication date: 19-Oct-2024
    • (2024)Real-Time Obstacle Detection Using YOLOv8 on Raspberry Pi 4 for Visually Challenged PeopleSmart Trends in Computing and Communications10.1007/978-981-97-1320-2_19(221-235)Online publication date: 14-Jun-2024
    • (2023)Buddy App: Virtual Assistant For Old Aged And Visually Challenged People2023 OITS International Conference on Information Technology (OCIT)10.1109/OCIT59427.2023.10430636(861-866)Online publication date: 13-Dec-2023
    • (2023)Robust Object Detection and Tracking Model for Visually Impaired People Using Deep Convolution Neural Network Model2023 International Conference on Sustainable Communication Networks and Application (ICSCNA)10.1109/ICSCNA58489.2023.10370133(1593-1598)Online publication date: 15-Nov-2023
    • (2023)Mobility Assistive Technology with Artificial Intelligence: Indoor Navigation Assistance for the Visually Impaired Using Arduino-Based Assistive Goggles2023 IEEE 15th International Conference on Humanoid, Nanotechnology, Information Technology, Communication and Control, Environment, and Management (HNICEM)10.1109/HNICEM60674.2023.10589004(1-6)Online publication date: 19-Nov-2023
    • (2023)Assistive Object Recognition and Obstacle Detection System for the Visually Impaired Using YOLO2023 13th International Conference on Cloud Computing, Data Science & Engineering (Confluence)10.1109/Confluence56041.2023.10048808(353-358)Online publication date: 19-Jan-2023
    • (2022)Modeling of Optimal Deep Learning Enabled Object Detection and Classification on Drone Imagery2022 International Conference on Augmented Intelligence and Sustainable Systems (ICAISS)10.1109/ICAISS55157.2022.10010957(303-309)Online publication date: 24-Nov-2022
    • (2022)Low-Cost Assistive Technologies for Disabled People Using Open-Source Hardware and Software: A Systematic Literature ReviewIEEE Access10.1109/ACCESS.2022.322144910(124894-124927)Online publication date: 2022
    • (2022)Recent trends in computer vision-driven scene understanding for VI/blind users: a systematic mappingUniversal Access in the Information Society10.1007/s10209-022-00868-w22:3(983-1005)Online publication date: 6-Feb-2022
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media