skip to main content
10.1145/3197768.3201529acmotherconferencesArticle/Chapter ViewAbstractPublication PagespetraConference Proceedingsconference-collections
research-article

Automata Classification with Convolutional Neural Networks for Use in Assistive Technologies for the Visually Impaired

Published: 26 June 2018 Publication History

Abstract

Our goal is to evaluate the use of Convolutional Neural Networks (CNN) in the recognition of automata images and to create a model that can be used in the construction of assistive tools. Visually impaired individuals that are studying Computer Science have difficulty in accessing and learning diagrams. Despite the solutions available in the literature to make diagrams accessible to blind students and allow the creation and manipulation of such material, we seek to give access to images of didactic materials and books. The method used consists of two steps: classification of the data using three types of CNN and the combination of the results to make a final decision. Two approaches were chosen to be tested: recognition of the type of automaton and recognition of the number of states of the automaton. Our best result was using late fusion of the three CNNs by the product rule, which resulted in an accuracy of 97% for the automaton type recognition and 91% for the recognition of the number of states of the automaton.

References

[1]
Suzanne P Balik, Sean P Mealin, Matthias F Stallmann, and Robert D Rodman. 2013. GSK: universally accessible graph sketching. In Proceeding of the 44th ACM technical symposium on Computer science education. ACM, 221--226.
[2]
Michael Banf and Volker Blanz. 2013. Man made structure detection and verification of object recognition in images for the visually impaired. In Proceedings of the 6th International Conference on Computer Vision/Computer Graphics Collaboration Techniques and Applications. ACM, 18.
[3]
Arne Bernin, Larissa Müller, Sobin Ghose, Kai von Luck, Christos Grecos, Qi Wang, and Florian Vogt. 2017. Towards More Robust Automatic Facial Expression Recognition in Smart Environments. In Proceedings of the 10th International Conference on PErvasive Technologies Related to Assistive Environments. ACM, 37--44.
[4]
Matt Calder, Robert F Cohen, Jessica Lanzoni, and Yun Xu. 2006. PLUMB:: an interface for users who are blind to display, create, and modify graphs. In Proceedings of the 8th international ACM SIGACCESS conference on Computers and accessibility. ACM, 263--264.
[5]
Robert F Cohen, Arthur Meacham, and Joelle Skaff. 2006. Teaching graphs to visually impaired students using an active auditory interface. In ACM SIGCSE Bulletin, Vol. 38. ACM, 279--282.
[6]
Pierluigi Crescenzi, Leonardo Rossi, and Gianluca Apollaro. 2012. Making turing machines accessible to blind students. In Proceedings of the 43rd ACM technical symposium on Computer Science Education. ACM, 167--172.
[7]
Joan M Francioni and Ann C Smith. 2002. Computer science accessibility for students with visual disabilities. In ACM SIGCSE Bulletin, Vol. 34. ACM, 91--95.
[8]
Richa Gupta, M Balakrishnan, and PVM Rao. 2017. Tactile Diagrams for the Visually Impaired. IEEE Potentials 36, 1 (2017), 14--18.
[9]
Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).
[10]
Josef Kittler, Mohamad Hatef, Robert PW Duin, and Jiri Matas. 1998. On combining classifiers. IEEE transactions on pattern analysis and machine intelligence 20, 3 (1998), 226--239.
[11]
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems. 1097--1105.
[12]
Rahul Kumar and Sukadev Meher. 2015. A Novel method for visually impaired using object recognition. In Communications and Signal Processing (ICCSP), 2015 International Conference on. IEEE, 0772--0776.
[13]
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. 1998. Gradient-based learning applied to document recognition. Proc. IEEE 86, 11 (1998), 2278--2324.
[14]
Qiang Liu, Ruihao Li, Huosheng Hu, and Dongbing Gu. 2016. Using semantic maps for room recognition to aid visually impaired people. In Automation and Computing (ICAC), 2016 22nd International Conference on. IEEE, 89--94.
[15]
John C Martin. 1991. Introduction to Languages and the Theory of Computation. Vol. 4. McGraw-Hill NY.
[16]
Alexander J Medeiros, Lee Stearns, Leah Findlater, Chuan Chen, and Jon E Froehlich. 2017. Recognizing Clothing Colors and Visual Textures Using a Finger-Mounted Camera: An Initial Investigation. (2017).
[17]
Dorian Miller. 2009. Can We Work Together? Master's thesis. University of North Carolina at Chapel Hill, United States of America.
[18]
Peter Mittler. 2012. Working towards inclusive education: Social contexts. Routledge.
[19]
João José Neto. 2009. A Teoria da Computação e o profissional de informática. Revista de Computação e Tecnologia (ReCeT). ISSN 2176-7998 1, 1 (2009), 4--21.
[20]
Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).
[21]
Joan Sosa-García and Francesca Odone. 2017. "Hands On" Visual Recognition for Visually Impaired Users. ACM Transactions on Accessible Computing (TACCESS) 10, 3 (2017), 8.
[22]
John Stamford and Brian Peach. 2016. Scene detection using convolutional neural networks. (2016).
[23]
Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. 2015. Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1--9.
[24]
Yacine Yaddaden, Abdenour Bouzouane, Mehdi Adda, and Bruno Bouchard. 2016. A New Approach of Facial Expression Recognition for Ambient Assisted Living. In Proceedings of the 9th ACM International Conference on PErvasive Technologies Related to Assistive Environments. ACM, 14.

Cited By

View all
  • (2024)Robotic Assistant for Object Recognition Using Convolutional Neural NetworkABUAD Journal of Engineering Research and Development (AJERD)10.53982/ajerd.2024.0701.01-j7:1(1-13)Online publication date: 12-Feb-2024
  • (2024)Intelligent environments and assistive technologies for assisting visually impaired people: a systematic literature reviewUniversal Access in the Information Society10.1007/s10209-024-01117-yOnline publication date: 3-May-2024
  • (2024)Accessible learning objects: a systematic literature reviewUniversal Access in the Information Society10.1007/s10209-023-01025-723:4(1931-1945)Online publication date: 1-Nov-2024
  • Show More Cited By

Index Terms

  1. Automata Classification with Convolutional Neural Networks for Use in Assistive Technologies for the Visually Impaired

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Other conferences
      PETRA '18: Proceedings of the 11th PErvasive Technologies Related to Assistive Environments Conference
      June 2018
      591 pages
      ISBN:9781450363907
      DOI:10.1145/3197768
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      In-Cooperation

      • NSF: National Science Foundation

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 26 June 2018

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. Computer Theory
      2. Convolutional Neural Network
      3. accessibility
      4. assistive technologies
      5. visually impaired

      Qualifiers

      • Research-article
      • Research
      • Refereed limited

      Conference

      PETRA '18

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)2
      • Downloads (Last 6 weeks)0
      Reflects downloads up to 10 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)Robotic Assistant for Object Recognition Using Convolutional Neural NetworkABUAD Journal of Engineering Research and Development (AJERD)10.53982/ajerd.2024.0701.01-j7:1(1-13)Online publication date: 12-Feb-2024
      • (2024)Intelligent environments and assistive technologies for assisting visually impaired people: a systematic literature reviewUniversal Access in the Information Society10.1007/s10209-024-01117-yOnline publication date: 3-May-2024
      • (2024)Accessible learning objects: a systematic literature reviewUniversal Access in the Information Society10.1007/s10209-023-01025-723:4(1931-1945)Online publication date: 1-Nov-2024
      • (2021)Classification of UML Diagrams to Support Software Engineering Education2021 36th IEEE/ACM International Conference on Automated Software Engineering Workshops (ASEW)10.1109/ASEW52652.2021.00030(102-107)Online publication date: Nov-2021
      • (2020)MannAccess: A Novel Low Cost Assistive Educational Tool of Digital Image for Visually Impaired2020 IEEE 44th Annual Computers, Software, and Applications Conference (COMPSAC)10.1109/COMPSAC48688.2020.00023(103-112)Online publication date: Jul-2020

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media