skip to main content
10.1145/3234804.3234815acmotherconferencesArticle/Chapter ViewAbstractPublication PagesicdltConference Proceedingsconference-collections
research-article

Deep Lang: A Language Learning App Empowered by Deep Learning

Published: 27 June 2018 Publication History

Abstract

With the amazing advancement of mobile technology, people can easily learn a new language from their mobile devices. Exploring popular language learning apps, we discovered that speech and handwriting recognition functionalities are lacking in these apps. To address this issue, we designed and implemented a deep learning-based language learning Android app that provides both speech recognition and handwriting recognition. We created the speech recognition module based on the Google Speech API, which supports speech recognition in 110 languages. For handwriting recognition, we created Convolutional Neural Networks (CNN) models that support dictation/handwriting training in Chinese and Japanese respectively.

References

[1]
https://www.duolingo.com/
[2]
https://www.babbel.com/
[3]
https://www.memrise.com/
[4]
https://www.rosettastone.com/
[5]
https://www.rocketlanguages.com/
[6]
Tang, Q. The Effectiveness of Dictation Method in College English Vocabulary Teaching.Theory & Practice in Language Studies. Jul2012, Vol. 2 Issue 7, p1472.
[7]
Smith, S. Becoming an Outstanding Language Teacher. Routledge. 2017.
[8]
A. Krizhevsky, I. Sutskever and G.E. Hinton. Imagenetclassification with deep convolutional neural networks.Advances in neural information processing systems, 1097--1105, 2012.
[9]
Kimonyan, K. and Zisserman, A. Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv Technical report, 2014. arXic:1409.1556{cs.CV}
[10]
C., Szegedy et al. Going Deeper with Convolutions. In Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston, MA, 2015. pp 1--9.
[11]
Ciresan, D., Meier, U., Masci, J., Gambardella, L., and Schmidhuber, J. Flexible, High Performance Convolutional Neural Networks for Image Classification. In Proceedings of International Joint Conference Aritificial Intelligence (IJCAI), pp. 1237--1242, 2011.
[12]
Yang, W., Jin, L., Xie, Z., and Feng, Z. Improved Deep Convolutional Neural Network for Online Handwritten Chinese Character Recognition Using Domain-Specific Knowledge. In Proceedings of International Conference Document Analysis and Recognition (ICDAR), 2015.
[13]
Zhong, Z., Jin, L., and Xie, Z. High Performance Offline Handwritten Chinese Character Recognition Using GoogleNet and Directional Feature Maps. In Proceeding of International Conference Document Analysis and Recognition (ICDAR), 2015.
[14]
https://github.com/IBM/tensorflow-hangul-recognition
[15]
https://github.com/KyotoSunshine/CNN-for-handwritten-kanji
[16]
Tsai, C. Recognizing Handwritten Japanese Characters Using Deep Convolutional Neural Networks. Technical report.
[17]
C. L. Liu, F. Yin, D. H. Wang and Q. F. Wang, CASIA Online and Offline Chinese Handwriting Databases. 2011 International Conference on Document Analysis and Recognition. Beijing, 2011, pp. 37--41.
[18]
http://www.nlpr.ia.ac.cn/databases/handwriting/Download.html
[19]
https://cloud.google.com/speech/
[20]
LeCun, Y., Bottou, L., Bengio, Y., and P. Haffner. Gradient-based learning applied to document recognition. In Proceedings of the IEEE, 86(11):2278--2324, 1998.
[21]
https://keras.io/
[22]
https://www.tensorflow.org/
[23]
https://cloud.google.com/
[24]
http://etlcdb.db.aist.go.jp/

Index Terms

  1. Deep Lang: A Language Learning App Empowered by Deep Learning

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Other conferences
      ICDLT '18: Proceedings of the 2018 2nd International Conference on Deep Learning Technologies
      June 2018
      112 pages
      ISBN:9781450364737
      DOI:10.1145/3234804
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      In-Cooperation

      • Chongqing University of Posts and Telecommunications
      • University of Electronic Science and Technology of China: University of Electronic Science and Technology of China

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 27 June 2018

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. convolutional neural network
      2. deep learning
      3. handwriting recognition
      4. speech recognition

      Qualifiers

      • Research-article
      • Research
      • Refereed limited

      Conference

      ICDLT '18

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • 0
        Total Citations
      • 221
        Total Downloads
      • Downloads (Last 12 months)12
      • Downloads (Last 6 weeks)1
      Reflects downloads up to 24 Jan 2025

      Other Metrics

      Citations

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media