Skip to main content
Springer Nature Link
Log in

Are Deep Learning Approaches Suitable for Natural Language Processing?

  • Conference paper
  • First Online:
Natural Language Processing and Information Systems (NLDB 2016)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 9612))

  • 2630 Accesses

Abstract

In recent years, Deep Learning (DL) techniques have gained much attention from Artificial Intelligence (AI) and Natural Language Processing (NLP) research communities because these approaches can often learn features from data without the need for human design or engineering interventions. In addition, DL approaches have achieved some remarkable results. In this paper, we have surveyed major recent contributions that use DL techniques for NLP tasks. All these reviewed topics have been limited to show contributions to text understanding, such as sentence modelling, sentiment classification, semantic role labelling, question answering, etc. We provide an overview of deep learning architectures based on Artificial Neural Networks (ANNs), Convolutional Neural Networks (CNNs), Long Short-Term Memory (LSTM), and Recursive Neural Networks (RNNs).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Ba, L., Caurana, R.: Do Deep Nets Really Need to be Deep ? 521(7553) 1–6 (2013). arXiv preprint arXiv:1312.6184

  2. Dong, L., Wei, F., Tan, C., Tang, D., Zhou, M., Xu, K.: Adaptive recursive neural network for target-dependent Twitter sentiment classification. In: ACL-2014, pp. 49–54 (2014)

    Google Scholar 

  3. Collobert, R., Weston, J.: Fast semantic extraction using a novel neural network architecture. In: Proceedings of the 45th Annual Meeting of the ACL, pp. 560–567 (2007)

    Google Scholar 

  4. Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuglu, K., Kuksa, P.: Natural language processing (almost) from scratch. J. Mach. Learn. Res. 12, 2493–2537 (2011)

    MATH  Google Scholar 

  5. Gao, H., Mao, J., Zhou, J., Huang, Z., Wang, L., Xu, W.: Are you talking to a machine? Dataset and methods for multilingual image question answering. In: Arxiv, pp. 1–10 (2015)

    Google Scholar 

  6. Goller, C., Kuchler, A.: Learning task-dependent distributed representations by backpropagation through structure. In: Proceedings of the ICNN 1996, pp. 347–352. IEEE (1996)

    Google Scholar 

  7. Golosio, B., Cangelosi, A., Gamotina, O., Masala, G.L.: A cognitive neural architecture able to learn and communicate through natural language. PLoS ONE 10(11), e0140866 (2015)

    Article  Google Scholar 

  8. Irsoy, O., Cardie, C.: Opinion mining with deep recurrent neural networks. In: EMNLP-2014, pp. 720–728 (2014)

    Google Scholar 

  9. Jean, S., Cho, K., Memisevic, R., Bengio, Y.: On using very large target vocabulary for neural machine translation. In: Proceedings of the ACL-IJCNLP (2015)

    Google Scholar 

  10. Elman, J.L.: Finding structure in time. Cogn. Sci. 14(2), 179–211 (1990)

    Article  Google Scholar 

  11. Johnson, R., Zhang, T.: Semi-supervised Convolutional Neural Networks for Text Categorization via Region Embedding, pp. 1–12 (2015)

    Google Scholar 

  12. Kalchbrenner, N., Grefenstette, E., Blunsom, P.: A convolutional neural network for modelling sentences. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, ACL 2014, 22–27 June 2014, Baltimore, MD, USA, vol. 1, pp. 655–665 (2014). Long Papers

    Google Scholar 

  13. Hermann, K.M., Blunsom, P.: Multilingual models for compositional distributional semantics. In Proceedings of ACL (2014)

    Google Scholar 

  14. Kim, Y.: Convolutional neural networks for sentence classification. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP 2014), pp. 1746–1751 (2014)

    Google Scholar 

  15. Iyyer, M., Boyd-Graber, J., Claudino, L., Socher, R., Daumé III, H.: A neural network for factoid question answering over paragraphs. In: EMNLP (2014)

    Google Scholar 

  16. Malinowski, M., Rohrbach, M., Fritz, M.: Ask your neurons: a neural-based approach to answering questions about images. In: IEEE International Conference on Computer Vision, pp. 1–9 (2015)

    Google Scholar 

  17. Pennington, J., Socher, R., Manning, C.D.: GloVe: global vectors for word representation. In: EMNLP (2014)

    Google Scholar 

  18. dos Santos, C.N., Gatti, M.: Deep convolutional neural networks for sentiment analysis of short texts. In: COLING-2014, pp. 69–78 (2014)

    Google Scholar 

  19. dos Santos, C.N., Guimarães, V.: Boosting named entity recognition with neural character embeddings. In: ACL 2014, pp. 25–33 (2015)

    Google Scholar 

  20. Schmidhuber, J.: Deep learning in neural networks: an overview. Neural Netw. 61, 85–117 (2015)

    Article  Google Scholar 

  21. Shang, L., Lu, Z., Li, H.: Neural responding machine for short-text conversation. In: ACL-2015, pp. 1577–1586 (2015)

    Google Scholar 

  22. Shen, Y., He, X., Gao, J., Deng, L., Mesnil, G.: A latent semantic model with convolutional-pooling structure for information retrieval. In: Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management - CIKM 2014, pp. 101–110 (2014)

    Google Scholar 

  23. Sun, Y., Lin, L., Tang, D., Yang, N., Ji, Z., Wang, X.: Modelling mention, context and entity with neural networks for entity disambiguation. In: IJCAI, pp. 1333–1339 (2015)

    Google Scholar 

  24. Mikolov, T., Corrado, G., Chen, K., Dean, J.: Efficient estimation of word representations in vector space. In: Proceedings of the International Conference on Learning Representations (ICLR 2013) (2013)

    Google Scholar 

  25. Wang, P., Xu, J., Xu, B., Liu, C., Zhang, H., Wang, F., Hao, H.: Semantic clustering and convolutional neural network for short text categorization. In: Proceedings of the ACL 2015, pp. 352–357 (2015)

    Google Scholar 

  26. Weston, J., America, N.E.C.L., Way, I.: A unified architecture for natural language processing: deep neural networks with multitask learning. In: ICML 2008, pp. 160–167 (2008)

    Google Scholar 

  27. LeCun, Y., Bengio, Y.: Convolutional networks for images, speech, and time-series. pp. 255–258. MIT Press (1995)

    Google Scholar 

  28. Yu, L., Hermann, K.M., Blunsom, P., Pulman, S.: Deep learning for answer sentence selection. In: NIPS Deep Learning Workshop, 9 p. (2014)

    Google Scholar 

  29. Zhang, X., LeCun, Y.: Text Understanding from Scratch (2015)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Cognitive Computing Research Group, Computer Science Department, University of Westminster, London, UK

    S. Alshahrani & E. Kapetanios

Authors
  1. S. Alshahrani
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. E. Kapetanios
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to S. Alshahrani .

Editor information

Editors and Affiliations

  1. ConservatoireNational desArts et Métiers, Paris, France

    Elisabeth Métais

  2. University of Salford, Salford, United Kingdom

    Farid Meziane

  3. University of Salford, Salford, United Kingdom

    Mohamad Saraee

  4. Oakland University, Rochester, Michigan, USA

    Vijayan Sugumaran

  5. University of Salford, Salford, United Kingdom

    Sunil Vadera

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Alshahrani, S., Kapetanios, E. (2016). Are Deep Learning Approaches Suitable for Natural Language Processing?. In: Métais, E., Meziane, F., Saraee, M., Sugumaran, V., Vadera, S. (eds) Natural Language Processing and Information Systems. NLDB 2016. Lecture Notes in Computer Science(), vol 9612. Springer, Cham. https://doi.org/10.1007/978-3-319-41754-7_33

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-41754-7_33

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-41753-0

  • Online ISBN: 978-3-319-41754-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics