Skip to main content
SpringerLink
Log in
SpringerLink
Log in

Character-Level Hybrid Convolutional and Recurrent Neural Network for Fast Text Categorization

  • Conference paper
  • First Online:
Proceedings of ELM 2018 (ELM 2018)

Part of the book series: Proceedings in Adaptation, Learning and Optimization ((PALO,volume 11))

Included in the following conference series:

Abstract

Text categorization, or text classification, is one of key tasks for representing the semantic information of documents. Traditional deep leaning models for text categorization are generally time-consuming with large-sized datasets due to slow convergence rate. In this paper, we propose a character-level model for short text classification with a combination of convolutional neural network (CNN), gated recurrent unit (GRU) and highway network (HN), which can capture both the global and the local textual semantics while having a tractable computational complexity. In addition, error minimization extreme learning machine (EM-ELM) is incorporated into the proposed model to improve the classification accuracy further. Extensive experiments show that our approach achieves the state-of-the-art performance when the hybrid model based on EM-ELM is trained using large-sized datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://www.cs.cornell.edu/people/pabo/moviereview-data/.

  2. 2.

    http://nlp.stanford.edu/sentiment/Data.

  3. 3.

    http://cogcomp.cs.illinois.edu/Data/QA/QC/.

  4. 4.

    http://www.cs.huji.ac.il/nogazas/pages/projects.html.

  5. 5.

    http://www.di.unipi.it/~gulli/AG_corpus_of_news_articles.html.

  6. 6.

    Yahoo! Webscope program’s Yahoo! Answers Comprehensive Questions and Answers version 1.0 dataset.

References

  1. Gong, Y., Ke, Q., Isard, M., Lazebnik, S.: A multi-view embedding space for modeling internet images, tags, and their semantics. Int. J. Comput. Vis. 106(2), 210–233 (2014)

    Article  Google Scholar 

  2. Poria, S., Cambria, E., Bajpai, R., Hussain, A.: A review of affective computing: from unimodal analysis to multimodal fusion. Inf. Fusion. 37, 98–125 (2017)

    Article  Google Scholar 

  3. Cambria, E.: Affective computing and sentiment analysis. IEEE Intell. Syst. 31(2), 102–107 (2016)

    Article  Google Scholar 

  4. Zhang, Y., Wang, S., Phillips, P.: Binary PSO with mutation operator for feature selection using decision tree applied to spam detection. Knowl. Based Syst. 64(1), 22–31 (2014)

    Article  Google Scholar 

  5. Poria, S., Cambria, E., Howard, N., Huang, G.-B., Hussain, A.: Fusing audio, visual and textual clues for sentiment analysis from multimodal content. Neurocomputing 174, 50–59 (2016)

    Article  Google Scholar 

  6. Cambria, E., White, B.: Jumping NLP curves: a review of natural language processing research. IEEE Comput. Intell. Mag. 9(2), 48–57 (2014)

    Article  Google Scholar 

  7. Collobert, R., Weston, J.: A unified architecture for natural language processing: deep neural networks with multitask learning. In: Proceedings of the 25th International Conference on Machine Learning, pp. 160–167. ACM (2008)

    Google Scholar 

  8. Kim, Y.: Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882 (2014)

  9. Cambria, E., Fu, J., Bisio, F., Poria, S.: Affective Space 2: enabling affective intuition for concept-level sentiment analysis. In: AAAI, Austin, pp. 508–514 (2015)

    Google Scholar 

  10. He, T., Huang, W., Qiao, Y.: Text-attentional convolutional neural network for scene text detection. IEEE Trans. Image Process. 25(6), 25–29 (2016)

    Article  MathSciNet  Google Scholar 

  11. Ren, X., Zhou, Y., He, J.: A convolutional neural network based chinese text detection algorithm via text structure modeling. IEEE Trans. Multimedia 19, 506–518 (2017)

    Article  Google Scholar 

  12. Chaturvedi, I., Ong, Y.-S., Tsang, I., Welsch, R., Cambria, E.: Learning word dependencies in text by means of a deep recurrent belief network. Knowl. Based Syst. 108, 144–154 (2016)

    Article  Google Scholar 

  13. Cho, K., Van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., Bengio, Y.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078 (2014)

  14. Feng, G., Huang, G.B., Lin, Q.: Error minimized extreme learning machine with growth of hidden nodes and incremental learning. IEEE Trans. Neural Netw. 20(8), 1352–1357 (2009)

    Article  Google Scholar 

  15. Majumder, N., Poria, S., Gelbukh, A., Cambria, E.: Deep learning based document modeling for personality detection from text. IEEE Intell. Syst. 32(2), 74–79 (2017)

    Article  Google Scholar 

  16. Tang, D., Qin, B., Liu, T.: Document modeling with gated recurrent neural network for sentiment classification. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 1422–1432 (2015)

    Google Scholar 

  17. Tai, K.S., Socher, R., Manning, C.D.: Improved semantic representations from tree-structured long short-term memory networks. arXiv preprint arXiv:1503.00075 (2015)

  18. Zhang, M.-L., Zhou, Z.-H.: Multilabel neural networks with applications to functional genomics and text categorization. IEEE Trans. Knowl. Data Eng. 18(10), 1338–1351 (2006)

    Article  Google Scholar 

  19. Nam, J., Kim, J., Mencía, E.L., Gurevych, I., Fürnkranz, J.: Largescale multi-label text classification—revisiting neural networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 437–452. Springer (2014)

    Google Scholar 

  20. Benites, F., Sapozhnikova, E.: HARAM: a hierarchical ARAM neural network for large-scale text classification. In: IEEE International Conference on Data Mining Workshop (ICDMW), pp. 847–854. IEEE (2015)

    Google Scholar 

  21. Nair, V., Hinton, G.E: Rectified linear units improve restricted Boltzmann machines. In: Proceedings of the 27th International Conference on Machine Learning (ICML 2010), pp. 807–814 (2010)

    Google Scholar 

  22. Dai, A.M., Le, Q.V.: Semi-supervised sequence learning. In: Advances in Neural Information Processing Systems, pp. 3079–3087 (2015)

    Google Scholar 

  23. Santos, C., Zadrozny, B.: Learning character-level representations for part-of-speech tagging. In: Proceedings of the 31st International Conference on Machine Learning, pp. 1818–1826 (2014)

    Google Scholar 

  24. Johnson, R., Zhang, T.: Effective use of word order for text categorization with convolutional neural networks. In: Proceeding of North American Chapter of the Association for Computational Linguistics, pp. 103–112 (2014)

    Google Scholar 

  25. Zhang, X, LeCun, Y.: Text Understanding from Scratch. Computation and Language, arXiv preprint arXiv:1502.01710 (2015)

  26. Ling, W., Luís, T., Marujo, L.: Finding function in form: compositional character models for open vocabulary word representation. Computer Science, pp. 1899–1907 (2015)

    Google Scholar 

  27. Huang, Z., Xu, W., Yu, K.: Bidirectional LSTM-CRF models for sequence tagging. Computer Science (2015)

    Google Scholar 

  28. Srivastava, R.K., Greff, K., Schmidhuber, J.: Training very deep networks. Computer Science, pp. 1–5 (2015)

    Google Scholar 

  29. Kingma, D., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

Download references

Acknowledgments

This work is supported by “the Fundamental Research Funds for the Central Universities” (NO. 2017XKQY082).

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, China University of Mining and Technology, Xuzhou, Jiangsu, China

    Bing Liu & Yong Zhou

  2. College of Information and Control, China University of Mining and Technology, Xuzhou, Jiangsu, China

    Wei Sun

Authors
  1. Bing Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yong Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wei Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bing Liu .

Editor information

Editors and Affiliations

  1. Institute of Information and Control, Hangzhou Dianzi University, Xiasha, Hangzhou, China

    Jiuwen Cao

  2. Department of Computer and Information Science, University of Macau, Taipa, Macao

    Chi Man Vong

  3. Nokia Bell Labs, Cybersecurity Research, Espoo, Finland

    Yoan Miche

  4. Department of Information and Logistics Technology, University of Houston, Houston, TX, USA

    Amaury Lendasse

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Liu, B., Zhou, Y., Sun, W. (2020). Character-Level Hybrid Convolutional and Recurrent Neural Network for Fast Text Categorization. In: Cao, J., Vong, C., Miche, Y., Lendasse, A. (eds) Proceedings of ELM 2018. ELM 2018. Proceedings in Adaptation, Learning and Optimization, vol 11. Springer, Cham. https://doi.org/10.1007/978-3-030-23307-5_12

Download citation

Publish with us

Policies and ethics

Search

Navigation