Skip to main content
SpringerLink
Log in

A visual attention-based keyword extraction for document classification

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Document classification plays an important role in natural language processing. Among that, keyword extraction algorithm shows its great potential in summarizing the entire document. Attention is the process of selectively concentrating on a discrete aspect of information, while ignoring other perceivable information. A new probabilistic keyword extraction algorithm is proposed, which is inspired by the visual attention mechanism. An unsupervised neural network based pre-training method is proposed for training the semantic attention based keyword extraction algorithm, which is helpful in extracting keywords with rich contextual information from the document. A bidirectional Long short-term memory network combined with the proposed semantic keyword extraction algorithm is designed for both topic and sentiment classification tasks. Experiments on four large scale datasets show that the proposed visual attention based keyword extraction algorithm gives a better performance than the baseline methods. The semantic attention based keyword extraction method is significant in summarizing the content of a document, which is very useful for large scale document classification.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Bahdanau D, Cho K, Bengio Y (2014) Neural machine translation by jointly learning to align and translate, arXiv:1409.0473

  2. Cheng J, Lapata M (2016) Neural summarization by extracting sentences and words, arXiv:1603.07252

  3. Chung J, Gulcehre C et al (2014) Empirical evaluation of gated recurrent neural networks on sequence modeling, arXiv:1412.3555

  4. Graves A, Jaitly N (2014) Towards end-to-end speech recognition with recurrent neural networks. In: Proceedings of the 31st international conference on machine learning, pp 1764–1772

  5. Guo Z, Gao L, Song J et al (2016) Attention-based LSTM with semantic consistency for videos captioning. In: Proceedings of the 2016 ACM on multimedia conference. ACM, pp 357–361

  6. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780

    Article  Google Scholar 

  7. Jianfei N, Jiangzhen L (2016) Using Word2vec with TextRank to extract keywords. New Technology of Library and Information Service, pp 20–27

  8. Kim Y (2014) Convolutional neural networks for sentence classification, arXiv:1408.5882

  9. Krizhevsky A, Sutskever I, Hinton G E (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inf Proces Syst 1097–1105

  10. LeCun Y, Bottou L, Bengio Y et al (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324

    Article  Google Scholar 

  11. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521 (7553):436–444

    Article  Google Scholar 

  12. Li J, Sun M (2007) Scalable term selection for text categorization. In: EMNLP-CoNLL, pp 774–782

  13. Matsuo Y, Ishizuka M (2004) Keyword extraction from a single document using word co-occurrence statistical information. Int J Artif Intell Tools 13(01):157–169

    Article  Google Scholar 

  14. Mikolov T, Sutskever I, Chen K et al (2013) Distributed representations of words and phrases and their compositionality. Adv Neural Inf Proces Syst 3111–3119

  15. Mnih V, Heess N, Graves A (2014) Recurrent models of visual attention. Adv Neural Inf Proces Syst 256–264

  16. Pennington J, Socher R, Manning C (2014) Glove: global vectors for word representation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp 1532–1543

  17. Song J, Guo Z, Gao L et al (2017) Hierarchical LSTM with adjusted temporal attention for video captioning. In: IJCAI

  18. Tang D, Qin B, Liu T (2015) Document modeling with gated recurrent neural network for sentiment classification. In: EMNLP, pp 1422–1432

  19. Xu K, Ba J, Kiros R et al (2015) Show, attend and tell: neural image caption generation with visual attention. In: International conference on machine learning, pp 2048–2057

  20. Yelp Dataset Challenge 2017, https://www.yelp.com/dataset/challenge (2017)

  21. Yuepeng L, Cui J, Junchuan J (2015) A keyword extraction algorithm based on Word2vec. E-science Technol Appl 4(2015):54–59

    Google Scholar 

  22. Zhang Y, Lai G, Zhang M et al (2014) Explicit factor models for explainable recommendation based on phrase-level sentiment analysis. In: Proceedings of the 37th international ACM SIGIR conference on Research & development in information retrieval. ACM, pp 83–92

  23. Zhang X, Zhao J, LeCun Y (2015) Character-level convolutional networks for text classification. Adv Neural Inf Proces Syst 649–657

Download references

Acknowledgements

This paper is supported by the project 61303094 supported by National Natural Science Foundation of China, by the Science and Technology Commission of Shanghai Municipality 16511102400 and 16111107800, by Innovation Program of Shanghai Municipal Education Commission (14YZ024).

Author information

Authors and Affiliations

  1. School of Computer Engineering and Science, Shanghai University, Shanghai, 200444, China

    Xing Wu, Zhikang Du & Yike Guo

  2. Shanghai Institute for Advanced Communication and Data Science, Shanghai, China

    Xing Wu, Zhikang Du & Yike Guo

Authors
  1. Xing Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhikang Du
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yike Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xing Wu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wu, X., Du, Z. & Guo, Y. A visual attention-based keyword extraction for document classification. Multimed Tools Appl 77, 25355–25367 (2018). https://doi.org/10.1007/s11042-018-5788-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-018-5788-9

Keywords

Search

Navigation