Abstract
Keyphrase generation aims to generate several words that can simply summarize the semantics of the article, which is the basis of many natural language processing tasks. Although most previous approaches have achieved good results, they neglect the independent word information in the source text. Previous models use attention mechanism to calculate the relationship between the encoder RNN hidden states and the target side. However hidden state \(h_t\) is the summarization of the first t words as a subsequence in the source sentence. We, in this paper, propose a novel sequence-to-sequence model called WordRNN, which can capture word level representations in the source text. Our model can enrich the expression of the source text by directly promoting the pure word level information. Moreover, we use fuse gate and simply concat operation to combine the subsequence level and word level contextual information. Experiment results demonstrate that our approach achieves higher performance than the state-of-the-art methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. In: 3rd International Conference on Learning Representations, ICLR 2015. Conference Track Proceedings, San Diego, CA, USA, 7–9 May 2015 (2015). http://arxiv.org/abs/1409.0473
Berend, G.: Opinion expression mining by exploiting keyphrase extraction. In: Fifth International Joint Conference on Natural Language Processing, IJCNLP 2011, Chiang Mai, Thailand, 8–13 November 2011, pp. 1162–1170 (2011). http://aclweb.org/anthology/I/I11/I11-1130.pdf
Bougouin, A., Boudin, F., Daille, B.: TopicRank: graph-based topic ranking for keyphrase extraction. In: Sixth International Joint Conference on Natural Language Processing, IJCNLP 2013, Nagoya, Japan, 14–18 October 2013, pp. 543–551 (2013). http://aclweb.org/anthology/I/I13/I13-1062.pdf
Chen, J., Zhang, X., Wu, Y., Yan, Z., Li, Z.: Keyphrase generation with correlation constraints. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, 31 October–4 November 2018, pp. 4057–4066 (2018). https://aclanthology.info/papers/D18-1439/d18-1439
Chen, W., Gao, Y., Zhang, J., King, I., Lyu, M.R.: Title-guided encoding for keyphrase generation. CoRR abs/1808.08575 (2018). http://arxiv.org/abs/1808.08575
Frank, E., Paynter, G.W., Witten, I.H., Gutwin, C., Nevill-Manning, C.G.: Domain-specific keyphrase extraction. In: Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence, IJCAI 1999, Stockholm, Sweden, 31 July–6 August 1999, vol. 2, 1450 p, pp. 668–673 (1999). http://ijcai.org/Proceedings/99-2/Papers/002.pdf
Gong, Y., Luo, H., Zhang, J.: Natural language inference over interaction space. In: 6th International Conference on Learning Representations, ICLR 2018. Conference Track Proceedings, Vancouver, BC, Canada, 30 April–3 May 2018 (2018). https://openreview.net/forum?id=r1dHXnH6-
Gu, J., Lu, Z., Li, H., Li, V.O.K.: Incorporating copying mechanism in sequence-to-sequence learning. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, Berlin, Germany, 7–12 August 2016, Long Papers, vol. 1 (2016). http://aclweb.org/anthology/P/P16/P16-1154.pdf
Hasan, K.S., Ng, V.: Automatic keyphrase extraction: a survey of the state of the art. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, ACL 2014, Baltimore, MD, USA, 22–27 June 2014, Long Papers, vol. 1, pp. 1262–1273 (2014). http://aclweb.org/anthology/P/P14/P14-1119.pdf
Hulth, A.: Improved automatic keyword extraction given more linguistic knowledge. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, EMNLP 2003, Sapporo, Japan, 11–12 July 2003 (2003). https://aclanthology.info/papers/W03-1028/w03-1028
Hulth, A., Megyesi, B.: A study on automatically extracted keywords in text categorization. In: ACL 2006, Proceedings of the Conference 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, Sydney, Australia, 17–21 July 2006 (2006). http://aclweb.org/anthology/P06-1068
Jones, S., Staveley, M.S.: Phrasier: a system for interactive document retrieval using keyphrases. In: SIGIR 1999: Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Berkeley, CA, USA, 15–19 August 1999, pp. 160–167 (1999). https://doi.org/10.1145/312624.312671
Kim, S.N., Medelyan, O., Kan, M., Baldwin, T.: SemEval-2010 task 5: automatic keyphrase extraction from scientific articles. In: Proceedings of the 5th International Workshop on Semantic Evaluation, SemEval@ACL 2010, Uppsala University, Uppsala, Sweden, 15–16 July 2010, pp. 21–26 (2010). http://aclweb.org/anthology/S/S10/S10-1004.pdf
Le, T.T.N., Nguyen, M.L., Shimazu, A.: Unsupervised keyphrase extraction: introducing new kinds of words to keyphrases. In: AI 2016: Advances in Artificial Intelligence - Proceedings of the 29th Australasian Joint Conference, Hobart, TAS, Australia, 5–8 December 2016, pp. 665–671 (2016). https://doi.org/10.1007/978-3-319-50127-7_58
Liu, Z., Chen, X., Zheng, Y., Sun, M.: Automatic keyphrase extraction by bridging vocabulary gap. In: Proceedings of the Fifteenth Conference on Computational Natural Language Learning, CoNLL 2011, Portland, Oregon, USA, 23–24 June 2011, pp. 135–144 (2011). http://aclweb.org/anthology/W/W11/W11-0316.pdf
Liu, Z., Li, P., Zheng, Y., Sun, M.: Clustering to find exemplar terms for keyphrase extraction. In: Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, EMNLP 2009, A Meeting of SIGDAT, a Special Interest Group of the ACL, Singapore, 6–7 August 2009, pp. 257–266 (2009). http://www.aclweb.org/anthology/D09-1027
Medelyan, O., Frank, E., Witten, I.H.: Human-competitive tagging using automatic keyphrase extraction. In: Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, EMNLP 2009, A Meeting of SIGDAT, a Special Interest Group of the ACL, Singapore, 6–7 August 2009, pp. 1318–1327 (2009). http://www.aclweb.org/anthology/D09-1137
Meng, R., Zhao, S., Han, S., He, D., Brusilovsky, P., Chi, Y.: Deep keyphrase generation. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, 30 July–4 August, Long Papers, vol. 1, pp. 582–592 (2017). https://doi.org/10.18653/v1/P17-1054
Mihalcea, R., Tarau, P.: TextRank: bringing order into text. In: Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing, EMNLP 2004, A Meeting of SIGDAT, a Special Interest Group of the ACL, held in conjunction with ACL 2004, Barcelona, Spain, 25–26 July 2004, pp. 404–411 (2004). http://www.aclweb.org/anthology/W04-3252
Nguyen, T.D., Kan, M.: Keyphrase extraction in scientific publications. In: Asian Digital Libraries. Looking Back 10 Years and Forging New Frontiers, Proceedings of the 10th International Conference on Asian Digital Libraries, ICADL 2007, Hanoi, Vietnam, 10–13 December 2007, pp. 317–326 (2007). https://doi.org/10.1007/978-3-540-77094-7_41
Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. In: Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, Montreal, Quebec, Canada, 8–13 December 2014, pp. 3104–3112 (2014). http://papers.nips.cc/paper/5346-sequence-to-sequence-learning-with-neural-networks
Wan, X., Xiao, J.: Single document keyphrase extraction using neighborhood knowledge. In: Proceedings of the Twenty-Third AAAI Conference on Artificial Intelligence, AAAI 2008, Chicago, Illinois, USA, 13–17 July 2008, pp. 855–860 (2008). http://www.aaai.org/Library/AAAI/2008/aaai08-136.php
Wang, L., Cardie, C.: Domain-independent abstract generation for focused meeting summarization. In: Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, ACL 2013, Sofia, Bulgaria, 4–9 August 2013, Long Papers, vol. 1, pp. 1395–1405 (2013). http://aclweb.org/anthology/P/P13/P13-1137.pdf
Wang, M., Zhao, B., Huang, Y.: PTR: phrase-based topical ranking for automatic keyphrase extraction in scientific publications. In: Neural Information Processing - Proceedings of the 23rd International Conference, ICONIP 2016, Kyoto, Japan, 16–21 October 2016, Part IV, pp. 120–128 (2016). https://doi.org/10.1007/978-3-319-46681-1_15
Witten, I.H., Paynter, G.W., Frank, E., Gutwin, C., Nevill-Manning, C.G.: KEA: practical automatic keyphrase extraction. In: Proceedings of the Fourth ACM Conference on Digital Libraries, Berkeley, CA, USA, 11–14 August 1999, pp. 254–255 (1999). https://doi.org/10.1145/313238.313437
Ye, H., Wang, L.: Semi-supervised learning for neural keyphrase generation. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, 31 October–4 November 2018, pp. 4142–4153 (2018). https://aclanthology.info/papers/D18-1447/d18-1447
Zhang, F., Huang, L., Peng, B.: WordTopic-multirank: a new method for automatic keyphrase extraction. In: Sixth International Joint Conference on Natural Language Processing, IJCNLP 2013, Nagoya, Japan, 14–18 October 2013, pp. 10–18 (2013). http://aclweb.org/anthology/I/I13/I13-1002.pdf
Zhang, Q., Wang, Y., Gong, Y., Huang, X.: Keyphrase extraction using deep recurrent neural networks on Twitter. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, Texas, USA, 1–4 November 2016, pp. 836–845 (2016). http://aclweb.org/anthology/D/D16/D16-1080.pdf
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Huang, H., Huang, T., Ma, L., Zhang, L. (2019). Keyphrase Generation with Word Attention. In: Gedeon, T., Wong, K., Lee, M. (eds) Neural Information Processing. ICONIP 2019. Lecture Notes in Computer Science(), vol 11955. Springer, Cham. https://doi.org/10.1007/978-3-030-36718-3_23
Download citation
DOI: https://doi.org/10.1007/978-3-030-36718-3_23
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-36717-6
Online ISBN: 978-3-030-36718-3
eBook Packages: Computer ScienceComputer Science (R0)