Skip to main content
SpringerLink
Log in

Reading More Efficiently: Multi-sentence Summarization with a Dual Attention and Copy-Generator Network

  • Conference paper
  • First Online:
PRICAI 2018: Trends in Artificial Intelligence (PRICAI 2018)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 11012))

Included in the following conference series:

  • 3275 Accesses

Abstract

Sequence-to-sequence neural networks with attention have been widely used in text summarization as the amount of textual data has exploded in recent years. The traditional approach to automatic summarization is based only on word attention and most of them focus on generating a single sentence summarization. In this work, we propose a novel model with a dual attention that considers both sentence and word information and then generates a multi-sentence summarization word by word. Additionally, we enhance our model with a copy-generator network to solve the out-of-vocabulary (OOV) problem. The model shows significant performance gains on the CNN/DailyMail corpus compared with the baseline model. Experimental results demonstrate that our method can obtain ROUGE-1 points of 37.48, ROUGE-2 points of 16.40 and ROUGE-L points of 34.36. Our work shows that several features of our proposed model contribute to further improvements in performance.

This work was supported by National Natural Science Foundation of China (61772075).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. CoRR abs/1409.0473 (2014). http://arxiv.org/abs/1409.0473

  2. Chopra, S., Auli, M., Rush, A.M.: Abstractive sentence summarization with attentive recurrent neural networks. In: The 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2016, San Diego, California, USA, 12–17 June 2016, pp. 93–98 (2016). http://aclweb.org/anthology/N/N16/N16-1012.pdf

  3. Chorowski, J., Bahdanau, D., Serdyuk, D., Cho, K., Bengio, Y.: Attention-based models for speech recognition. In: Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, 7–12 December 2015, Montreal, Quebec, Canada, pp. 577–585 (2015). http://papers.nips.cc/paper/5847-attention-based-models-for-speech-recognition

  4. Dorr, B., Zajic, D., Schwartz, R.: Hedge trimmer: a parse-and-trim approach to headline generation. In: Proceedings of the HLT-NAACL 2003 on Text Summarization Workshop, vol. 5, pp. 1–8. Association for Computational Linguistics (2003)

    Google Scholar 

  5. Duchi, J.C., Hazan, E., Singer, Y.: Adaptive subgradient methods for online learning and stochastic optimization. J. Mach. Learn. Res. 12, 2121–2159 (2011)

    MathSciNet  MATH  Google Scholar 

  6. Flick, C.: Rouge: a package for automatic evaluation of summaries. In: The Workshop on Text Summarization Branches Out, p. 10 (2004)

    Google Scholar 

  7. Gu, J., Lu, Z., Li, H., Li, V.O.K.: Incorporating copying mechanism in sequence-to-sequence learning. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, 7–12 August 2016, Berlin, Germany, Volume 1: Long Papers (2016). http://aclweb.org/anthology/P/P16/P16-1154.pdf

  8. Gülçehre, Ç., Ahn, S., Nallapati, R., Zhou, B., Bengio, Y.: Pointing the unknown words. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, 7–12 August 2016, Berlin, Germany, Volume 1: Long Papers (2016). http://aclweb.org/anthology/P/P16/P16-1014.pdf

  9. Hermann, K.M., Kociský, T., Grefenstette, E., Espeholt, L., Kay, W., Suleyman, M., Blunsom, P.: Teaching machines to read and comprehend. In: Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, 7–12 December 2015, Montreal, Quebec, Canada, pp. 1693–1701 (2015). http://papers.nips.cc/paper/5945-teaching-machines-to-read-and-comprehend

  10. Nallapati, R., Zhai, F., Zhou, B.: Summarunner: a recurrent neural network based sequence model for extractive summarization of documents. In: Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, 4–9 February 2017, San Francisco, California, USA, pp. 3075–3081 (2017). http://aaai.org/ocs/index.php/AAAI/AAAI17/paper/view/14636

  11. Nallapati, R., Zhou, B., Ma, M.: Classify or select: neural architectures for extractive document summarization. CoRR abs/1611.04244 (2016). http://arxiv.org/abs/1611.04244

  12. Nallapati, R., Zhou, B., dos Santos, C.N., Gülçehre, Ç., Xiang, B.: Abstractive text summarization using sequence-to-sequence RNNs and beyond. In: Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning, CoNLL 2016, Berlin, Germany, 11–12 August 2016, pp. 280–290 (2016). http://aclweb.org/anthology/K/K16/K16-1028.pdf

  13. Nam, H., Ha, J., Kim, J.: Dual attention networks for multimodal reasoning and matching. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, 21–26 July 2017, pp. 2156–2164 (2017). https://doi.org/10.1109/CVPR.2017.232,

  14. Neto, J.L., Freitas, A.A., Kaestner, C.A.A.: Automatic text summarization using a machine learning approach. In: Bittencourt, G., Ramalho, G.L. (eds.) SBIA 2002. LNCS (LNAI), vol. 2507, pp. 205–215. Springer, Heidelberg (2002). https://doi.org/10.1007/3-540-36127-8_20

    Chapter  Google Scholar 

  15. Paulus, R., Xiong, C., Socher, R.: A deep reinforced model for abstractive summarization. CoRR abs/1705.04304 (2017). http://arxiv.org/abs/1705.04304

  16. Ren, P., Chen, Z., Ren, Z., Wei, F., Ma, J., de Rijke, M.: Leveraging contextual sentence relations for extractive summarization using a neural attention model. In: Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, Shinjuku, Tokyo, Japan, 7–11 August 2017, pp. 95–104 (2017). https://doi.org/10.1145/3077136.3080792

  17. Rush, A.M., Chopra, S., Weston, J.: A neural attention model for abstractive sentence summarization. In: EMNLP, pp. 379–389. The Association for Computational Linguistics (2015)

    Google Scholar 

  18. See, A., Liu, P.J., Manning, C.D.: Get to the point: summarization with pointer-generator networks. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, 30 July–4 August, Volume 1: Long Papers, pp. 1073–1083 (2017). https://doi.org/10.18653/v1/P17-1099

  19. Seo, S., Huang, J., Yang, H., Liu, Y.: Interpretable convolutional neural networks with dual local and global attention for review rating prediction. In: Proceedings of the Eleventh ACM Conference on Recommender Systems, RecSys 2017, Como, Italy, 27–31 August 2017, pp. 297–305 (2017). https://doi.org/10.1145/3109859.3109890

  20. Tong, B., et al.: Learning to generate rock descriptions from multivariate well logs with hierarchical attention. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Halifax, NS, Canada, 13–17 August 2017, pp. 2031–2040 (2017). https://doi.org/10.1145/3097983.3098132

  21. Venugopalan, S., Xu, H., Donahue, J., Rohrbach, M., Mooney, R.J., Saenko, K.: Translating videos to natural language using deep recurrent neural networks. In: The 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2015, Denver, Colorado, USA, 31 May–5 June 2015, pp. 1494–1504 (2015). http://aclweb.org/anthology/N/N15/N15-1173.pdf

  22. Vinyals, O., Fortunato, M., Jaitly, N.: Pointer networks. In: Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, 7–12 December 2015, Montreal, Quebec, Canada, pp. 2692–2700 (2015). http://papers.nips.cc/paper/5866-pointer-networks

  23. Wang, X., et al.: Dynamic attention deep model for article recommendation by learning human editors’ demonstration. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Halifax, NS, Canada, 13–17 August 2017, pp. 2051–2059 (2017). https://doi.org/10.1145/3097983.3098096

  24. Zeng, W., Luo, W., Fidler, S., Urtasun, R.: Efficient summarization with read-again and copy mechanism. CoRR abs/1611.03382 (2016). http://arxiv.org/abs/1611.03382

  25. Zhou, Q., Yang, N., Wei, F., Zhou, M.: Selective encoding for abstractive sentence summarization. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, 30 July–4 August, Volume 1: Long Papers, pp. 1095–1104 (2017). https://doi.org/10.18653/v1/P17-1101

Download references

Author information

Authors and Affiliations

  1. Beijing Institute of Technology, No. 5, Zhongguancun South Street, Haidian Distirct, Beijing, 100081, China

    Xi Zhang, Hua-ping Zhang & Lei Zhao

Authors
  1. Xi Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hua-ping Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lei Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hua-ping Zhang .

Editor information

Editors and Affiliations

  1. Southeast University, Nanjing, China

    Xin Geng

  2. University of Tasmania, Hobart, Tasmania, Australia

    Byeong-Ho Kang

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhang, X., Zhang, Hp., Zhao, L. (2018). Reading More Efficiently: Multi-sentence Summarization with a Dual Attention and Copy-Generator Network. In: Geng, X., Kang, BH. (eds) PRICAI 2018: Trends in Artificial Intelligence. PRICAI 2018. Lecture Notes in Computer Science(), vol 11012. Springer, Cham. https://doi.org/10.1007/978-3-319-97304-3_62

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-97304-3_62

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-97303-6

  • Online ISBN: 978-3-319-97304-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation