Skip to main content

Advertisement

Springer Nature Link
Log in

Structure-to-word dynamic interaction model for abstractive sentence summarization

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Abstractive text summarization aims to capture important information from text and integrate contextual information to guide the summary generation. However, effective integration of important and relevant information remains a challenging problem. Existing graph-based methods only consider either word relations or structure information, but neglect the correlation between them. To simultaneously capture the word relations and structure information from sentences, we propose a novel Structure-to-Word dynamic interaction model for Abstractive Sentence Summarization (SWSum). Specifically, we first represent structure and word relation information of sentences by constructing semantic scenario graph and semantic word relation graph based on FrameNet. We subsequently stack multiple graph-based dynamic interaction layers that iteratively enhance their correlation to learn node representations. Finally, a graph fusion module is designed to obtain better overall graph representations, which provide an attention-based context vector for the decoder to generate summary. Experimental results demonstrate our model outperforms existing state-of-the-art methods on two popular benchmark datasets, i.e., Gigaword and DUC 2004.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data availability

The data that support the findings of this study are available within this article.

Notes

  1. https://framenet.icsi.berkeley.edu/fndrupal/frameIndex.

  2. https://framenet.icsi.berkeley.edu/fndrupal/ASRL.

  3. https://github.com/harvardnlp/sent-summary.

  4. https://duc.nist.gov/data.html.

  5. For DUC 2004 data, the generated summares are cut-off after 75-characters.

  6. https://github.com/huggingface/pytorch-pretrained-BERT.

References

  1. Zhou Q, Yang N, Wei F, Zhou M (2017) Selective encoding for abstractive sentence summarization. In: Proceedings of ACL, pp. 1095–1104

  2. Svore K, Vanderwende L, Burges C (2007) Enhancing single-document summarization by combining ranknet and third-party sources. In: Proceedings of EMNLP-CoNLL, pp. 448–457

  3. Hsu W-T. Lin C-K, Lee M-Y, Min K, Tang J, Sun M (2018) A unified model for extractive and abstractive summarization using inconsistency loss. In: Proceedings of ACL, pp. 132–141. https://doi.org/10.18653/v1/P18-1013

  4. Dauphin YN, Fan A, Auli M, Grangier D (2017) Language modeling with gated convolutional networks. In: Proceedings of ICML, pp. 933–941

  5. Narayan S, Cohen SB, Lapata M (2018) Ranking sentences for extractive summarization with reinforcement learning. In: Proceedings of NAACL, pp. 1747–1759. https://doi.org/10.18653/v1/N18-1158

  6. Rush AM, Chopra S, Weston J (2015) A neural attention model for abstractive sentence summarization. In: Proceedings of EMNLP, pp. 379–389

  7. See A, Liu PJ, Manning CD (2017) Get to the point: Summarization with pointer-generator networks. In: Proceedings of ACL, pp. 1073–1083. https://doi.org/10.18653/v1/P17-1099

  8. Rothe S, Narayan S, Severyn A (2020) Leveraging pre-trained checkpoints for sequence generation tasks. In: Proceedings of TACL 8, 264–280

  9. Takase S, Suzuki J, Okazaki N, Hirao T, Nagata M (2016) Neural headline generation on Abstract Meaning Representation. In: Proceedings of EMNLP, pp. 1054–1059. https://doi.org/10.18653/v1/D16-1112

  10. Jin H, Wang T, Wan X (2020) Semsum: Semantic dependency guided neural abstractive summarization. In: Proceedings of AAAI, vol. 34, pp. 8026–8033

  11. Xu J, Gan Z, Cheng Y, Liu J (2020) Discourse-aware neural extractive text summarization. In: Proceedings of ACL, pp. 5021–5031

  12. Zhu C, Hinthorn W, Xu R, Zeng Q, Zeng M, Huang X, Jiang M (2020) Boosting factual correctness of abstractive summarization with knowledge graph. arXiv 2003.08612

  13. Jin H, Wang T, Wan X (2020) Semsum: Semantic dependency guided neural abstractive summarization. In: Proceedings of AAAI 34(05), 8026–8033

  14. Oepen S, Kuhlmann M, Miyao Y, Zeman D, Cinková S, Flickinger D, Hajič J, Ivanova A, Urešová Z (2016) Towards comparability of linguistic graph Banks for semantic parsing. In: Proceedings of LREC, pp. 3991–3995

  15. Fillmore CJ, et al (1976) Frame semantics and the nature of language. In: Annals of the New York Academy of Sciences: Conference on the Origin and Development of Language and Speech, vol. 280, pp. 20–32

  16. Baker CF, Fillmore CJ, Lowe JB (1998) The Berkeley FrameNet project. In: Proceedings of ACL, pp. 86–90

  17. Guo S, Li R, Tan H, Li X, Guan Y, Zhao H, Zhang Y (2020) A frame-based sentence representation for machine reading comprehension. In: Proceedings of ACL, pp. 891–896

  18. Guan Y, Guo S, Li R, Li X, Zhang H (2021) Frame semantics guided network for abstractive sentence summarization. Knowl-Based Syst 221:106973

    Article  MATH  Google Scholar 

  19. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser u, Polosukhin I (2017) Attention is all you need. In: Proceedings of NIPS, pp. 6000–6010. Curran Associates Inc., Red Hook, NY, USA

  20. Gambhir M, Gupta V (2017) Recent automatic text summarization techniques: a survey. Artif Intell Rev 47(1):1–66

    Article  MATH  Google Scholar 

  21. Knight K, Marcu D (2002) Summarization beyond sentence extraction: A probabilistic approach to sentence compression. Artif Intell 139(1):91–107

    Article  MATH  Google Scholar 

  22. Zhou L, Hovy E (2004) Template-filtered headline summarization. In: In the Proceedings of the ACL Workshop, Text Summarization Branches Out, pp. 56–60

  23. Yadav AK, Sharma A, Yadav V, Kalia N (2023) Rfpssih: reducing false positive text detection sequels in scenery images using hybrid technique. International Journal of System Assurance Engineering and Management 14:2289–2300

    MATH  Google Scholar 

  24. Wang Q, Zhang W, Lei T, Cao Y, Peng D, Wang X (2023) Clsep: Contrastive learning of sentence embedding with prompt. Knowledge-Based Systems 266, 110381 https://doi.org/10.1016/j.knosys.2023.110381

  25. Yadav AK, Ranvijay Yadav RS, Maurya AK 2023) State-of-the-art approach to extractive text summarization: a comprehensive review. Multimedia Tools and Applications 82(19), 29135–29197

  26. Nallapati R, Zhou B, Santos C, Gulcehre C, Xiang B (2016) Abstractive text summarization using sequence-to-sequence RNNs and beyond. In: Riezler, S., Goldberg, Y. (eds.) Proceedings of SIGNLL, pp. 280–290. Association for Computational Linguistics, Berlin, Germany. https://doi.org/10.18653/v1/K16-1028. https://aclanthology.org/K16-1028

  27. Zhao F, Quan B, Yang J, Chen J, Zhang Y, Wang X (2019) Document summarization using word and part-of-speech based on attention mechanism. J Phys: Conf Ser 1168(3):032008. https://doi.org/10.1088/1742-6596/1168/3/032008

    Article  Google Scholar 

  28. Moro G, Ragazzi L, Valgimigli L, Frisoni G, Sartori C, Marfia G (2023) Efficient memory-enhanced transformer for long-document summarization in low-resource regimes. Sensors 23(7)

  29. Bahdanau D, Cho K, Bengio Y (2015) Neural machine translation by jointly learning to align and translate. In: Bengio, Y., LeCun, Y. (eds.) In: Proceedings of ICLR

  30. Lu S, Zhu Y, Zhang W, Wang J, Yu Y (2018) Neural text generation: Past, present and beyond. arXiv preprint arXiv:1803.07133

  31. Devlin J, Chang M-W, Lee K, Toutanova K (2019) BERT: Pre-training of deep bidirectional transformers for language understanding. In: Proceedings of NAACL, pp. 4171–4186

  32. Clark K, Luong M-T, Le QV, Manning CD (2020) ELECTRA: Pre-training text encoders as discriminators rather than generators. In: Proceedings of ICLR. https://openreview.net/pdf?id=r1xMH1BtvB

  33. Zhang H, Liu X, Zhang J (2023) Extractive summarization via ChatGPT for faithful summary generation. In: Bouamor, H., Pino, J., Bali, K. (eds.) Findings of EMNLP, pp. 3270–3278. https://doi.org/10.18653/v1/2023.findings-emnlp.214. https://aclanthology.org/2023.findings-emnlp.214

  34. Ladhak F, Durmus E, Suzgun M, Zhang T, Jurafsky D, McKeown K, Hashimoto T (2023) When do pre-training biases propagate to downstream tasks? a case study in text summarization. In: Vlachos, A., Augenstein, I. (eds.) Proceedings of EACL, pp. 3206–3219. https://doi.org/10.18653/v1/2023.eacl-main.234. https://aclanthology.org/2023.eacl-main.234

  35. Graph-based extractive text summarization based on single document (2023) Yadav, A.K., Ranvijay, Yadav, R.S., Maurya, A.K. Multim. Tools Appl. 83:18987–19013

    Google Scholar 

  36. Kipf TN, Welling M (2017) Semi-Supervised Classification with Graph Convolutional Networks. In: Proceedings of ICLR

  37. Velič ković P, Cucurull G, Casanova A, Romero A, Liò P, Bengio Y (2018) Graph attention networks. In: Proceedings of ICLR. https://openreview.net/forum?id=rJXMpikCZ

  38. Zhang J, Shi X, Xie J, Ma H, King I, Yeung D (2018) Gaan: Gated attention networks for learning on large and spatiotemporal graphs. In: Proceedings of the Thirty-Fourth Conference on Uncertainty in Artificial Intelligence, pp. 339–349

  39. Yu W, Zheng C, Cheng W, Aggarwal CC, Song D, Zong B, Chen H, Wang W (2018) Learning deep network representations with adversarially regularized autoencoders. In: Proceedings of SIGKDD, New York, NY, USA, pp. 2663–2671. https://doi.org/10.1145/3219819.3220000

  40. Hong SH, Ryu S, Lim J, Kim WY (2020) Molecular generative model based on an adversarially regularized autoencoder. J Chem Inf Model 60(1):29–36

    Article  MATH  Google Scholar 

  41. Yasunaga M, Zhang R, Meelu K, Pareek A, Srinivasan K, Radev D (2017) Graph-based neural multi-document summarization. In: Proceedings of CoNLL, pp. 452–462. https://doi.org/10.18653/v1/K17-1045

  42. Tan J, Wan X, Xiao J (2017) Abstractive document summarization with a graph-based attentional neural model. In: Proceedings of ACL, pp. 1171–1181. https://doi.org/10.18653/v1/P17-1108

  43. Fernandes P, Allamanis M, Brockschmidt M (2019) Structured neural summarization. In: Proceedings of ICLR. https://openreview.net/forum?id=H1ersoRqtm

  44. Damonte M, Cohen SB (2019) Structural neural encoders for AMR-to-text generation. In: Proceedings of NAACL, Minneapolis, Minnesota, pp. 3649–3658. https://doi.org/10.18653/v1/N19-1366

  45. Das D, Chen D, Martins AF, Schneider N, Smith NA (2014) Frame-semantic parsing. Computational linguistics 40(1):9–56

    Article  Google Scholar 

  46. Pan L, Xie Y, Feng Y, Chua T-S, Kan M-Y (2020) Semantic graphs for generating deep questions. In: Proceedings of ACL, pp. 1463–1475

  47. Li Y, Zemel R, Brockschmidt M, Tarlow D (2016) Gated graph sequence neural networks. In: Proceedings of ICLR

  48. Wang D, Hu W, Cao E, Sun W (2020) Global-to-local neural networks for document-level relation extraction. In: Proceedings of EMNLP, pp. 3711–3721. https://doi.org/10.18653/v1/2020.emnlp-main.303

  49. Yin Y, Meng F, Su J, Zhou C, Yang Z, Zhou J, Luo J (2020) A novel graph-based multi-modal fusion encoder for neural machine translation. In: Proceedings of ACL, pp. 3025–3035. https://doi.org/10.18653/v1/2020.acl-main.273

  50. Napoles C, Gormley M, Van Durme B (2012) Annotated gigaword. In: Proceedings of the Joint Workshop on Automatic Knowledge Base Construction and Web-scale Knowledge Extraction, pp. 95–100

  51. Over P, Dang H, Harman D (2007) Duc in context. Information Processing & Management 43(6):1506–1520

    Article  MATH  Google Scholar 

  52. Cao Z, Li W, Li S, Wei F (2018) Retrieve, rerank and rewrite: Soft template based neural summarization. In: Proceedings of ACL, pp. 152–161. https://doi.org/10.18653/v1/P18-1015

  53. Yan Y, Qi W, Gong Y, Liu D, Duan N, Chen J, Zhang R, Zhou M (2020) Prophetnet: Predicting future n-gram for sequence-to-sequence pre-training. arXiv preprint arXiv:2001.04063

  54. Chopra S, Auli M, Rush AM (2016) Abstractive sentence summarization with attentive recurrent neural networks. In: Proceedings of NAACL, pp. 93–98. https://doi.org/10.18653/v1/N16-1012

  55. Klein G, Kim Y, Deng Y, Senellart J, Rush AM (2017) Opennmt: Open-source toolkit for neural machine translation. arXiv preprint arXiv:1701.02810

  56. Gao Y, Wang Y, Liu L, Guo Y, Huang H (2019) Neural abstractive summarization fusing by global generative topics. Neural Computing and Applications, 1–10

  57. Zhang J, Zhao Y, Saleh M, Liu PJ (2019) PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization

  58. Song K, Tan X, Qin T, Lu J, Liu T-Y (2019) MASS: Masked sequence to sequence pre-training for language generation. In: Proceedings of ICML, vol. 97, pp. 5926–5936

  59. Dong L, Yang N, Wang W, Wei F, Liu X, Wang Y, Gao J, Zhou M, Hon H-W (2019) Unified language model pre-training for natural language understanding and generation. In: Advances in Neural Information Processing Systems, pp. 13063–13075

  60. Zhang Z, Han X, Liu Z, Jiang X, Sun M, Liu Q (2019) ERNIE: enhanced language representation with informative entities. In: Proceedings of ACL, pp. 1441–1451

  61. Guan S, Padmakumar V (2023) Extract, select and rewrite: A modular sentence summarization method. In: Dong, Y., Xiao, W., Wang, L., Liu, F., Carenini, G. (eds.) Proceedings of the 4th New Frontiers in Summarization Workshop, pp. 41–48. https://doi.org/10.18653/v1/2023.newsum-1.4. https://aclanthology.org/2023.newsum-1.4

  62. Wang K, Quan X, Wang R (2019) BiSET: Bi-directional selective encoding with template for abstractive summarization. In: Proceedings of ACL, Florence, Italy, pp. 2153–2162. https://doi.org/10.18653/v1/P19-1207

  63. Kouris P, Alexandridis G, Stafylopatis A (2021) Abstractive Text Summarization: Enhancing Sequence-to-Sequence Models Using Word Sense Disambiguation and Semantic Content Generalization. Comput Linguist 47(4):813–859

    Article  MATH  Google Scholar 

  64. Sakhare DY (2023) A sequence-to-sequence text summarization using long short-term memory based neural approach. International Journal of Intelligent Engineering & Systems 16(2)

  65. Li H, Zhu J, Zhang J, Zong C (2018) Ensure the correctness of the summary: Incorporate entailment knowledge into abstractive sentence summarization. In: Proceedings of COLING, Santa Fe, New Mexico, USA, pp. 1430–1441

  66. Kingma DP, Ba J (2014) Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980

  67. Louviere JJ, Flynn TN, Marley AAJ (2015) Best-Worst Scaling: Theory, Methods and Applications. Cambridge University Press, ???

  68. OpenAI: GPT-4 Technical Report (2023). https://arxiv.org/pdf/2303.08774.pdf

Download references

Funding

This work is supported by the National Natural Science Foundation of China Youth Project (No. 62406162), Fundamental Research Program of Shanxi Province (No.202403021211092), and Scientific and Technological Innovation Programs of Higher Education Institutions in Shanxi (No.2024L003).

Author information

Authors and Affiliations

  1. School of Computer & Information Technology, Shanxi University, Shanxi, China

    Yong Guan, Shaoru Guo & Ru Li

  2. Key Laboratory of Computational Intelligence and Chinese Information Processing of Ministry of Education, Shanxi University, Shanxi, China

    Ru Li

Authors
  1. Yong Guan
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Shaoru Guo
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Ru Li
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Shaoru Guo.

Ethics declarations

Conflict of interest

No Conflict of interest exists in the submission of this manuscript, and the manuscript is approved by all authors for publication.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Guan, Y., Guo, S. & Li, R. Structure-to-word dynamic interaction model for abstractive sentence summarization. Neural Comput & Applic 37, 6567–6581 (2025). https://doi.org/10.1007/s00521-024-10970-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-024-10970-0

Keywords