Skip to main content
SpringerLink
Log in

A global and local information extraction model incorporating selection mechanism for abstractive text summarization

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

A global and local information extraction model incorporating selection mechanism is presented to address the concerns of insufficient semantic coding and redundant semantic information in the abstractive summary. The model, unlike single coding, encodes the source text twice. The global semantic information is extracted by the encoder based on Bidirectional Gated Recurrent Unit network, while the local feature vector is extracted by the encoder based on Dilated Convolution Network. The selection gate is in charge of filtering out redundant data. The output of the two encoders is fused as the input of the decoder to improve the feature representation at the source to generate diversified summaries. The model has good performance and effectively enhances the quality of summary, according on the experimental findings on two tough datasets, CNN/DailyMail and DUC 2004.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. https://www.tensorflow.org/

References

  1. Bahdanau D, Cho K, Bengio Y (2014) Neural machine translation by jointly learning to align and translate. Computer Science

  2. Bansal M, Lobiyal D K (2021) Multilingual sequence to sequence convolutional machine translation. Multimed Tools Applic 80(25):33701–33726. https://doi.org/10.1007/s11042-021-11345-6

    Article  Google Scholar 

  3. Chen L C, Papandreou G, Schroff F, Adam H (2017) Rethinking atrous convolution for semantic image segmentation

  4. Chen L-C, Zhu Y, Papandreou G, Schroff F, Adam H (2018) Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Ferrari V, Hebert M, Sminchisescu C, Weiss Y (eds) Computer Vision - ECCV 2018, PT VII, vol 11211. Springer International Publishing AG, Cham, pp 833–851

  5. Chopra S, Auli M, Rush A M (2016) Abstractive sentence summarization with attentive recurrent neural networks. In: Conference of the North American chapter of the association for computational linguistics: human language technologies

  6. Ding J, Li Y, Wang J (2019) Automatic summarization method of short text based on dual encoder. Comput Applic 39(12):6

    Google Scholar 

  7. Fudholi D H, Nayoan R A N, Hidayatullah A F, Arianto D B (2022) A hybrid cnn-bilstm model for drug named entity recognition. J Eng Sci Technol 17(1):730–744

    Google Scholar 

  8. Gambhir M, Gupta V Deep learning-based extractive text summarization with word-level attention mechanism. Multimedia Tools And Applications

  9. Gao W, Ma H, Li D, Yu P (2021) Research and implementation of chinese text summarization technology based on dual encoder. Comput Eng Des 42(9):9

    Google Scholar 

  10. Gehring J, Auil M, Grangier D, Yarats D, Dauphin Y N (2017) Convolutional sequence to sequence learning. In: Precup D, Teh YW (eds) International conference on machine learning, vol 70. JMLR-Journal Machine Learning Research, San Diego

  11. Ghosh R A recurrent neural network based deep learning model for text and non-text stroke classification in online handwritten devanagari document. Multimedia Tools And Applications. https://doi.org/10.1007/s11042-022-12767-6

  12. He J, Zhang S, Yang M, Shan Y, Huang T (2022) Bdcn: Bi-directional cascade network for perceptual edge detection. IEEE Trans Pattern Anal Mach Intell 44:100–113. https://doi.org/10.1109/TPAMI.2020.3007074

    Article  Google Scholar 

  13. Hermann K M, Kocisky T, Grefenstette E, Espeholt L, Kay W, Suleyman M, Blunsom P (2015) Teaching machines to read and comprehend. In: Cortes C, Lawrence ND, Lee DD, Sugiyama M, Garnett R (eds) Advances in neural information processing systems 28 (NIPS 2015), vol 28. Neural Information Processing Systems (NIPS), La Jolla

  14. Hinton G E, Salakhutdinov R R (2006) Reducing the dimensionality of data with neural networks. Science 313:504–507. https://doi.org/10.1126/science.1127647

    Article  MathSciNet  Google Scholar 

  15. Hu B, Chen Q, Zhu F (2015) Lcsts: a large scale Chinese short text summarization dataset

  16. Huang Y, Yu Z, Guo J, Xiang Y, Yu Z, Xian Y (2022) Abstractive document summarization via multi-template decoding. Appl Intell 52:9650–9663. https://doi.org/10.1007/s10489-021-02607-9

    Article  Google Scholar 

  17. Kalchbrenner N, Espeholt L, Simonyan K, Oord A, Graves A, Kavukcuoglu K (2016) Neural machine translation in linear time

  18. Kedzie C, Mckeown K, H D III (2018) Content selection in deep learning models of summarization

  19. Kim Y (2014) Convolutional neural networks for sentence classification. Eprint Arxiv

  20. Liang J, Du M (2022) Two-way neural network chinese-english machine translation model fused with attention mechanism. Sci Program, 2022. https://doi.org/10.1155/2022/1270700

  21. Liang Z, Du J, Li C (2020) Abstractive social media text summarization using selective reinforced seq2seq attention model. Neurocomputing 410:432–440. https://doi.org/10.1016/j.neucom.2020.04.137

    Article  Google Scholar 

  22. Liao K, Lebanoff L, Liu F (2018) Abstract meaning representation for multi-document summarization

  23. Lin C-Y (2004) Rouge: a package for automatic evaluation of summaries. In: Workshop on text summarization branches out, post-conference workshop of ACL 2004, Barcelona, Spain, pp 74–81. https://www.microsoft.com/en-us/research/publication/rouge-a-package-for-automatic-evaluation-of-summaries/

  24. Lin H, Ng V (2019) Abstractive summarization: a survey of the state of the art. In: Thirty-Third AAAI conference on artificial intelligence / thirty-first innovative applications of artificial intelligence conference / Ninth AAAI symposium on educational advances in artificial intelligence, Assoc Advancement Artificial Intelligence. Assoc Advancement Artificial Intelligence, Palo Alto, pp 9815–9822

  25. Lin J, Sun X, Ma S, Su Q (2018) Global encoding for abstractive summarization. In: Gurevych I, Miyao Y (eds) Proceedings of the 56th annaul meeting of the association for computational linguistics, vol 2. Assoc Computational Linguistics-ACL, Stroudsburg, pp 163–169

  26. Liu S, Huang D, Wang Y (2018) Receptive field block net for accurate and fast object detection. In: Ferrari V, Hebert M, Sminchisescu C, Weiss Y (eds) Computer Vision - ECCV 2018, PT XI, vol 11215. Springer International Publishing AG, Cham, pp 404–419

  27. Ma T, Pan Q, Rong H, Qian Y, Tian Y, Al-Nabhan N (2022) T-bertsum: topic-aware text summarization based on bert. IEEE Trans Comput Soc Syst 9(3):879–890. https://doi.org/10.1109/TCSS.2021.3088506

    Article  Google Scholar 

  28. Nallapati R, Zhou B, Santos C, Gulcehre C, Bing X (2016) Abstractive text summarization using sequence-to-sequence rnns and beyond

  29. Nallapati R, Zhai F, Zhou B (2017) Summarunner: a recurrent neural network based sequence model for extractive summarization of documents. In: Thirty-First AAAI conference on artificial intelligence, Assoc Advancement Artificial Intelligence. Assoc Advancement Artificial Intelligence, PALO ALTO, pp 3075–3081

  30. Napoles C, Gormley M, Durme B V (2012) Annotated gigaword. In: Proceedings of the joint workshop on automatic knowledge base construction and web-scale knowledge extraction

  31. Niu G, Xu H, He B, Xiao X, Wu H, Gao S (2019) Enhancing local feature extraction with global representation for neural text classification. In: Inui K, Jiang J, Ng V, Wan X (eds) Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3-7, 2019. Association for Computational Linguistics, pp 496–506, DOI https://doi.org/10.18653/v1/D19-1047

  32. Oord A, Dieleman S, Zen H, Simonyan K, Kavukcuoglu K (2016) Wavenet: a generative model for raw audio

  33. Over P, Dang H, Harman D (2007) Duc in context. Inf Process Manag 43:1506–1520. https://doi.org/10.1016/j.ipm.2007.01.019

    Article  Google Scholar 

  34. Qiu D, Yang B (2022) Text summarization based on multi-head self-attention mechanism and pointer network. Complex Intell Syst 8:555–567. https://doi.org/10.1007/s40747-021-00527-2

    Article  Google Scholar 

  35. Rahman M M, Siddiqui F H (2021) Multi-layered attentional peephole convolutional lstm for abstractive text summarization. ETRI J 43 (2):288–298. https://doi.org/10.4218/etrij.2019-0016

    Article  Google Scholar 

  36. Rani R, Lobiyal D K (2021) An extractive text summarization approach using tagged-lda based topic modeling. Multimed Tools Applic 80(3):3275–3305

    Article  Google Scholar 

  37. Rani R, Lobiyal D K (2021) A weighted word embedding based approach for extractive text summarization. Exp Syst Applic, 186. https://doi.org/10.1016/j.eswa.2021.115867

  38. Rush A M, Chopra S, Weston J (2015) A neural attention model for abstractive sentence summarization. Computer Science

  39. See A, Liu P J, Manning C D (2017) Get to the point: summarization with pointer-generator networks. In: Barzilay R, Kan MY (eds) Proceedings of the 55th annual mtting of the association for computation linguistics (ACL 2017), vol 1. Assoc Computational Linguistics-Acl, Stroudsburg, pp 1073–1083

  40. Shaojie Bai V K (2018) An empirical evaluation of generic convolutional and recurrent networks for sequence modeling

  41. Song S, Huang H, Ruan T (2019) Abstractive text summarization using lstm-cnn based deep learning. Multimed Tools Applic 78(1):857–875. https://doi.org/10.1007/s11042-018-5749-3

    Article  Google Scholar 

  42. Takase S, Suzuki J, Okazaki N, Hirao T, Nagata M (2016) Neural headline generation on abstract meaning representation. In: Proceedings of the 2016 conference on empirical methods in natural language processing

  43. Wang B (2018) Disconnected recurrent neural networks for text categorization. In: Gurevych I, Miyao Y (eds) Proceedings of the 56th annual meeting of the Association for Computional Linguistics (ACL), vol 1. Assoc Computational Linguistics-ACL, Stroudsburg, pp 2311–2320

  44. Wang W, Chang B (2016) Graph-based dependency parsing with bidirectional lstm. In: Proceedings of the 54th annual meeting of the association for computational linguistics (volume 1: long papers)

  45. Wang Q, Ren J (2021) Summary-aware attention for social media short text abstractive summarization. Neurocomputing 425:290–299. https://doi.org/10.1016/j.neucom.2020.04.136

    Article  Google Scholar 

  46. Wang P, Chen P, Yuan Y, Liu D, Huang Z, Hou X, Cottrell G (2018) Understanding convolution for semantic segmentation. In: 2018 IEEE Winter Conference on Applications of Computer Vision (WACV 2018). IEEE, New York, pp 1451–1460

  47. Wang K, Quan X, Wang R (2019) Biset: Bi-directional selective encoding with template for abstractive summarization. In: Korhonen A, Traum D, Marquez L (eds) 57th Annual Meeting of the Association for Computational Linguistics (ACL 2019). Assoc Computational Linguistics-ACL, Stroudsburg, pp 2153–2162

  48. Wang L, Yang M, Li C, Shen Y, Xu R (2021) Abstractive text summarization with hierarchical multi-scale abstraction modeling and dynamic memory. Assoc Computing Machinery, New York, pp 2086–2090

  49. Xiao W, Carenini G (2019) Extractive summarization of long documents by combining global and local context

  50. Xu W, Li C, Lee M, Zhang C (2020) Multi-task learning for abstractive text summarization with key information guide network. EURASIP J Adv Signal Process 2020:1. https://doi.org/10.1186/s13634-020-00674-7

    Article  Google Scholar 

  51. Yadav V, Bethard S (2019) A survey on recent advances in named entity recognition from deep learning models

  52. Yao K, Zhang L, Du D, Luo T, Tao L, Wu Y (2020) Dual encoding for abstractive text summarization. IEEE Trans Cybern 50:985–996. https://doi.org/10.1109/TCYB.2018.2876317

    Article  Google Scholar 

  53. Yu F, Koltun V (2016) Multi-scale context aggregation by dilated convolutions

  54. Zhang Z, Wang X, Jung C (2019) Dcsr: dconvolutions for single image super-resolution. IEEE Trans Image Proceed 28:1625–1635. https://doi.org/10.1109/TIP.2018.2877483

    Article  Google Scholar 

  55. Zhou Q, Yang N, Wei F, Zhou M (2017) Selective encoding for abstractive sentence summarization. In: Barzilay R, Kan MY (eds) Proceedings of the 55th annual meeting of the assocaition for computational linguistics (ACL 2017), vol 1. Assoc Computational Linguistics-ACL, Stroudsburg, pp 1095–1104

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (Grant Nos. 61802107); Handan science and technology R & D plan project(Grant Nos. 21422093285); Hebei Natural Science Foundation (Youth) project (Grant Nos. D2021402043).

Funding

Funding This work was supported by the National Natural Science Foundation of China (Grant Nos. 61802107); Handan science and technology R & D plan project(Grant Nos. 21422093285); Hebei Natural Science Foundation (Youth) project (Grant Nos. D2021402043).

Author information

Author notes

    Authors and Affiliations

    1. School of Information and Electrical Engineering, Hebei University of Engineering, Congtai, Handan, 056038, Hebei, China

      Yuanyuan Li, Yuan Huang, Weijian Huang & Wei Wang

    Authors
    1. Yuanyuan Li
      View author publications

      You can also search for this author in PubMed Google Scholar

    2. Yuan Huang
      View author publications

      You can also search for this author in PubMed Google Scholar

    3. Weijian Huang
      View author publications

      You can also search for this author in PubMed Google Scholar

    4. Wei Wang
      View author publications

      You can also search for this author in PubMed Google Scholar

    Corresponding author

    Correspondence to Yuanyuan Li.

    Ethics declarations

    Conflict of Interests

    The authors have no relevant financial or non-financial interests to disclose.

    Additional information

    Publisher’s note

    Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

    Yuan Huang, Weijian Huang and Wei Wang contributed equally to this work.

    Rights and permissions

    Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

    Reprints and permissions

    About this article

    Check for updates. Verify currency and authenticity via CrossMark

    Cite this article

    Li, Y., Huang, Y., Huang, W. et al. A global and local information extraction model incorporating selection mechanism for abstractive text summarization. Multimed Tools Appl 83, 4859–4886 (2024). https://doi.org/10.1007/s11042-023-15274-4

    Download citation

    • Received:

    • Revised:

    • Accepted:

    • Published:

    • Issue Date:

    • DOI: https://doi.org/10.1007/s11042-023-15274-4

    Keywords

    Search

    Navigation