Skip to main content
SpringerLink
Log in

A Survey of Text Summarization Approaches Based on Deep Learning

  • Survey
  • Published:
Journal of Computer Science and Technology Aims and scope Submit manuscript

Abstract

Automatic text summarization (ATS) has achieved impressive performance thanks to recent advances in deep learning (DL) and the availability of large-scale corpora. The key points in ATS are to estimate the salience of information and to generate coherent results. Recently, a variety of DL-based approaches have been developed for better considering these two aspects. However, there is still a lack of comprehensive literature review for DL-based ATS approaches. The aim of this paper is to comprehensively review significant DL-based approaches that have been proposed in the literature with respect to the notion of generic ATS tasks and provide a walk-through of their evolution. We first give an overview of ATS and DL. The comparisons of the datasets are also given, which are commonly used for model training, validation, and evaluation. Then we summarize single-document summarization approaches. After that, an overview of multi-document summarization approaches is given. We further analyze the performance of the popular ATS models on common datasets. Various popular approaches can be employed for different ATS tasks. Finally, we propose potential research directions in this fast-growing field. We hope this exploration can provide new insights into future research of DL-based ATS.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Canhasi E. Graph-based models for multi-document summarization [Ph.D. Thesis]. Doktora Tezi, Ljubljana Universitesi, 2014.

  2. Mihalcea R, Tarau P. TextRank: Bringing order into text. In Proc. the 2004 Conference on Empirical Methods in Natural Language Processing, July 2004, pp.404-411.

  3. Hou S L, Huang Y, Fei C Q, Zhang S M, Lu R Q. Holographic lexical chain and its application in Chinese text summarization. In Proc. the 1st Asia-Pacific Web (APWeb) and Web-Age Information Management (WAIM) Joint Conference on Web and Big Data, July 2017, pp.266-281. DOI: 10.1007/978-3-319-63579-8_21.

  4. Berg-Kirkpatrick T, Gillick D, Klein D. Jointly learning to extract and compress. In Proc. the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, June 2011, pp.481-490.

  5. Gillick D, Favre B. A scalable global model for summarization. In Proc. the NAACL HLT Workshop on Integer Linear Programming for Natural Language Processing, June 2009, pp.10-18. DOI: https://doi.org/10.3115/1611638.1611640.

  6. Fattah M A. A hybrid machine learning model for multidocument summarization. Applied Intelligence, 2014, 40(4): 592-600. DOI: https://doi.org/10.1007/s10489-013-0490-0.

    Article  Google Scholar 

  7. Rush A M, Chopra S, Weston J. A neural attention model for abstractive sentence summarization. In Proc. the 2015 Conference on Empirical Methods in Natural Language Processing, 2015, pp.379-389. DOI: 10.18653/v1/d15-1044.

  8. Cheng J P, Lapata M. Neural summarization by extracting sentences and words. In Proc. the 54th Annual Meeting of the Association for Computational Linguistics, August 2016, pp.484-494. DOI: 10.18653/v1/p16-1046.

  9. Chu E, Liu P. MeanSum: A neural model for unsupervised multi-document abstractive summarization. In Proc. the 36th International Conference on Machine Learning, June 2019, pp.1223-1232.

  10. Young T, Hazarika D, Poria S, Cambria E. Recent trends in deep learning based natural language processing. IEEE Computational Intelligence Magazine, 2018, 13(3): 55-75. DOI: https://doi.org/10.1109/mci.2018.2840738.

    Article  Google Scholar 

  11. Lin C. ROUGE: A package for automatic evaluation of summaries. In Proc. the Workshop on Text Summarization Branches Out, July 2004, pp.74-81.

  12. Cheng J P, Dong L, Lapata M. Long short-term memory networks for machine reading. In Proc. the 2016 Conference on Empirical Methods in Natural Language Processing, November 2016, pp.551-561. DOI: 10.18653/v1/d16-1053.

  13. Cho K, Merrienboer B, Gulcehre C, Bahdanau D, Bougares F, Schwenk H, Bengio Y. Learning phrase representations using RNN encoder decoder for statistical machine translation. In Proc. the 2014 Conference on Empirical Methods in Natural Language Processing, October 2014, pp.1724-1734. DOI: 10.3115/v1/d14-1179.

  14. Kim Y. Convolutional neural networks for sentence classification. In Proc. the 2014 Conference on Empirical Methods in Natural Language Processing, October 2014, pp.1746-1751. DOI: 10.3115/v1/d14-1181.

  15. Kipf T N, Welling M. Semi-supervised classification with graph convolutional networks. arXiv:1609.02907, 2016. https://arxiv.org/abs/1609.02907, June 2020.

  16. Socher R, Perelygin A, Wu J, Chuang J, Manning C D, Ng A, Potts C. Recursive deep models for semantic compositionality over a sentiment treebank. In Proc. the 2013 Conference on Empirical Methods in Natural Language Processing, October 2013, pp.1631-1642.

  17. Li J W, Li R M, Hovy E. Recursive deep models for discourse parsing. In Proc. the 2014 Conference on Empirical Methods in Natural Language Processing, October 2014, pp.2061-2069. DOI: 10.3115/v1/d14-1220.

  18. Zhu X D, Sobihani P, Guo H Y. Long short-term memory over recursive structures. In Proc. the 32nd International Conference on Machine Learning, July 2015, pp.1604-1612.

  19. Lai S W, Xu L H, Liu K, Zhao J. Recurrent convolutional neural networks for text classification. In Proc. the 29th AAAI Conference on Artificial Intelligence, January 2015, pp.2267-2273.

  20. Sutskever I, Vinyals O, Le Q V. Sequence to sequence learning with neural networks. In Proc. the Annual Conference on Neural Information Processing Systems, December 2014, pp.3104-3112.

  21. Bahdanau D, Cho K, Bengio Y. Neural machine translation by jointly learning to align and translate. In Proc. the 3rd International Conference on Learning Representations, May 2015.

  22. Luong T, Pham H, Manning C D. Effective approaches to attention-based neural machine translation. In Proc. the 2015 Conference on Empirical Methods in Natural Language Processing, September 2015, pp.1412-1421. DOI: 10.18653/v1/d15-1166.

  23. Xu K, Ba J, Kiros R, Cho K, Courville A, Salakhudinov R, Zemel R, Bengio Y. Show, attend and tell: Neural image caption generation with visual attention. In Proc. the 32nd International Conference on Machine Learning, July 2015, pp.2048-2057.

  24. Paulus R, Xiong C, Socher R. A deep reinforced model for abstractive summarization. arXiv:1705.04304, 2017. https://arxiv.org/abs/1705.04304, June 2020.

  25. Nallapati R, Zhou B W, Santos C, Gulcehre C, Xiang B. Abstractive text summarization using sequence-to-sequence RNNs and beyond. In Proc. the 20th SIGNLL Conference on Computational Natural Language Learning, August 2016, pp.280-290. DOI: 10.18653/v1/k16-1028.

  26. See A, Liu P J, Manning C D. Get to the point: Summarization with pointer-generator networks. In Proc. the 55th Annual Meeting of the Association for Computational Lin-guistics, July 2017, pp.1073-1083. DOI: 10.18653/v1/p17-1099.

  27. Vinyals O, Fortunato M, Jaitly N. Pointer networks. In Proc. the Annual Conference on Neural Information Processing Systems, December 2015, pp.2692-2700.

  28. Gu J T, Lu Z D, Li H, Li V O. Incorporating copying mechanism in sequence-to-sequence learning. In Proc. the 54th Annual Meeting of the Association for Computational Linguistics, August 2016, pp.1631-1640. DOI: 10.18653/v1/p16-1154.

  29. Tu Z P, Lu Z D, Liu Y, Liu X H, Li H. Modeling coverage for neural machine translation. In Proc. the 54th Annual Meeting of the Association for Computational Linguistics, August 2016, pp.76-85. DOI: 10.18653/v1/p16-1008.

  30. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez A N, Kaiser L, Polosukhin I. Attention is all you need. In Proc. the Annual Conference on Neural Information Processing Systems, December 2017, pp.5998-6008.

  31. Hermann K M, Kocisky T, Grefenstette E, Espeholt L, Kay W, Suleyman M, Blunsom P. Teaching machines to read and comprehend. In Proc. the Annual Conference on Neural Information Processing Systems, December 2015, pp.1693-1701.

  32. Nallapati R, Zhai F F, Zhou B W. SummaRuNNer: A recurrent neural network based sequence model for extractive summarization of documents. In Proc. the 31st AAAI Conference on Artificial Intelligence, February 2017, pp.3075-3081.

  33. Sandhaus E. The New York times annotated corpus. Technical Report, The New York Times Company, Research and Development, 2008. https://catalog.ldc.upenn.edu/docs/LDC2008T19/new york times annotated corpus.pdf, June 2020.

  34. Durrett G, Berg-Kirkpatrick T, Klein D. Learning-based single-document summarization with compression and anaphoricity constraints. In Proc. the 54th Annual Meeting of the Association for Computational Linguistics, August 2016, pp.1998-2008. DOI: 10.18653/v1/p16-1188.

  35. Xu J C, Durrett G. Neural extractive text summarization with syntactic compression. arXiv:1902.00863, 2019. https://arxiv.org/abs/1902.00863, June 2020.

  36. Çelikyilmaz A, Bosselut A, He X, Choi Y. Deep communicating agents for abstractive summarization. In Proc. the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, June 2018, pp.1662-1675. DOI: 10.18653/v1/n18-1150.

  37. Liu P J, Saleh M, Pot E, Goodrich B, Sepassi R, Kaiser L, Shazeer N. Generating Wikipedia by summarizing long sequences. In Proc. the 6th International Conference on Learning Representations, May 2018.

  38. Liu Y, Lapata M. Hierarchical transformers for multidocument summarization. arXiv:1905.13164, 2019. https://arxiv.org/abs/1905.13164, June 2020.

  39. Perez-Beltrachini L, Liu Y, Lapata M. Generating summaries with topic templates and structured convolutional decoders. In Proc. the 57th Annual Meeting of the Association for Computational Linguistics, July 2019, pp.5107-5116. DOI: 10.18653/v1/p19-1504.

  40. Grusky M, Naaman M, Artzi Y. Newsroom: A dataset of 1.3 million summaries with diverse extractive strategies. arXiv:1804.11283, 2018. https://arxiv.org/abs/1804.11283, June 2020.

  41. Hu B T, Chen Q C, Zhu F Z. LCSTS: A large scale Chinese short text summarization dataset. In Proc. the 2015 Conference on Empirical Methods in Natural Language Processing, September 2015, pp.1967-1972. DOI: 10.18653/v1/d15-1229.

  42. Narayan S, Cohen S B, Lapata M. Don't give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization. In Proc. the 2018 Conference on Empirical Methods in Natural Language Processing, October 2018, pp.1797-1807. DOI: 10.18653/v1/d18-1206.

  43. Fabbri A R, Li I, She T W, Li S Y, Radev D R. Multi-news: A large-scale multi-document summarization dataset and abstractive hierarchical model. In Proc. the 57th Conference of the Association for Computational Linguistics, July 2019, pp.1074-1084. DOI: 10.18653/v1/p19-1102.

  44. Ranzato M, Chopra S, Auli M, Zaremba W. Sequence level training with recurrent neural networks. arXiv:1511.06732, 2015. https://arxiv.org/abs/1511.06732, June 2020.

  45. Chopra S, Auli M, Rush A M. Abstractive sentence summarization with attentive recurrent neural networks. In Proc. the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, June 2016, pp.93-98. DOI: 10.18653/v1/n16-1012.

  46. Takase S, Suzuki J, Okazaki N, Hirao T, Nagata M. Neural headline generation on abstract meaning representation. In Proc. the 2016 Conference on Empirical Methods in Natural Language Processing, November 2016, pp.1054-1059. DOI: 10.18653/v1/d16-1112.

  47. Wang C, Xue N W, Pradhan S. A transition-based algorithm for AMR parsing. In Proc. the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, May 2015, pp.366-375. DOI: 10.3115/v1/n15-1040.

  48. Tai K S, Socher R, Manning C D. Improved semantic representations from tree-structured long short-term memory networks. In Proc. the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, July 2015, pp.1556-1566. DOI: 10.3115/v1/p15-1150.

  49. Lopyrev K. Generating news headlines with recurrent neural networks. arXiv:1512.01712, 2015. https://arxiv.org/abs/1512.01712, June 2020.

  50. Goodfellow I, Bengio Y, Courville A. Deep Learning. MIT Press, 2016.

  51. Chen Q, Zhu X D, Ling Z H, Wei S, Jiang H. Distractionbased neural networks for modeling document. In Proc. the 25th International Joint Conference on Artificial Intelligence, July 2016, pp.2754-2760.

  52. Inan H, Khosravi K, Socher R. Tying word vectors and word classi_ers: A loss framework for language modeling. arXiv:1611.01462, 2016. https://arxiv.org/abs/1611.01462, June 2020.

  53. Tan J W, Wan X J, Xiao J G. Abstractive document summarization with a graph-based attentional neural model. In Proc. the 55th Annual Meeting of the Association for Computational Linguistics, July 2017, pp.1171-1181. DOI: 10.18653/v1/p17-1108.

  54. Gehrmann S, Deng Y, Rush A. Bottom-up abstractive summarization. In Proc. the 2018 Conference on Empirical Methods in Natural Language Processing, October 2018, pp.4098-4109. DOI: 10.18653/v1/d18-1443.

  55. Hsu W T, Lin C K, Lee M Y, Min K, Tang J, Sun M. A unified model for extractive and abstractive summarization using inconsistency loss. In Proc. the 56th Annual Meeting of the Association for Computational Linguistics, July 2018, pp.132-141. DOI: 10.18653/v1/p18-1013.

  56. Fan A, Grangier D, Auli M. Controllable abstractive summarization. In Proc. the 2nd Workshop on Neural Machine Translation and Generation, July 2018, pp.45-54. DOI: 10.18653/v1/w18-2706.

  57. Blei D M, Ng A Y, Jordan M I. Latent Dirichlet allocation. Journal of Machine Learning Research, 2003, 3(1): 993-1022.

    MATH  Google Scholar 

  58. Cai T, Shen M J, Peng H L, Jiang L, Dai Q. Improving transformer with sequential context representations for abstractive text summarization. In Proc. the 8th CCF International Conference on Natural Language Processing and Chinese Computing, October 2019, pp.512-524. DOI: 10.1007/978-3-030-32233-5_40.

  59. Gulcehre C, Ahn S, Nallapati R, Zhou B W, Bengio Y. Pointing the unknown words. In Proc. the 54th Annual Meeting of the Association for Computational Linguistics, 2016, pp.140-149. DOI: 10.18653/v1/p16-1014.

  60. Jean S, Cho K, Memisevic R, Bengio Y. On using very large target vocabulary for neural machine translation. In Proc. the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, July 2015, pp.1-10. DOI: 10.3115/v1/p15-1001.

  61. Zeng W Y, Luo W J, Fidler S, Urtasun R. Efficient summarization with read-again and copy mechanism. arXiv:1611.03382, 2016. https://arxiv.org/abs/1611.03382, June 2020.

  62. Li P J, Lam W, Bing L D, Wang Z H. Deep recurrent generative decoder for abstractive text summarization. In Proc. the 2017 Conference on Empirical Methods in Natural Language Processing, September 2017, pp.2091-2100. DOI: 10.18653/v1/d17-1222.

  63. Li W, Xiao X Y, Lyu Y J, Wang Y Z. Improving neural abstractive document summarization with explicit information selection modeling. In Proc. the 2018 Conference on Empirical Methods in Natural Language Processing, October 2018, pp.1787-1796. DOI: 10.18653/v1/d18-1205.

  64. Chen Y C, Bansal M. Fast abstractive summarization with reinforce-selected sentence rewriting. In Proc. the 56th Annual Meeting of the Association for Computational Linguistics, July 2018, pp.675-686. DOI: 10.18653/v1/p18-1063.

  65. Jiang X P, Hu P, Hou L W, Wang X. Improving pointergenerator network with keywords information for Chinese abstractive summarization. In Proc. the 7th CCF International Conference on Natural Language Processing and Chinese Computing, August 2018, pp.464-474. DOI: 10.1007/978-3-319-99495-6_39.

  66. Cohan A, Dernoncourt F, Kim D S, Bui T, Kim S, Chang W, Goharian N. A discourse-aware attention model for abstractive summarization of long documents. In Proc. the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, June 2018, pp.615-621. DOI: 10.18653/v1/n18-2097.

  67. Williams R J. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning, 1992, 8(3): 229-256. DOI: https://doi.org/10.1007/978-1-4615-3618-5_2.

    Article  MATH  Google Scholar 

  68. Miao Y S, Blunsom P. Language as a latent variable: Discrete generative models for sentence compression. In Proc. the 2016 Conference on Empirical Methods in Natural Language Processing, November 2016, pp.319-328. DOI: 10.18653/v1/d16-1031.

  69. Pasunuru R, Bansal M. Multi-reward reinforced summarization with saliency and entailment. In Proc. the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, June 2018, pp.646-653. DOI: 10.18653/v1/n18-2102.

  70. Sukhbaatar S, Szlam A, Fergus R. Learning multiagent communication with backpropagation. In Proc. the Annual Conference on Neural Information Processing Systems, December 2016, pp.2244-2252.

  71. Liu L Q, Lu Y, Yang M, Qu Q, Zhu J, Li H Y. Generative adversarial network for abstractive text summarization. In Proc. the 32nd AAAI Conference on Artificial Intelligence, February 2018, pp.8109-8110.

  72. Guo H, Pasunuru R, Bansal M. Soft layer-specific multitask summarization with entailment and question generation. In Proc. the 56th Annual Meeting of the Association for Computational Linguistics, July 2018, pp.687-697. DOI: 10.18653/v1/p18-1064.

  73. Cao Z Q, Wei F R, Li W J, Li S J. Faithful to the original: Fact aware neural abstractive summarization. In Proc. the 32nd AAAI Conference on Artificial Intelligence, February 2018, pp.4784-4791.

  74. Amplayo R K, Lim S, Hwang S. Entity commonsense representation for neural abstractive summarization. In Proc. the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, June 2018, pp.697-707. DOI: 10.18653/v1/n18-1064.

  75. Kryscinski W, Paulus R, Xiong C M, Socher R. Improving abstraction in text summarization. In Proc. the 2018 Conference on Empirical Methods in Natural Language Processing, October 2018, pp.1808-1817. DOI: 10.18653/v1/d18-1207.

  76. Zhang H Y, Cai J J, Xu J J, Wang J. Pretrainingbased natural language generation for text summarization. In Proc. the 23rd Conference on Computational Natural Language Learning, November 2019, pp.789-797. DOI: 10.18653/v1/k19-1074.

  77. Dong L, Yang N, Wang W H, Wei F R, Liu X D, Wang Y, Gao J F, Zhou M, Hon H. Unified language model pretraining for natural language understanding and generation. In Proc. the Annual Conference on Neural Information Processing Systems, December 2019, pp.13042-13054.

  78. Song K T, Tan X, Qin T, Lu J F, Liu T. MASS: Masked sequence to sequence pre-training for language generation. In Proc. the 36th International Conference on Machine Learning, June 2019, pp.5926-5936.

  79. Cao Z Q, Li W J, Li S J, Wei F R. Retrieve, rerank and rewrite: Soft template based neural summarization. In Proc. the 56th Annual Meeting of the Association for Computational Linguistics, July 2018, pp.152-161. DOI: 10.18653/v1/p18-1015.

  80. Wang L, Yao J L, Tao Y Z, Zhong L, Liu W, Du Q. A reinforced topic-aware convolutional sequence-to-sequence model for abstractive text summarization. In Proc. the 27th International Joint Conference on Artificial Intelligence, July 2018, pp.4453-4460. DOI: 10.24963/ijcai.2018/619.

  81. Jadhav A, Rajan V. Extractive summarization with SWAPNET: Sentences and words from alternating pointer networks. In Proc. the 56th Annual Meeting of the Association for Computational Linguistics, July 2018, pp.142-151. DOI: 10.18653/v1/p18-1014.

  82. Narayan S, Papasarantopoulos N, Cohen S B, Lapata M. Neural extractive summarization with side information. arXiv:1704.04530, 2017. https://arxiv.org/abs/1704.04530, June 2020.

  83. Isonuma M, Fujino T, Mori J, Matsuo Y, Sakata I. Extractive summarization using multi-task learning with document classification. In Proc. the 2017 Conference on Empirical Methods in Natural Language Processing, September 2017, pp.2101-2110. DOI: 10.18653/v1/d17-1223.

  84. Zhou Q Y, Yang N, Wei F R, Huang S H, Zhou M, Zhao T J. Neural document summarization by jointly learning to score and select sentences. In Proc. the 56th Annual Meeting of the Association for Computational Linguistics, July 2018, pp.654-663. DOI: 10.18653/v1/p18-1061.

  85. Zhang X X, Wei F R, Zhou M. Hibert: Document level pre-training of hierarchical bidirectional transformers for document summarization. arXiv:1905.06566, 2019. https://arxiv.org/abs/1905.06566, June 2020.

  86. Liu Y. Fine-tune BERT for extractive summarization. arXiv:1903.10318, 2019. https://arxiv.org/abs/1903.10318, June 2020.

  87. Narayan S, Cohen S B, Lapata M. Ranking sentences for extractive summarization with reinforcement learning. In Proc. the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, June 2018, pp.1747-1759. DOI: 10.18653/v1/n18-1158.

  88. Wu Y X, Hu B T. Learning to extract coherent summary via deep reinforcement learning. In Proc. the 32nd AAAI Conference on Artificial Intelligence, February 2018, pp.5602-5609.

  89. Dong Y, Shen Y K, Crawford E, van Hoof H, Cheung J C K. BanditSum: Extractive summarization as a contextual bandit. In Proc. the 2018 Conference on Empirical Methods in Natural Language Processing, October 2018, pp.3739-3748. DOI: 10.18653/v1/d18-1409.

  90. Zhang X X, Lapata M, Wei F R, Zhou M. Neural latent extractive document summarization. In Proc. the 2018 Conference on Empirical Methods in Natural Language Processing, October 2018, pp.779-784. DOI: 10.18653/v1/d18-1088.

  91. Chen X Y, Gao S, Tao C Y, Song Y, Zhao D Y, Yan R. Iterative document representation learning towards summarization with polishing. In Proc. the 2018 Conference on Empirical Methods in Natural Language Processing, October 2018, pp.4088-4097. DOI: 10.18653/v1/d18-1442.

  92. Sukhbaatar S, Weston J, Fergus R et al. End-to-end memory networks. In Proc. the Annual Conference on Neural Information Processing Systems, December 2015, pp.2440-2448.

  93. Peters M E, Neumann M, Iyyer M, Gardner M, Clark C, Lee K, Zettlemoyer L. Deep contextualized word representations. In Proc. the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, June 2018, pp.2227-2237.

  94. Kedzie C, McKeown K, Daume III H. Content selection in deep learning models of summarization. In Proc. the 2018 Conference on Empirical Methods in Natural Language Processing, October 2018, pp.1818-1828. DOI: 10.18653/v1/d18-1208.

  95. Carbonell J G, Goldstein J. The use of MMR, diversitybased reranking for reordering documents and producing summaries. In Proc. the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, August 1998, pp.335-336. DOI: 10.1145/290941.291025.

  96. Wang H, Wang X, Xiong W H, Yu M, Guo X X, Chang S Y, Wang W Y. Self-supervised learning for contextualized extractive summarization. In Proc. the 57th Conference of the Association for Computational Linguistics, July 2019, pp.2221-2227. DOI: 10.18653/v1/p19-1214.

  97. Kågebäck M, Mogren O, Tahmasebi N, Dubhashi D. Extractive summarization using continuous vector space models. In Proc. the 2nd Workshop on Continuous Vector Space Models and their Compositionality, April 2014, pp.31-39. DOI: 10.3115/v1/w14-1504.

  98. Socher R, Huang E H, Pennington J, Ng A Y, Manning C D. Dynamic pooling and unfolding recursive autoencoders for paraphrase detection. In Proc. 25th Annual Conference on Neural Information Processing Systems, December 2011, pp.801-809.

  99. Lin H, Bilmes J. A class of submodular functions for document summarization. In Proc. the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, June 2011, pp.510-520.

  100. YinWP, Pei Y L. Optimizing sentence modeling and selection for document summarization. In Proc. the 24th International Joint Conference on Artificial Intelligence, July 2015, pp.1383-1389.

  101. Mikolov T, Chen K, Corrado G, Dean J. Efficient estimation of word representations in vector space. arXiv:1301.3781, 2013. https://arxiv.org/abs/1301.3781, June 2020.

  102. Mnih A, Teh Y. A fast and simple algorithm for training neural probabilistic language models. In Proc. the 29th International Conference on Machine Learning, June 2012.

  103. Cao Z Q, Wei F R, Dong L, Li S J, Zhou M. Ranking with recursive neural networks and its application to multidocument summarization. In Proc. the 29th AAAI Conference on Artificial Intelligence, January 2015, pp.2153-2159.

  104. Cao Z Q, Li W J, Li S J, Wei F R. Improving multidocument summarization via text classification. In Proc. the 31st AAAI Conference on Artificial Intelligence, February 2017, pp.3053-3059.

  105. Christensen J, Soderland S, Etzioni O. Towards coherent multi-document summarization. In Proc. the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, June 2013, pp.1163-1173.

  106. Yasunaga M, Zhang R, Meelu K, Pareek A, Srinivasan K, Radev D. Graph-based neural multi-document summarization. In Proc. the 21st Conference on Computational Natural Language Learning, August 2017, pp.452-462. DOI: 10.18653/v1/k17-1045.

  107. Ren P J, Wei F R, Chen Z M, Ma J, Zhou M. A redundancy-aware sentence regression framework for extractive summarization. In Proc. the 26th International Conference on Computational Linguistics, December 2016, pp.33-43.

  108. Ren P J, Chen Z M, Ren Z C, Wei F R, Ma J, de Rijke M. Leveraging contextual sentence relations for extractive summarization using a neural attention model. In Proc. the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, August 2017, pp.95-104. DOI: 10.1145/3077136.3080792.

  109. Li P J, Wang Z H, Lam W, Ren Z C, Bing L D. Salience estimation via variational auto-encoders for multi-document summarization. In Proc. the 31st AAAI Conference on Artificial Intelligence, February 2017, pp.3497-3503.

  110. Cho S, Lebanoff L, Foroosh H, Liu F. Improving the similarity measure of determinantal point processes for extractive multi-document summarization. In Proc. the 57th Conference of the Association for Computational Linguistics, July 2019, pp.1027-1038. DOI: 10.18653/v1/p19-1098.

  111. Hinton G E, Sabour S, Frosst N. Matrix capsules with EM routing. In Proc. the 6th International Conference on Learning Representations, April 2018.

  112. Wang L, Ling W. Neural network-based abstract generation for opinions and arguments. In Proc. the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, June 2016, pp.47-57. DOI: 10.18653/v1/n16-1007.

  113. Zhang J M, Tan J W, Wan X J. Adapting neural single document summarization model for abstractive multidocument summarization: A pilot study. In Proc. the 11th International Conference on Natural Language Generation, November 2018, pp.381-390. DOI: 10.18653/v1/w18-6545.

  114. Lebanoff L, Song K, Liu F. Adapting the neural encoderdecoder framework from single to multi-document summarization. In Proc. the 2018 Conference on Empirical Methods in Natural Language Processing, October 2018, pp.4131-4141. DOI: 10.18653/v1/d18-1446.

  115. Nenkova A, Vanderwende L. The impact of frequency on summarization. Technical Report, Microsoft Research, 2005. https://www.cs.bgu.ac.il/~elhadad/nlp09/sumbasic. pdf, June 2020.

  116. Wu Y H, Schuster M, Chen Z F et al. Google's neural machine translation system: Bridging the gap between human and machine translation. arXiv:1609.08144, 2016. https://arxiv.org/abs/1609.08144, June 2020.

  117. Gehring J, Auli M, Grangier D, Yarats D, Dauphin Y N. Convolutional sequence to sequence learning. In Proc. the 34th International Conference on Machine Learning, August 2017, pp.1243-1252.

  118. Lebanoff L, Song K, Dernoncourt F, Kim D S, Kim S, Chang W, Liu F. Scoring sentence singletons and pairs for abstractive summarization. arXiv:1906.00077, 2019. https://arxiv.org/abs/1906.00077, June 2020.

  119. Devlin J, Chang M W, Lee K, Toutanova K. BERT: Pretraining of deep bidirectional transformers for language understanding. In Proc. the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, June 2019, pp.4171-4186.

  120. Chu E, Liu P J. Unsupervised neural multi-document abstractive summarization. arXiv:1810.05739, 2018. https://arxiv.org/abs/1810.05739, June 2020.

  121. Denkowski M, Lavie A. Meteor universal: Language specific translation evaluation for any target language. In Proc. the 9th Workshop on Statistical Machine Translation, June 2014, pp.376-380. DOI: 10.3115/v1/w14-3348.

  122. Schluter N. The limits of automatic summarisation according to ROUGE. In Proc. the 15th Conference of the European Chapter of the Association for Computational Linguistics, April 2017, pp.41-45. DOI: 10.18653/v1/e17-2007.

  123. Zhong M, Liu P F,Wang D Q, Qiu X P, Huang X J. Searching for effective neural extractive summarization: What works and what's next. In Proc. the 57th Conference of the Association for Computational Linguistics, July 2019, pp.1049-1058. DOI: 10.18653/v1/p19-1100.

  124. Lu R Q, Jin X L, Zhang S M, Qiu M K, Wu X D. A study on big knowledge and its engineering issues. IEEE Transactions on Knowledge and Data Engineering, 2018, 31(9): 1630-1644.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Computing Technology, Chinese Academy of Sciences, Beijing, 100190, China

    Sheng-Luan Hou, Chao-Qun Fei & Shu-Han Zhang

  2. University of Chinese Academy of Sciences, Beijing, 100049, China

    Sheng-Luan Hou, Xi-Kun Huang, Chao-Qun Fei, Shu-Han Zhang, Qi-Lin Sun & Chuan-Qing Wang

  3. Academy of Mathematics and Systems Sciences, Chinese Academy of Sciences, Beijing, 100190, China

    Xi-Kun Huang, Yang-Yang Li, Qi-Lin Sun & Chuan-Qing Wang

  4. Key Laboratory of Management, Decision and Information System, Chinese Academy of Sciences, Beijing, 100190, China

    Xi-Kun Huang, Yang-Yang Li, Qi-Lin Sun & Chuan-Qing Wang

Authors
  1. Sheng-Luan Hou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xi-Kun Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chao-Qun Fei
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shu-Han Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yang-Yang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Qi-Lin Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Chuan-Qing Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sheng-Luan Hou.

Supplementary Information

ESM 1

(PDF 139 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hou, SL., Huang, XK., Fei, CQ. et al. A Survey of Text Summarization Approaches Based on Deep Learning. J. Comput. Sci. Technol. 36, 633–663 (2021). https://doi.org/10.1007/s11390-020-0207-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11390-020-0207-x

Keywords

Search

Navigation