Abstract
Extractive summarization aims to extract sentences containing critical information from the original text, one of the mainstream methods for summarization. Generally, extractive summarization is regarded as a sentence binary classification task in many works. Still, the positive samples selected by these methods are incomplete, and the negative samples are composed of random single sentences, which leads to unsatisfactory classification results and incomplete abstract sentences. To address this issue, we propose a Dynamic Programming BERT (DP-BERT), which can dynamically select the positive example with the closest meaning of the reference abstract and adjusts the corresponding negative samples. Specifically, we design a selector responsible for the dynamic selection of positive and negative samples and then utilize the BERT pre-training model to fine-tune the sentence classifier. Extensive experiments show that DP-BERT can better extract the original text’s key sentences and achieve state-of-the-art performance on two widely-used benchmarks.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Chen, Y.C., Bansal, M.: Fast abstractive summarization with reinforce-selected sentence rewriting. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 675–686. Association for Computational Linguistics (2018)
Cheng, J., Lapata, M.: Neural summarization by extracting sentences and words. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 484–494. Association for Computational Linguistics (2016)
Cui, P., Hu, L., Liu, Y.: Enhancing extractive text summarization with topic-aware graph neural networks. In: Proceedings of the 28th International Conference on Computational Linguistics, COLING 2020, pp. 5360–5371. International Committee on Computational Linguistics (2020)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 4171–4186. Association for Computational Linguistics, Minneapolis, Minnesota (June 2019)
Erkan, G., Radev, D.R.: Lexrank: graph-based lexical centrality as salience in text summarization. J. Artif. Intell. Res. 22, 457–479 (2004)
Fernandes, P., Allamanis, M., Brockschmidt, M.: Structured neural summarization. In: 7th International Conference on Learning Representations, ICLR 2019. OpenReview.net (2019)
Haghighi, A., Vanderwende, L.: Exploring content models for multi-document summarization. In: Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pp. 362–370 (June 2009)
Hermann, K.M., et al.: Teaching machines to read and comprehend. In: Advances in Neural Information Processing Systems, vol. 28, pp. 1693–1701. Curran Associates, Inc. (2015)
Jones, K.S.: Automatic summarising: the state of the art. Inf. Process. Manag. 43(6), 1449–1481 (2007)
Lebanoff, L., et al.: Scoring sentence singletons and pairs for abstractive summarization. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 2175–2189. Association for Computational Linguistics (2019)
Liu, Y.: Fine-tune BERT for extractive summarization. CoRR abs/1903.10318 (2019)
Liu, Y., Lapata, M.: Text summarization with pretrained encoders. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 3730–3740. Association for Computational Linguistics (November 2019)
Liu, Y., Titov, I., Lapata, M.: Single document summarization as tree induction. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 1745–1755 (2019)
Liu, Z., Shi, K., Chen, N.F.: Conditional neural generation using sub-aspect functions for extractive news summarization. In: Cohn, T., He, Y., Liu, Y. (eds.) Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings, EMNLP 2020, pp. 1453–1463. Association for Computational Linguistics (2020)
Nallapati, R., Zhai, F., Zhou, B.: Summarunner: a recurrent neural network based sequence model for extractive summarization of documents. In: Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, pp. 3075–3081. AAAI Press (2017)
Narayan, S., Cohen, S.B., Lapata, M.: Don’t give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 1797–1807. Association for Computational Linguistics (2018)
Narayan, S., Cohen, S.B., Lapata, M.: Ranking sentences for extractive summarization with reinforcement learning. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pp. 1747–1759. Association for Computational Linguistics (2018)
Shen, D., Sun, J., Li, H., Yang, Q., Chen, Z.: Document summarization using conditional random fields. In: Veloso, M.M. (ed.) IJCAI 2007, Proceedings of the 20th International Joint Conference on Artificial Intelligence, pp. 2862–2867 (2007)
Tan, J., Wan, X., Xiao, J.: Abstractive document summarization with a graph-based attentional neural model. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1171–1181. Association for Computational Linguistics (2017)
Vanderwende, L., Suzuki, H., Brockett, C., Nenkova, A.: Beyond sumbasic: task-focused summarization with sentence simplification and lexical expansion. Inf. Process. Manage. 43(6), 1606–1618 (2007)
Yuan, R., Wang, Z., Li, W.: Fact-level extractive summarization with hierarchical graph mask on BERT. In: Proceedings of the 28th International Conference on Computational Linguistics, COLING 2020, pp. 5629–5639. International Committee on Computational Linguistics (2020)
Zhang, X., Wei, F., Zhou, M.: HIBERT: Document level pre-training of hierarchical bidirectional transformers for document summarization. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 5059–5069. Association for Computational Linguistics, Florence, Italy (July 2019)
Zheng, H., Lapata, M.: Sentence centrality revisited for unsupervised summarization. In: Korhonen, A., Traum, D.R., Màrquez, L. (eds.) Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Volume 1: Long Papers, pp. 6236–6247. Association for Computational Linguistics (2019)
Zhong, M., Liu, P., Chen, Y., Wang, D., Qiu, X., Huang, X.: Extractive summarization as text matching. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, pp. 6197–6208 (2020)
Acknowledgement
This research was partially supported by the Key Program of the National Natural Science Foundation of China under Grant No. U1903213, the Guangdong Basic and Applied Basic Research Foundation (No. 2019A1515011387). In addition, thanks to Tencent Pattern Recognition Center and Yankai Lin for their support in the research process of this article.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Cao, S., Yang, Y. (2021). DP-BERT: Dynamic Programming BERT for Text Summarization. In: Fang, L., Chen, Y., Zhai, G., Wang, J., Wang, R., Dong, W. (eds) Artificial Intelligence. CICAI 2021. Lecture Notes in Computer Science(), vol 13070. Springer, Cham. https://doi.org/10.1007/978-3-030-93049-3_24
Download citation
DOI: https://doi.org/10.1007/978-3-030-93049-3_24
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-93048-6
Online ISBN: 978-3-030-93049-3
eBook Packages: Computer ScienceComputer Science (R0)