Abstract
As one of the most essential tasks for information aggregation, multi-document summarization is faced with information redundancy of source document clusters. Recent works have attempted to avoid redundancy while generating summaries. Most state-of-the-art multi-document summarization systems are either extractive or abstractive with an external extractive model. In this paper, we propose an end-to-end abstractive model based on Transformer to generate summaries, considering relevance and redundancy dynamically and jointly. Specifically, we employ sentence masks and design a sentence-level transformer layer for learning sentence representations in a hierarchical manner. Then we use a dynamic Max Marginal Relevance (MMR) model to discern summary-worthy sentences and modify the encoder-decoder attention. We also utilize the pointer mechanism, taking the mean attention of all transformer heads as the probability to copy words from the source text. Experimental results demonstrate that our proposed model outperforms several strong baselines. We also conduct ablation studies to verify the effectiveness of our key mechanisms.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Angelidis, S., Lapata, M.: Summarizing opinions: aspect extraction meets sentiment prediction and they are both weakly supervised. In: EMNLP, pp. 3675–3686. Association for Computational Linguistics (2018)
Antognini, D., Faltings, B.: Learning to create sentence semantic relation graphs for multi-document summarization. In: EMNLP-IJCNLP 2019, vol. 32 (2019)
Cao, Z., Li, W., Li, S., Wei, F.: Improving multi-document summarization via text classification. In: AAAI, pp. 3053–3059. AAAI Press (2017)
Carbonell, J.G., Goldstein, J.: The use of MMR, diversity-based reranking for reordering documents and producing summaries. In: SIGIR, pp. 335–336. ACM (1998)
Cho, S., Lebanoff, L., Foroosh, H., Liu, F.: Improving the similarity measure of determinantal point processes for extractive multi-document summarization. In: ACL, vol. 1, pp. 1027–1038. Association for Computational Linguistics (2019)
Christensen, J., Mausam, Soderland, S., Etzioni, O.: Towards coherent multi-document summarization. In: HLT-NAACL, pp. 1163–1173. The Association for Computational Linguistics (2013)
Chu, E., Liu, P.J.: MeanSum: a neural model for unsupervised multi-document abstractive summarization. In: Proceedings of Machine Learning Research, PMLR, vol. 97, pp. 1223–1232. ICML (2019)
Erkan, G., Radev, D.R.: LexRank: graph-based lexical centrality as salience in text summarization. J. Artif. Intell. Res. 22, 457–479 (2004)
Fabbri, A.R., Li, I., She, T., Li, S., Radev, D.R.: Multi-news: a large-scale multi-document summarization dataset and abstractive hierarchical model. In: ACL, 1, pp. 1074–1084. Association for Computational Linguistics (2019)
Gehrmann, S., Deng, Y., Rush, A.M.: Bottom-up abstractive summarization. In: EMNLP, pp. 4098–4109. Association for Computational Linguistics (2018)
Hao, T.: Overview of DUC 2005. In: Proceedings of the Document Understanding Conference (DUC 2005) (2005)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: ICLR (Poster) (2015)
Lebanoff, L., Song, K., Liu, F.: Adapting the neural encoder-decoder framework from single to multi-document summarization. In: EMNLP, pp. 4131–4141. Association for Computational Linguistics (2018)
Lin, C.Y.: ROUGE: a package for automatic evaluation of summaries. In: Text Summarization Branches Out, pp. 74–81 (2004)
Liu, P.J., et al.: Generating wikipedia by summarizing long sequences. In: ICLR (Poster). OpenReview.net (2018)
Liu, Y., Lapata, M.: Hierarchical transformers for multi-document summarization. In: ACL, vol. 1, pp. 5070–5081. Association for Computational Linguistics (2019)
Louviere, J.J., Flynn, T.N., Marley, A.A.J.: Best-Worst Scaling: Theory Methods and Applications. Cambridge University Press, Cambridge (2015)
Luo, W., Liu, F., Liu, Z., Litman, D.J.: Automatic summarization of student course feedback. In: HLT-NAACL, pp. 80–85. The Association for Computational Linguistics (2016)
Mihalcea, R., Tarau, P.: TextRank: bringing order into text. In: EMNLP, pp. 404–411. ACL (2004)
See, A., Liu, P.J., Manning, C.D.: Get to the point: summarization with pointer-generator networks. In: ACL, vol. 1, pp. 1073–1083. Association for Computational Linguistics (2017)
Vaswani, A., et al.: Attention is all you need. In: NIPS, pp. 5998–6008 (2017)
Vinyals, O., Fortunato, M., Jaitly, N.: Pointer networks. In: NIPS, pp. 2692–2700 (2015)
Yang, Z., Dai, Z., Yang, Y., Carbonell, J.G., Salakhutdinov, R., Le, Q.V.: XLNet: generalized autoregressive pretraining for language understanding. In: NeurIPS, pp. 5754–5764 (2019)
Yasunaga, M., Zhang, R., Meelu, K., Pareek, A., Srinivasan, K., Radev, D.R.: Graph-based neural multi-document summarization. In: CoNLL, pp. 452–462. Association for Computational Linguistics (2017)
Zhang, J., Tan, J., Wan, X.: Adapting neural single-document summarization model for abstractive multi-document summarization: a pilot study. In: INLG, pp. 381–390. Association for Computational Linguistics (2018)
Acknowledgments
This research work has been funded by the National Natural Science Foundation of China (Grant No. 61772337, U1736207).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Liu, Y., Fan, X., Zhou, J., He, C., Liu, G. (2020). Learning to Consider Relevance and Redundancy Dynamically for Abstractive Multi-document Summarization. In: Zhu, X., Zhang, M., Hong, Y., He, R. (eds) Natural Language Processing and Chinese Computing. NLPCC 2020. Lecture Notes in Computer Science(), vol 12430. Springer, Cham. https://doi.org/10.1007/978-3-030-60450-9_38
Download citation
DOI: https://doi.org/10.1007/978-3-030-60450-9_38
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-60449-3
Online ISBN: 978-3-030-60450-9
eBook Packages: Computer ScienceComputer Science (R0)
