Abstract
Recently, the use of Reinforcement Learning with Neural Networks in Abstractive Summarization is getting more popular, but still currently restricted. In this paper, we propose PEARL as a novel framework to expand the proficiency of Reinforcement Learning approach in Abstractive Text Summarization. PEARL consists of two out-of-the-box Reinforcement Learning algorithms: \(F_{Rouge}\) and \(D_{Threshold}\), where \(F_{Rouge}\) reconstructs the training objective, and \(D_{Threshold}\) helps to improve the flexibility for the arbitrary data. We evaluate PEARL in the large-scale CNN/DailyMail and the medium-scale VNTC-Abs datasets. Results show that our PEARL produces significantly greater Rouge scores than baselines as well as achieves the new state-of-the-art model without either pre-trained models or extra training data. This research provides proof of validity based on data analysis.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
CNN News: edition.cnn.com.
- 2.
VNExpress: vnexpress.net.
References
Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. ArXiv 1409, September 2014
Cho, J., Seo, M., Hajishirzi, H.: Mixture content selection for diverse sequence generation. In: EMNLP (2019)
Chopra, S., Auli, M., Rush, A.M.: Abstractive sentence summarization with attentive recurrent neural networks. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 93–98. Association for Computational Linguistics, San Diego, California, June 2016. https://doi.org/10.18653/v1/N16-1012, https://www.aclweb.org/anthology/N16-1012
Gehrmann, S., Deng, Y., Rush, A.: Bottom-up abstractive summarization. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 4098–4109. Association for Computational Linguistics, Brussels, Belgium, October-November 2018. https://doi.org/10.18653/v1/D18-1443, https://www.aclweb.org/anthology/D18-1443
Hermann, K.M., et al.: Teaching machines to read and comprehend. In: Advances in Neural Information Processing Systems, pp. 1693–1701 (2015)
Huang, L., Wu, L., Wang, L.: Knowledge graph-augmented abstractive summarization with semantic-driven cloze reward, pp. 5094–5107, January 2020. https://doi.org/10.18653/v1/2020.acl-main.457
Li, S., Lei, D., Qin, P., Wang, W.Y.: Deep reinforcement learning with distributional semantic rewards for abstractive summarization. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 6038–6044. Association for Computational Linguistics, Hong Kong, China, November 2019. https://doi.org/10.18653/v1/D19-1623, https://www.aclweb.org/anthology/D19-1623
Lin, C.Y.: ROUGE: a package for automatic evaluation of summaries. In: Text Summarization Branches Out, pp. 74–81. Association for Computational Linguistics, Barcelona, Spain, July 2004. https://www.aclweb.org/anthology/W04-1013
Luketina, J., et al.: A survey of reinforcement learning informed by natural language (2019)
Nallapati, R., Zhou, B., dos Santos, C., Caglar, G., Xiang, B.: Abstractive text summarization using sequence-to-sequence RNNs and beyond. In: Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning, pp. 280–290. Association for Computational Linguistics, Berlin, Germany, August 2016. https://doi.org/10.18653/v1/K16-1028, https://www.aclweb.org/anthology/K16-1028
Nguyen, T.P.N., Tran, N.T.: Contour: penalty and spotlight mask for abstractive summarization. In: Hong, T.P., Wojtkiewicz, K., Chawuthai, R., Sitek, P. (eds.) Recent Challenges in Intelligent Information and Database Systems, pp. 174–187. Springer Singapore, Singapore (2021)
Paulus, R., Xiong, C., Socher, R.: A deep reinforced model for abstractive summarization (2017)
Raychaudhuri, S.: Introduction to Monte Carlo simulation. In: 2008 Winter Simulation Conference, pp. 91–100 (2008). https://doi.org/10.1109/WSC.2008.4736059
Rennie, S.J., Marcheret, E., Mroueh, Y., Ross, J., Goel, V.: Self-critical sequence training for image captioning (2017)
Rush, A.M., Chopra, S., Weston, J.: A neural attention model for abstractive sentence summarization. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 379–389. Association for Computational Linguistics, Lisbon, Portugal, September 2015. https://doi.org/10.18653/v1/D15-1044, https://www.aclweb.org/anthology/D15-1044
See, A., Liu, P.J., Manning, C.D.: Get to the point: summarization with pointer-generator networks. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1073–1083. Association for Computational Linguistics, Vancouver, Canada, July 2017. https://doi.org/10.18653/v1/P17-1099, https://www.aclweb.org/anthology/P17-1099
Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. CoRR abs/1409.3215 (2014). http://arxiv.org/abs/1409.3215
Vaswani, A., et al.: Attention is all you need. In: Guyon, I., et al. (eds.) Advances in Neural Information Processing Systems, vol. 30, pp. 5998–6008. Curran Associates, Inc. (2017). http://papers.nips.cc/paper/7181-attention-is-all-you-need.pdf
Wang, W.Y., Li, J., He, X.: Deep reinforcement learning for NLP. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics: Tutorial Abstracts, pp. 19–21. Association for Computational Linguistics, Melbourne, Australia, July 2018. https://doi.org/10.18653/v1/P18-5007, https://www.aclweb.org/anthology/P18-5007
Williams, R.J.: Simple statistical gradient-following algorithms for connectionist reinforcement learning. Mach. Learn. 8, 229–256 (1992)
Acknowledgement
We thank the anonymous reviewers for their helpful suggestions on this paper. This research was supported by the Department of Knowledge Engineering funded by the Faculty of Information Technology under grant number CNTT 2020-12 from Vietnam National University, Ho Chi Minh City University of Science.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Nguyen, TP.N., Van, NC., Tran, NT. (2021). Performance-Driven Reinforcement Learning Approach for Abstractive Text Summarization. In: Pham, D.N., Theeramunkong, T., Governatori, G., Liu, F. (eds) PRICAI 2021: Trends in Artificial Intelligence. PRICAI 2021. Lecture Notes in Computer Science(), vol 13032. Springer, Cham. https://doi.org/10.1007/978-3-030-89363-7_14
Download citation
DOI: https://doi.org/10.1007/978-3-030-89363-7_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-89362-0
Online ISBN: 978-3-030-89363-7
eBook Packages: Computer ScienceComputer Science (R0)