Abstract
Poetry generation has been a classic natural language generation task recently. But so far the methods for this topic mainly imitate and reproduce the poems on the training data set, which indicates that they either have not much connotation or overfit too much like plagiarism of the existing poems. To solve this problem, unlike previous work, instead of tuning the trade-off between connotation and innovation, we propose a distributed reinforcement learning framework, which consists of two stages of training, to generate creative and meaningful poetry. At the first stage we train a model in parallel on a large poetry corpus at word level to master how poets write poems. At the second stage we train the model with a distributed architecture to learn how connotation is developed in human literary art works at sentence level and force the model to imitate itself when it composes some ‘good poems’ to further improve performance. Experiments on generating classical Chinese poetry demonstrate that the proposed model is able to achieve better performance and the high efficiency of training compared to the state-of-the-art.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
The data sets are publicly available from: https://github.com/chinese-poetry/chinese-poetry.
- 2.
For GPT-based method, User may have to register a Wechat account and add
or
. - 3.
seqGAN code is available from: https://github.com/LantaoYu/SeqGAN.
- 4.
Jiuge is available from: http://118.190.162.99:8080/.
- 5.
The Natural Language Processing Group at the Department of Computer Science and Technology, Tsinghua University.
or
.References
Chen, H., Yi, X., Sun, M., Li, W., Yang, C., Guo, Z.: Sentiment-controllable Chinese poetry generation. In: IJCAI, pp. 4925–4931 (2019)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Horgan, D., et al.: Distributed prioritized experience replay (2018)
Liang, S.: Unsupervised semantic generative adversarial networks for expert retrieval. In: WWW (2019)
Liao, Y., Wang, Y., Liu, Q., Jiang, X.: Gpt-based generation for classical chinese poetry. arXiv preprint arXiv:1907.00151 (2019)
Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. Computer ence (2013)
Mikolov, T., Karafiát, M., Burget, L., Cernock, J., Khudanpur, S.: Recurrent neural network based language model. In: INTERSPEECH 2010 (2010)
Oh, J., Guo, Y., Singh, S., Lee, H.: Self-imitation learning. arXiv preprint arXiv:1806.05635 (2018)
Papineni, K., Roukos, S., Ward, T., Zhu, W.J.: Bleu: a method for automatic evaluation of machine translation, October 2002. https://doi.org/10.3115/1073083.1073135
Radford, A., Narasimhan, K., Salimans, T., Sutskever, I.: Improving language understanding by generative pre-training (2018)
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal policy optimization algorithms (2017)
Song, Y., Shi, S., Li, J., Zhang, H.: Directional skip-gram: explicitly distinguishing left and right context for word embeddings. In: Proceedings of ACL 2018 (2018)
Sun, M., Yi, X., Li, W.: Stylistic chinese poetry generation via unsupervised style disentanglement, pp. 3960–3969, January 2018. https://doi.org/10.18653/v1/D18-1430
Sutton, R., Barto, A.: Reinforcement Learning: An Introduction. MIT Press, United States (1998)
Wang, Z., et al.: Chinese poetry generation with planning based neural network. arXiv:1610.09889 (2016)
Yan, R., Jiang, H., Lapata, M., Lin, S.D., Lv, X., Li, X.: I, poet: automatic chinese poetry composition through a generative summarization framework under constrained optimization. In: 23rd IJCAI (2013)
Yi, X., Li, R., Sun, M.: Chinese poetry generation with a salient-clue mechanism. CoNLL, 241–250 (2018)
Yi, X., Sun, M., Li, R., Li, W.: Automatic poetry generation with mutual reinforcement learning. Proc. EMNLP 2018, 3143–3153 (2018)
Yi, X., Sun, M., Li, R., Zonghan, Y.: Chinese poetry generation with a working memory model, September 2018
Yu, L., Zhang, W., Wang, J., Yu, Y.: Seqgan: sequence generative adversarial nets with policy gradient. In: AAAI-17 (2017)
Zhipeng, G., et al.: Jiuge: a human-machine collaborative chinese classical poetry generation system. In: Proceedings of ACL 2019: System Demonstrations, pp. 25–30 (2019)
Zinkevich, M., Weimer, M., Smola, A.J., Li, L.: Parallelized stochastic gradient descent. In: Proceedings of NIPS 2010 (2011)
Acknowledgements
This work is supported by National Key R & D Program of China Project #2017YFB0203201, Key-Area Research and Development Plan of Guangdong Province 2020B010164003.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
A More Examples of Our Method
A More Examples of Our Method
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Ma, L., Shen, H., Liang, S. (2021). A Novel Distributed Reinforcement Learning Method for Classical Chinese Poetry Generation. In: Zhang, Y., Xu, Y., Tian, H. (eds) Parallel and Distributed Computing, Applications and Technologies. PDCAT 2020. Lecture Notes in Computer Science(), vol 12606. Springer, Cham. https://doi.org/10.1007/978-3-030-69244-5_3
Download citation
DOI: https://doi.org/10.1007/978-3-030-69244-5_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-69243-8
Online ISBN: 978-3-030-69244-5
eBook Packages: Computer ScienceComputer Science (R0)