Zekai He
ABSTRACT
From prehistoric times to the present, the creation of poetry has long been considered the exclusive domain of humans. With the development of deep learning, many researchers have begun to address the challenge of how to generate poetry using algorithms. To capture more contextual continuity and semantically related information in Chinese poetry, this paper applies the BERT (Bidirectional Encoder Representations from Transformers) model with improvement in the full Tang dynasty poem dataset. In addition, this model is also used for inference to generate acrostic poetry and sequel poetry. Under the automatic evaluation metric BLEURT algorithm, Tang Dynasty poetry generated by the model used outperforms those generated by Long Short-Term Memory model. Good poetry generated from the model used was also approved by Chinese poets. This paper suggests that the BERT model can generate higher quality and more various forms of poetry, which is of some certain reference and application value in the field of poetry.
- John, B., Ryan, M., & Fernando, P. (2006) Domain adaptation with structural correspondence learning, Empirical Methods in Natural Language Processing, W06-16.: 120-128.Google Scholar
- Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S., & Dean, J. (2013). Distributed representations of words and phrases and their compositionality. Advances in neural information processing systems, 26.Google Scholar
- Pennington, J., Socher, R., & Manning, C. D. (2014, October). Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP) (pp. 1532-1543).Google ScholarCross Ref
- Dai, A. M., & Le, Q. V. (2015). Semi-supervised sequence learning. Advances in neural information processing systems, 28.Google Scholar
- Kiros, R., Zhu, Y., Salakhutdinov, R. R., Zemel, R., Urtasun, R., Torralba, A., & Fidler, S. (2015). Skip-thought vectors. Advances in neural information processing systems, 28.Google Scholar
- Parikh, A. P., Täckström, O., Das, D., & Uszkoreit, J. (2016). A decomposable attention model for natural language inference. arXiv preprint arXiv:1606.01933.Google Scholar
- Peters, M. E., Ammar, W., Bhagavatula, C., & Power, R. (2017). Semi-supervised sequence tagging with bidirectional language models. arXiv preprint arXiv:1705.00108.Google Scholar
- Conneau, A., Kiela, D., Schwenk, H., Barrault, L., & Bordes, A. (2017). Supervised learning of universal sentence representations from natural language inference data. arXiv preprint arXiv:1705.02364.Google Scholar
- Matthew E., P., Mark, N., Mohit, I., Matt, G., Christopher, C., Kenton, L., & Luke S., Z. (2018) Deep contextualized word representations., Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing, abs/1802.05365.Google Scholar
- Peters, M. E., Neumann, M., Zettlemoyer, L., & Yih, W. T. (2018). Dissecting contextual word embeddings: Architecture and representation. arXiv preprint arXiv:1808.08949.Google Scholar
- Howard, J., & Ruder, S. (2018). Universal language model fine-tuning for text classification. arXiv preprint arXiv:1801.06146.Google Scholar
- Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). Minneapolis, Minnesota, 4171–4186.Google Scholar
- Xiaofeng Wu, Naoko Tosa, and Ryohei Nakatsu. 2009. New Hitch Haiku: An Interactive Renku Poem Composition Supporting Tool Applied for Sightseeing Navigation System. In Proceedings of the 8th International Conference on Entertainment Computing (ICEC '09). Springer-Verlag, Berlin, Heidelberg, 191–196. https://doi.org/10.1007/978-3-642-04052-8_19Google ScholarDigital Library
- Oliveira, H. 2009. Automatic generation of poetry: an overview. Universidade de Coimbra.Google Scholar
- Rui Yan, Han Jiang, Mirella Lapata, Shou-De Lin, Xueqiang Lv, and Xiaoming Li. 2013. I, poet: automatic Chinese poetry composition through a generative summarization framework under constrained optimization. In Proceedings of the Twenty-Third international joint conference on Artificial Intelligence (IJCAI '13). AAAI Press, 2197–2203.Google Scholar
- Long Jiang and Ming Zhou. 2008. Generating Chinese couplets using a statistical MT approach. In Proceedings of the 22nd International Conference on Computational Linguistics - Volume 1 (COLING '08). Association for Computational Linguistics, USA, 377–384.Google ScholarCross Ref
- Jing He, Ming Zhou, and Long Jiang. 2012. Generating chinese classical poems with statistical machine translation models. In Proceedings of the Twenty-Sixth AAAI Conference on Artificial Intelligence (AAAI'12). AAAI Press, 1650–1656.Google Scholar
- Xingxing Zhang and Mirella Lapata. 2014. Chinese Poetry Generation with Recurrent Neural Networks. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, Doha, Qatar, 670–680.Google ScholarCross Ref
- Qixin Wang, Tianyi Luo, and Dong Wang. 2016. Can Machine Generate Traditional Chinese Poetry? A Feigenbaum Test. International Conference on Brain Inspired Cognitive Systems, vol 10023. Springer, Cham, 34-46. https://doi.org/10.1007/978-3-319-49685-6_4Google ScholarCross Ref
- Marjan Ghazvininejad, Xing Shi, Yejin Choi, and Kevin Knight. 2016. Generating topical poetry. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, Austin, Texas, 1183-1191.Google ScholarCross Ref
- Xiaoyuan Yi, Ruoyu Li, and Maosong Sun. 2017. Generating Chinese Classical Poems with RNN Encoder-Decoder. In Chinese Computational Linguistics and Natural Language Processing Based on Naturally Annotated Big Data, vol 10565. Springer, Cham, 211–223.Google Scholar
- Hochreiter S., and Schmidhuber J. (1997). Long short-term memory. Neural computation, 9(8), 1735-1780.Google ScholarDigital Library
- Wancheng Wei, Wenming Huang, Jing Wang, and Zhenrong Deng. 2019. Chinese Classical Poetry and Couplet Generation Based on Multi-task Learning. Journal of Chinese Information Processing, 33(9): 107-114,140.Google Scholar
- Kyunghyun Cho, Bart Merrienboer, Caglar Gulcehre, Fethi Bougares, Holger Schwenk, and Y. Bengio. 2014. Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, Doha, Qatar (2014), 1724–1734. https://doi.org/10.3115/v1/D14-1179.Google ScholarCross Ref
- Jing Lyu, Lili Chu, and Ruixue Gong. 2022. Research on Automatic Generation System of Ancient Poetry Based on Improved LSTM Model. China Computer & Communication, 34(01): 50-54.Google Scholar
- Donghua Zhao. 2013. Modern Information Technology and Appreciation of Ancient Poems. Language Planning, (34): 79-80.Google Scholar
- Jianli Zhao and Jong Lee Hyo. 2022. Automatic Generation and Evaluation of Chinese Classical Poetry with Attention-Based Deep Neural Network. Applied Sciences. 2022, 12(13): 6497. https://doi.org/10.3390/app12136497Google ScholarCross Ref
- Ashish, V., Noam, S., Niki, P., Jakob, U., Llion, J., Aidan N., G., Lukasz, K., & Illia, P. (2017) Attention Is All You Need., ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 30 (NIPS 2017), 30.: 5998-6008.Google Scholar
- Sellam, T., Das, D., & Parikh, A. P. (2020). BLEURT: Learning robust metrics for text generation. arXiv preprint arXiv:2004.04696.Google Scholar
Index Terms
- Generation of Chinese Tang Dynasty Poetry Based on BERT Model
Recommendations
TPoet: Topic-Enhanced Chinese Poetry Generation
Chinese poetry generation has been a challenging part of natural language processing due to the unique literariness and aesthetics of poetry. In most cases, the content of poetry is topic related. In other words, specific thoughts or emotions are usually ...
Chinese Poetry Generation with Metrical Constraints
Natural Language Processing and Chinese ComputingAbstractPoetry is a kind of literary art, which conveys emotion with aesthetic expressions. Poetry automatic generation is challenging because it is required to confirm the semantic representation (content) and metrical constraints (form). Most previous ...
A Novel Distributed Reinforcement Learning Method for Classical Chinese Poetry Generation
Parallel and Distributed Computing, Applications and TechnologiesAbstractPoetry generation has been a classic natural language generation task recently. But so far the methods for this topic mainly imitate and reproduce the poems on the training data set, which indicates that they either have not much connotation or ...
Comments