Skip to main content
SpringerLink
Log in

Ancient poetry generation with an unsupervised method

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

It is challenging to use unsupervised machine translation models to generate ancient poems. The current method has solved the problems of Under-translation and Over-translation caused by the huge length difference between the translated sentence pairs. However, the above method lacks guidance in generating intermediate vectors, and the denoising ability of the model is very poor. In this paper, we guide vector space distribution during training to improve the quality of the generated ancient poems and the convergence speed of the model. We also introduce the target language information while adding noise, which effectively avoids the recurrence of the Under-translation problem while improving the model's denoising ability. Experiment results on the VP dataset show that our model obtains state-of-the-art results with faster convergence speed. In addition to the BLEU scores, we also made a comparative analysis of ancient poetry sentences generated by different models. The analysis results show that the optimization method proposed in this paper is indeed helpful for generating high-quality ancient poems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Yi X, Sun M, Li R, Yang Z (2018) Chinese Poetry Generation with a Working Memory Model. In: Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, 4553–4559

  2. Lample G, Denoyer L, Ranzato MA (2018) Unsupervised Machine Translation Using Monolingual Corpora Only. arXiv:1711.00043

  3. Yang Z, Cai P, Feng Y, Li F, Feng W, Chiu EY, Yu H (PMC7255431) Generating Classical Chinese Poems from Vernacular Chinese. In: Proceedings of the conference on empirical methods in natural language processing. Conference on empirical methods in natural language processing, Nov 2019. pp 6155–6164. https://doi.org/10.18653/v1/d19-1637

  4. Artetxe M, Labaka G, Agirre E, Cho K (2018) Unsupervised Neural Machine Translation. arXiv:1710.11041

  5. Och FJ, Ney H (2003) A systematic comparison of various statistical alignment models. Comput Linguist 29(1):19–51

    Article  Google Scholar 

  6. Bahdanau D, Cho K, Bengio Y (2014) Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473

  7. Ji Y, Zhang H, Zhang Z, Liu M (2021) CNN-based encoder-decoder networks for salient object detection: A comprehensive review and recent advances. Information Sciences 546:835–857 %@ 0020–0255

  8. Cho K, Van Merriënboer B, Gulcehre C, Bahdanau D, Bougares F, Schwenk H, Bengio Y (2014) Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078

  9. Kalchbrenner N, Blunsom P (2013) Recurrent continuous translation models. In: EMNLP. pp 1700–1709

  10. Luong M-T, Sutskever I, Le QV, Vinyals O, Zaremba W (2014) Addressing the rare word problem in neural machine translation. arXiv preprint arXiv:1410.8206

  11. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I (2017) Attention is All you Need. In: NIPS. pp 5998--6008

  12. Devlin J, Chang M-W, Lee K, Toutanova K (2018) Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805

  13. Mehta S, Ghazvininejad M, Iyer S, Zettlemoyer L, Hajishirzi H (2020) DeLighT: Very Deep and Light-weight Transformer. arXiv preprint arXiv:2008.00623

  14. Zhu J, Xia Y, Wu L, He D, Qin T, Zhou W, Li H, Liu T-Y (2020) Incorporating bert into neural machine translation. arXiv preprint arXiv:2002.06823

  15. Zhang Z, Ren S, Liu S, Wang J, Chen P, Li M, Zhou M, Chen E (2018) Style Transfer as Unsupervised Machine Translation. arXiv:1808.07894

  16. Hämäläinen M, Hengchen S (2019) From the Paft to the Fiiture: a Fully Automatic NMT and Word Embeddings Method for OCR Post-Correction. In: RANLP

  17. Weng W-H, Chung Y-A, Szolovits P (2019) Unsupervised clinical language translation. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

  18. Conneau A, Lample G, Ranzato MA, Denoyer L, Jégou H (2018) Word Translation Without Parallel Data. arXiv:1710.04087

  19. Artetxe M, Labaka G, Agirre E (2018) A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddings. In: ACL. pp. 789–798

  20. Wan Y, Yang B, Wong DF, Chao LS, Du H (2020) Ao BCH Unsupervised Neural Dialect Translation with Commonality and Diversity Modeling. In: AAAI

  21. He J, Zhou M, Jiang L (2012) Generating Chinese classical poems with statistical machine translation models. In: AAAI, 1650–1656

  22. Yi X, Li R, Sun M (2017) Generating Chinese classical poems with RNN Encoder-Decoder. arXiv:1604.01537

  23. Zhang J, Feng Y, Wang D, Wang Y, Abel A, Zhang S, Zhang A (2017) Flexible and Creative Chinese Poetry Generation Using Neural Memory. arXiv:1705.03773

  24. Yi X, Sun M, Li R, Li W (2018) Automatic Poetry Generation with Mutual Reinforcement Learning. In: EMNLP. 3143–3153

  25. Yang X, Lin X, Suo S, Li M (2018) Generating Thematic Chinese Poetry using Conditional Variational Autoencoders with Hybrid Decoders. In: IJCAI, 4539–4545

  26. Vincent P, Larochelle H, Bengio Y, Manzagol P-A (2008) Extracting and composing robust features with denoising autoencoders. In: Proceedings of the 25th international conference on Machine learning. 1096–1103

  27. Hill F, Cho K, Korhonen (2016) A learning distributed representations of sentences from Unlabelled Data. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego, California. 1367–1377

  28. Sennrich R, Haddow B, Birch A (2016) Improving Neural Machine Translation Models with Monolingual Data. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, Berlin, Germany. 86–96

  29. Chang P-C, Galley M, Manning CD (2008) Optimizing Chinese Word Segmentation for Machine Translation Performance. In: Proceedings of the Third Workshop on Statistical Machine Translation. 224–232

  30. Rennie SJ, Marcheret E, Mroueh Y, Ross J, Goel V (2017) Self-Critical Sequence Training for Image Captioning. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR):1179–1195

  31. Wu Y, Schuster M, Chen Z, Le QV, Norouzi M, Macherey W, Krikun M, Cao Y, Gao Q, Macherey K, Klingner J, Shah A, Johnson M, Liu X, Kaiser L, Gouws S, Kato Y, Kudo T, Kazawa H, Stevens K, Kurian G, Patil N, Wang W, Young C, Smith J, Riesa J, Rudnick A, Vinyals O, Corrado GS, Hughes M, Dean J (2016) Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation. arXiv:1609.08144

  32. Yang Z, Chen W, Wang F, Xu B (2018) Unsupervised Neural Machine Translation with Weight Sharing. In: ACL. 46–55

  33. Kingma DP, Ba J (2015) Adam: A Method for Stochastic Optimization. CoRR abs/1412.6980

  34. Papineni K, Roukos S, Ward T, Zhu W-J (2002) Bleu: a Method for Automatic Evaluation of Machine Translation. In: ACL. 311–318

Download references

Acknowledgements

The authors wish to thank the reviewers for their helpful comments. This work is supported by The National Key Research and Development Program of China (2018YFB0204301).

Author information

Authors and Affiliations

  1. College of Computer, National University of Defense Technology, Changsha, China

    Zhanjun Zhang, Haoyu Zhang, Qian Wan, Xiangyu Jia & Zhe Zhang

  2. Science and Technology on Parallel and Distributed Processing Laboratory, National University of Defense Technology, Changsha, China

    Jie Liu

  3. Laboratory of Software Engineering for Complex Systems, National University of Defense Technology, Changsha, China

    Jie Liu

Authors
  1. Zhanjun Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Haoyu Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qian Wan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiangyu Jia
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zhe Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jie Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jie Liu.

Ethics declarations

Conflict of interest

We declare that we have no financial and personal relationships with other people or organizations that can inappropriately influence our work, there is no professional or other personal interest of any nature or kind in any product, service and/or company that could be construed as influencing the position presented in, or the review of, the manuscript entitled “Ancient Poetry Generation with an Unsupervised Method.”

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix: Examples of Generated Poems

Appendix: Examples of Generated Poems

Table 10 is some examples of the achievements generated by our model. The content in the table proves that the ancient poems generated in this paper are very close to the professional level.

Table 10 Some examples of ancient poems generated by our model
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, Z., Zhang, H., Wan, Q. et al. Ancient poetry generation with an unsupervised method. Neural Comput & Applic 34, 8525–8538 (2022). https://doi.org/10.1007/s00521-021-06571-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-021-06571-w

Keywords

Search

Navigation