Skip to main content
SpringerLink
Log in

Text style transfer between classical and modern chinese through prompt-based reinforcement learning

  • Published:
World Wide Web Aims and scope Submit manuscript

Abstract

Text style transfer aims at converting the stylistic features of a sentence to another style while preserving its content. Despite the remarkable progress achieved in English style transfer, Chinese style transfer still relies heavily on manual processing. Taking classical and modern Chinese style transfer as an example, most of the existing method cannot carry out this task due to the lack of sufficient parallel corpus for supervised learning and the special language phenomenon in Chinese. In this paper, we propose an unsupervised prompt-based reinforcement learning (PBRL) framework to transfer text between classical and modern Chinese styles via an entangled approach. The PBRL framework mainly consists of two stages, i.e., a prompt-based fine-tuning stage and a bi-directional reinforcement learning stage. In the first stage, we leverage a priori knowledge-based synonym dictionary to build a pseudo-parallel corpus for prompt learning to provide the system a warm start. Then the style-transfer-accuracy reward and content-preservation reward are specially designed for bi-directional-reinforcement optimization. Experimental evaluations show that our model outperforms state-of-art networks by a large margin.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1

Similar content being viewed by others

Code Availability

Not applicable

Notes

  1. https://huggingface.co/google/mt5-base

  2. https://github.com/bojone/t5_in_bert4keras

References

  1. Klahold, A., Fathi, M.: Computer aided writing. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-27439-9

  2. Kim, D., Kim, S.: Newspaper companies’ determinants in adopting robot journalism. Technol. Forecast. Soc. Chang. 117, 184–195 (2017)

    Article  Google Scholar 

  3. Gong, H., Bhat, S., Wu, L., Xiong, J., Hwu, W.-m.: Reinforcement learning based text style transfer without parallel training corpus. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 3168–3180. Association for Computational Linguistics (2019)

  4. Kabbara, J., Cheung, J.C.K.: Stylistic transfer in natural language generation systems using recurrent neural networks. In: Proceedings of the Workshop on Uphill Battles in Language Processing: Scaling Early Achievements to Robust Methods, pp. 43–47 (2016)

  5. Zhou, L., Gao, J., Li, D., Shum, H. -Y.: The design and implementation of xiaoice, an empathetic social chatbot. Computational Linguistics 46(1), 53–93 (2020)

    Article  Google Scholar 

  6. Gao, W., Peng, M., Wang, H., Zhang, Y., Xie, Q., Tian, G.: Incorporating word embeddings into topic modeling of short text. Knowl. Inf. Syst. 61(2), 1123–1145 (2019)

    Article  Google Scholar 

  7. Huang, J., Peng, M., Wang, H., Cao, J., Gao, W., Zhang, X.: A probabilistic method for emerging topic tracking in microblog stream. World Wide Web 20(2), 325–350 (2017)

    Article  Google Scholar 

  8. Jiang, H., Zhou, R., Zhang, L., Wang, H., Zhang, Y.: Sentence level topic models for associated topics extraction. World Wide Web 22(6), 2545–2560 (2019)

    Article  Google Scholar 

  9. Jin, D., Jin, Z., Hu, Z., Vechtomova, O., Mihalcea, R.: Deep learning for text attribute transfer: A survey. arXiv:2011.00416 (2021)

  10. Luo, F., Li, P., Zhou, J., Yang, P., Chang, B., Sui, Z., Sun, X.: A dual reinforcement learning framework for unsupervised text style transfer. IJCAI, 5116–5122 (2019)

  11. Riley, P., Constant, N., Guo, M., Kumar, G., Uthus, D.C., Parekh, Z.: Textsettr: few-shot text style extraction and tunable targeted restyling. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 3786–3800 (2021)

  12. Yi, X., Liu, Z., Li, W., Sun, M.: Text style transfer via learning style instance supported latent space. IJCAI 2020, 3801–3807 (2020)

  13. Abhilasha, S., Kundan, K., Vasan, B.S., Anandhavelu, N.: Reinforced rewards framework for text style transfer. European Conference on Information Retrieval, pp. 545–560 (2020)

  14. Rao, S., Tetreault, J.: Dear sir or madam, may i introduce the gyafc dataset: Corpus, benchmarks and metrics for formality style transfer. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pp. 129–140. Association for Computational Linguistics. https://doi.org/10.18653/v1/N18-1012 (2018)

  15. Carlson, K., Riddell, A., Rockmore, D.: Evaluating prose style transfer with the bible. Royal Society Open Science 5(10), 171920 (2018)

    Article  Google Scholar 

  16. Jhamtani, H., Gangal, V., Hovy, E., Nyberg, E.: Shakespearizing modern language using copy-enriched sequence to sequence models. In: Proceedings of the Workshop on Stylistic Variation, Sep 2017, pp 10–19. Association for Computational Linguistics. https://doi.org/10.18653/v1/W17-4902 (2017)

  17. Xu, W., Ritter, A., Dolan, W.B., Grishman, R., Cherry, C.: Paraphrasing for style. In: Proceedings of COLING 2012, pp. 2899–2914 (2012)

  18. Zhu, Q., Zhang, W., Liu, T., Wang, W.Y.: Neural stylistic response generation with disentangled latent variables. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 4391–4401 (2021)

  19. Laugier, L., Pavlopoulos, J., Sorensen, J., Dixon, L.: Civil rephrases of toxic texts with self-supervised transformers. In: Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pp. 1442–1461 (2021)

  20. Hu, Z., Lee, R.K.-W., Aggarwal, C.C., Zhang, A.: Text style transfer: A review and experimental evaluation. arXiv:2010.12742 (2020)

  21. Sudhakar, A., Upadhyay, B., Maheswaran, A.: ”transforming” delete, retrieve, generate approach for controlled text style transfer. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 3269–3279 (2019)

  22. Dai, N., Liang, J., Qiu, X., Huang, X.: Style transformer: Unpaired text style transfer without disentangled latent representation. ACL, 5997–6007 (2019)

  23. Li, X., Chen, G., Lin, C., Li, R.: Dgst: a dual-generator network for text style transfer. Empirical Methods in Natural Language Processing, 7131–7136 (2020)

  24. Xu, P., Cheung, J.C.K., Cao, Y.: On variational learning of controllable representations for text without supervision. In: Proceedings of the 37th International Conference on Machine Learning, vol. 119, pp. 10534–10543 (2020)

  25. Xu, M., Peng, M., Liu, F.: A dual reinforcement network for classical and modern chinese text style transfer. In: International Conference on Web Information Systems Engineering, pp. 306–320 (2021). Springer

  26. Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., Liu, P.J.: Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res. 21(140), 1–67 (2020)

    MathSciNet  MATH  Google Scholar 

  27. Xue, L., Constant, N., Roberts, A., Kale, M., Al-Rfou, R., Siddhant, A., Barua, A., Raffel, C.: mt5: A massively multilingual pre-trained text-to-text transformer. arXiv:2010.11934 (2020)

  28. Li, J., Jia, R., He, H., Liang, P.: Delete, retrieve, generate: a simple approach to sentiment and style transfer. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pp. 1865–1874. Association for Computational Linguistics (2018)

  29. Lample, G., Subramanian, S., Smith, E.M., Denoyer, L., Ranzato, M., Boureau, Y. -L.: Multiple-attribute text rewriting. In: 7th International Conference on Learning Representations, {ICLR} 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net. https://openreview.net/forum?id=H1g2NhC5KQ (2019)

  30. Jin, Z., Jin, D., Mueller, J., Matthews, N., Santus, E.: Imat: Unsupervised text attribute transfer via iterative matching and translation. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 3088–3100 (2019)

  31. Zhang, Z., Ren, S., Liu, S., Wang, J., Chen, P., Li, M., Zhou, M., Chen, E.: Style transfer as unsupervised machine translation. Corr abs/1808.07894 (2018)

  32. Shen, T., Lei, T., Barzilay, R., Jaakkola, T.: Style transfer from non-parallel text by cross-alignment. In: Advances in Neural Information Processing Systems, vol. 30, pp 6830–6841. Curran Associates Inc. (2017)

  33. Prabhumoye, S., Tsvetkov, Y., Salakhutdinov, R., Black, A.W.: Style transfer through back-translation. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 866–876. Association for Computational Linguistics (2018)

  34. Sutton, R.S., Barto, A.G.: Reinforcement learning: An introduction neural information processing systems. IEEE Trans Neural Networks 9(5), 1054 (1998). https://doi.org/10.1109/TNN.1998.712192

  35. Wang, L, et al.: The Dictionary of Common Characters in Classical Chinese. The Commercial Press (2016)

  36. Kusner, M.J., Sun, Y., Kolkin, N.I., Weinberger, K.Q.: From word embeddings to document distances. In: Bach, F.R., Blei, D.M. (eds.) Proceedings of the 32nd International Conference on Machine Learning (ICML), 2015, Lille, France, 6-11 July 2015, JMLR Workshop and Conference Proceedings, vol. 37, pp. 957–966. JMLR.org (2015)

  37. Wiseman, S., Rush, M.A.: Sequence-to-sequence learning as beam-search optimization. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing(EMNLP), pp. 1296–1306 (2016)

  38. Papineni, K., Roukos, S., Ward, T., Zhu, W-j: Bleu: a method for automatic evaluation of machine translation. In: ACL ’02 Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, pp. 311–318 (2002)

  39. Turc, I., Chang, M. -W., Lee, K., Toutanova, K.: Well-read students learn better: On the importance of pre-training compact models. arXiv:1908.08962 (2019)

  40. Ranzato, M., Chopra, S., Auli, M., Zaremba, W.: Sequence level training with recurrent neural networks international conference on learning representations. In: Bengio, Y., LeCun Y. (eds.) Sequence Level Training with Recurrent Neural Networks, 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings (2016)

  41. Lin, C. -Y.: Rouge: A package for automatic evaluation of summaries. In: Text Summarization Branches Out, pp. 74–81 (2004)

  42. Mindspore. https://www.mindspore.cn/ (2020). Accessed 2021

Download references

Acknowledgements

This paper was supported by the 2030 National Key AI Program of China (Grant No.2021ZD0113304), General Program of Natural Science Foundation of China (NSFC) (Grant No.62072346), Key R&D Project of Hubei Province (Grant NO.2020BAA021, NO.2021BBA099), Application Foundation Frontier Project of Wuhan (Grant NO.2020010601012168), and MindSpore.

Funding

Not applicable

Author information

Authors and Affiliations

  1. School of Computer Science, Wuhan University, Wuhan, 430064, Hubei, China

    Minzhang Xu, Min Peng & Fang Liu

  2. Department of Information Technology, Wuhan City College, Wuhan, 430075, Hubei, China

    Fang Liu

Authors
  1. Minzhang Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Min Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fang Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Min Peng.

Ethics declarations

The article is original, has been written by the stated authors who are all aware of its content and approve its submission, has not been published previously, it is not under consideration for publication elsewhere, no conflict of interest exists, if accepted, the article will not be published elsewhere in the same form, in any language, without the written consent of the publisher.

Additional information

This article belongs to the Topical Collection: Special Issue on Web Information Systems Engineering 2021 Guest Editors: Hua Wang, Wenjie Zhang, Lei Zou, and Zakaria Maamar

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xu, M., Peng, M. & Liu, F. Text style transfer between classical and modern chinese through prompt-based reinforcement learning. World Wide Web 26, 733–750 (2023). https://doi.org/10.1007/s11280-022-01083-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11280-022-01083-6

Keywords

Search

Navigation