Xianjie Shen
ABSTRACT
Unsupervised style transfer aims to transfer the intrinsic style of text while preserving its content without parallel datasets. Many sophisticated methods using reinforcement learning and neural networks have been developed to address this problem, however, their performance is not very ideal yet. We observe that given massive unpaired texts, there would exist high-quality sentence pairs that have similar style-independent content but different style words. Inspiring by this observation, in this paper, we propose a simple yet effective method without any neural network. Specifically, we consider both embedding similarity and BLEU score to locate similar sentences of different styles for a pseudo-parallel dataset construction. From this pseudo-parallel dataset, we distill the style words and align them into pairs based on statistical signals. We further refine our pseudo-parallel dataset by ignoring the identified style words during similarity calculation. After the style word pairs converged, we put them together as a lookup table to recognize and replace style words for style transfer. Extensive experiments demonstrate that our method is effective in different style transferring settings, such as sentiment and formality, outperforming state-of-the-art methods.
- M. ARTETXE, G. LABAKA, E. AGIRRE and K. CHO. 2017. Unsupervised neural machine translation. CoRR, abs/1710.11041. https://doi.org/10.48550/arXiv.1710.11041.Google Scholar
- Z. FU, X. TAN, N. PENG, D. ZHAO and R. YAN. 2017. Style transfer in text: Exploration and evaluation. In Proceedings of the AAAI Conference on Artificial Intelligence. AIII, New Orleans, USA, Vol.32. https://doi.org/10.1609/aaai.v32i1.11330.Google Scholar
- L. HAN, T. FININ, P. MCNAMEE, A. JOSHI and Y. YESHA. 2013. Improving word similarity by augmenting pmi with estimates of word polysemy. IEEE Transactions on Knowledge and Data Engineering, 25, pp. 1307–1322. https://doi.org/10.1109/TKDE.2012.30.Google ScholarDigital Library
- J. HE, X. WANG, G. NEUBIG and T. BERG-KIRKPATRICK. 2020. A probabilistic formulation of unsupervised text style transfer. https://doi.org/10.48550/arXiv.2002.03912.Google Scholar
- Z. HU, Z. YANG, X. LIANG, R. SALAKHUTDINOV and E. P. XING. 2017. Controllable text generation. CoRR, abs/1703.00955. https://doi.org/10.48550/arXiv.1703.00955.Google Scholar
- V. JOHN, L. MOU, H. BAHULEYAN and O. VECHTOMOVA. 2019. Disentangled representation learning for non-parallel text style transfer, In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy, Association for Computational Linguistics, pp. 424–434. https://doi.org/10.18653/v1/P19-1041.Google ScholarCross Ref
- G. LAMPLE, M. OTT, A. CONNEAU, L. DENOYER and M. RANZATO. 2018. Phrase-based & neural unsupervised machine translation, CoRR, abs/1804.07755,pp.5039-5049. https://doi.org/10.48550/arXiv.1804.07755.Google Scholar
- J. H. LAU and T. BALDWIN. 2016. An empirical evaluation of doc2vec with practical insights into document embedding generation. In Proceedings of the 1st Workshop on Representation Learning for NLP, Rep4NLP@ACL 2016, Berlin, Germany, pp. 78-86. https://doi.org/10.18653/v1/W16-1609.Google ScholarCross Ref
- J. LI, R. JIA, H. HE and P. LIANG. 2018. Delete, retrieve, generate: a simple approach to sentiment and style transfer. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Vol.1, New Orleans, Louisiana, Association for Computational Linguistics, pp. 1865–1874 https://doi.org/10.18653/v1/N18-1169.Google ScholarCross Ref
- D. LIU, J. FU, Y. ZHANG, C. PAL and J. LV. 2019. Revision in continuous space: Fine-grained control of text style transfer. CoRR, abs/1905.12304. https://doi.org/10.48550/arXiv.1905.12304.Google Scholar
- F. LUO, P. LI, J. ZHOU, P. YANG, B. CHANG, Z. SUI and X. SUN. 2019. A dual reinforcement learning framework for unsupervised text style transfer. In Proceedings of the IJCAI 2019, pp.5116-5122. abs/1905.10060. https://doi.org/10.48550/arXiv.1905.10060.Google ScholarCross Ref
- I. MELNYK, C. N. DOS SANTOS, K. WADHAWAN, I. PADHI and A. KUMAR. 2017. Improved neural text attribute transfer with non-parallel data. In Proceedings of the NIPS 2017 Workshop on Learning Disentangled Representations: from Perception to Control. abs/1711.09395. https://doi.org/10.48550/arXiv.1711.09395.Google Scholar
- K. PAPINENI, S. ROUKOS, T. WARD and W. JING ZHU. 2002. Bleu: a method for automatic evaluation of machine translation, In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics (ACL '02). Association for Computational Linguistics, USA, pp.311–318. https://doi.org/10.3115/1073083.1073135.Google ScholarDigital Library
- S. PRABHUMOYE, Y. TSVETKOV, R. SALAKHUTDINOV and A. W. BLACK. 2018. Style transfer through backtranslation. In Proceedings of ACL 2018., abs/1804.09000. https://doi.org/10.48550/arXiv.1804.09000.Google Scholar
- S. RAO and J. R. TETREAULT. 2018. Dear sir or madam, may I introduce the YAFC corpus: Corpus, benchmarks and metrics for formality style transfer. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. abs/1803.06535. https://doi.org/10.48550/arXiv.1803.06535.Google Scholar
- T. SHEN, T. LEI, R. BARZILAY and T. S. JAAKKOLA. 2017. Style transfer from non-parallel text by cross-alignment. In Proceedings of NIPS 2017. abs/1705.09655. https://doi.org/10.48550/arXiv.1705.09655.Google Scholar
- S. SUBRAMANIAN, G. LAMPLE, E. M. SMITH, L. DENOYER, M. RANZATO and Y. BOUREAU. 2018. Multiple-attribute text style transfer. CoRR, abs/1811.00552. https://doi.org/10.48550/arXiv.1811.00552.Google Scholar
- J. XU, X. SUN, Q. ZENG, X. REN, X. ZHANG, H. WANG and W. LI. 2018. Unpaired sentiment-tosentiment translation: A cycled reinforcement learning approach. In Proceedings of ACL 2018. abs/1805.05181 . https://doi.org/10.48550/arXiv.1805. 05181.Google Scholar
Index Terms
- Unsupervised Sentiment and Style Transfer from Massive Texts
Recommendations
MSSRNet: Manipulating Sequential Style Representation for Unsupervised Text Style Transfer
KDD '23: Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data MiningUnsupervised text style transfer task aims to rewrite a text into target style while preserving its main content. Traditional methods rely on the use of a fixed-sized vector to regulate text style, which is difficult to accurately convey the style ...
Token-level disentanglement for unsupervised text style transfer
AbstractText style transfer is the task of altering the style of a source text to a desired style while preserving the style-independent content. A common approach involves disentangling a given sentence into a style-agnostic content representation ...
Unsupervised Joint PoS Tagging and Stemming for Agglutinative Languages
The number of possible word forms is theoretically infinite in agglutinative languages. This brings up the out-of-vocabulary (OOV) issue for part-of-speech (PoS) tagging in agglutinative languages. Since inflectional morphology does not change the PoS ...
Comments