Abstract
Split-and-rephrase is a strategy known to be used when humans need to break down a complex sentence into a meaning preserving sequence of shorter ones. Recent work proposed to model split-and-rephrase as a supervised sequence generation problem. However, different from other types of sequence generations, the task of split-and-rephrase inevitably introduces overlaps across splits to compensate for the missing context caused by separating a sentence. Serving as the baseline of this task, the vanilla SEQ2SEQ model usually suffers from inappropriate duplication because of the lack of a mechanism to plan how the source sentence should be split into shorter units. This work demonstrates that the problem of inappropriate duplication can be tackled by explicitly modeling the hierarchy within split-and-rephrase: Our model first introduces a separator network capable of selecting semantic components from the source sentence to form a representation for each split. Then, a decoder generates each split on the basis of its representation. Analyses demonstrate that with the aid of the separator, a model can effectively learn attention to avoid duplication and detect clues for splitting a sentence. Experimental results on the WikiSplit corpus show that our model outperforms the non-hierarchical SEQ2SEQ model by 1.4 points in terms of duplication rate and by 0.3 points in terms of coverage rate.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Wikipedia is a registered trademark of the Wikimedia Foundation, Inc.
- 2.
References
Narayan, S., Gardent, C., Cohen, S.B., Shimorina, A.: Split and rephrase. In: Proceedings of EMNLP, pp. 606–616 (2017)
Braud, C., Lacroix, O., Søgaard, A.: Cross-lingual and cross-domain discourse segmentation of entire documents. In: Proceedings of ACL, vol. 2, pp. 237–243 (2017)
Koehn, P., Knowles, R.: Six challenges for neural machine translation. In: Proceedings of the 1st Workshop on NMT, pp. 28–39 (2017)
Pouget-Abadie, J., Bahdanau, D., van Merrienboer, B., Cho, K., Bengio, Y.: Overcoming the curse of sentence length for neural machine translation using automatic segmentation. In: Proceedings of SSST-8, pp. 78–85 (2014)
Chandrasekar, R., Doran, C., Srinivas, B.: Motivations and methods for text simplification. In: Proceedings of COLING, vol. 2, pp. 1041–1044 (1996)
Tomita, M.: Efficient Parsing for Natural Language: A Fast Algorithm for Practical Systems. SECS, vol. 8. Springer, Boston (1986). https://doi.org/10.1007/978-1-4757-1885-0
Chandrasekar, R., Bangalore, S.: Automatic induction of rules for text simplification. Knowl.-Based Syst. 10, 183–190 (1997)
McDonald, R., Nivre, J.: Analyzing and integrating dependency parsers. Comput. Linguist. 37(1), 197–230 (2011)
JelÃnek, T.: Improvements to dependency parsing using automatic simplification of data. In: Proceedings of LREC, pp. 73–77 (2014)
Zhang, Y., Zhong, V., Chen, D., Angeli, G., Manning, C.D.: Position-aware attention and supervised data improve slot filling. In: Proceedings of EMNLP, pp. 35–45 (2017)
Schuster, M., Paliwal, K.K.: Bidirectional recurrent neural networks. IEEE TSP 45(11), 2673–2681 (1997)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. In: Proceedings of ICLR (2015)
Press, O., Wolf, L.: Using the output embedding to improve language models. In: Proceedings of EACL, vol. 2, pp. 157–163 (2017)
Papineni, K., Roukos, S., Ward, T., Zhu, W.-J.: BLEU: a method for automatic evaluation of machine translation. In: Proceedings of ACL, pp. 311–318 (2002)
Botha, J.A., Faruqui, M., Alex, J., Baldridge, J., Das, D.: Learning to split and rephrase from Wikipedia edit history. In: Proceedings of EMNLP, pp. 732–737 (2018)
Aharoni, R., Goldberg, Y.: Split and rephrase: better evaluation and stronger baselines. In: Proceedings of ACL, vol. 2, pp. 719–724 (2018)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: Proceedings of ICLR (2015)
Dyer, C., Chahuneau, V., Smith, N.A.: A simple, fast, and effective reparameterization of IBM model 2. In: Proceedings of NAACL-HLT, pp. 644–648 (2013)
Vu, T., Hu, B., Munkhdalai, T., Yu, H.: Sentence simplification with memory-augmented neural networks. In: Proceedings of NAACL-HLT, pp. 79–85 (2018)
Zuo, S., Xu, Z.: A hierarchical neural network for sequence-to-sequences learning, vol. arXiv preprint arXiv:1811.09575 (2018)
Brown, P.F., Pietra, S.A.D., Pietra, V.J.D., Mercer, R.L., Mohanty, S.: Dividing and conquering long sentences in a translation system. In: Proceedings of the Workshop on Speech and Natural Language, pp. 267–271 (1992)
Li, J., Luong, T., Jurafsky, D.: A hierarchical neural autoencoder for paragraphs and documents. In: Proceedings of ACL and IJCNLP, vol. 1, pp. 1106–1115 (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Wang, M., Ozaki, H., Koreeda, Y., Yanai, K. (2020). Split First and Then Rephrase: Hierarchical Generation for Sentence Simplification. In: Nguyen, LM., Phan, XH., Hasida, K., Tojo, S. (eds) Computational Linguistics. PACLING 2019. Communications in Computer and Information Science, vol 1215. Springer, Singapore. https://doi.org/10.1007/978-981-15-6168-9_2
Download citation
DOI: https://doi.org/10.1007/978-981-15-6168-9_2
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-6167-2
Online ISBN: 978-981-15-6168-9
eBook Packages: Computer ScienceComputer Science (R0)