Abstract
Recent end-to-end encoder-decoder neural models for data-to-text generation can produce fluent and seemingly informative texts despite these models disregard the traditional content selection and surface realization architecture. However, texts generated by such neural models are often missing important facts and contradict the input data, particularly in generation of long texts. To address these issues, we propose a Fact Guided Training (FGT) model to improve both content selection and surface realization by leveraging an information extraction (IE) system. The IE system extracts facts mentioned in reference data and generates texts which provide fact-guided signals. First, a content selection loss is designed to penalize content deviation between generated texts and their references. Moreover, with the selection of proper content for generation, a consistency verification mechanism is designed to inspect fact discrepancy between generated texts and their corresponding input data. The consistency signal is non-differentiable and is optimized via reinforcement learning. Experimental results on a recent challenging dataset ROTOWIRE show our proposed model outperforms neural encoder-decoder models in both automatic and human evaluations.
H. Chen—Equal Contribution, work was done when the first and second author internships at Microsoft.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
A template example, where the players and scores are emitted in the sentence. \(\texttt {<player>}\) scored \(\texttt {<pts>}\) points \((\texttt {<fgm>}\)-\(\texttt {<fga>}\) FG, \(\texttt {<tpm>}\)-\(\texttt {<tpa>}\) 3PT, \(\texttt {<ftm>}\)-\(\texttt {<fta>}\) FT).
- 2.
We do not apply dropout in RL training.
- 3.
Wiseman17 have recently updated the dataset to fix some mistakes. We cannot directly use the results which is reported in their paper and rerun the author’s code.
- 4.
The complete game summary is relatively long, we presents a part of summary for brevity.
References
Angeli, G., Liang, P., Klein, D.: A simple domain-independent probabilistic approach to generation. In: EMNLP (2010)
Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. In: ICLR (2015)
Barzilay, R., Lapata, M.: Collective content selection for concept-to-text generation. In: EMNLP (2005)
Bosselut, A., Çelikyilmaz, A., He, X., Gao, J., Huang, P., Choi, Y.: Discourse-aware neural rewards for coherent text generation. In: NAACL (2018)
Chisholm, A., Radford, W., Hachey, B.: Learning to generate one-sentence biographies from wikidata. CoRR abs/1702.06235 (2017)
Cho, K., et al.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. In: ACL (2014)
Duboué, P.A., McKeown, K.R.: Statistical acquisition of content selection rules for natural language generation. In: EMNLP (2003)
Dušek, O., Jurcicek, F.: Sequence-to-sequence generation for spoken dialogue via deep syntax trees and strings. In: ACL (2016)
Gatt, A., Krahmer, E.: Survey of the state of the art in natural language generation: core tasks, applications and evaluation. J. Artif. Intell. Res. 61, 65–170 (2018)
Gu, J., Lu, Z., Li, H., Li, V.O.K.: Incorporating copying mechanism in sequence-to-sequence learning. In: ACL (2016)
Kim, J., Mooney, R.J.: Generative alignment and semantic parsing for learning from ambiguous supervision. In: COLING (2010)
Kukich, K.: Design of a knowledge-based report generator. In: ACL, pp. 145–150 (1983)
Lebret, R., Grangier, D., Auli, M.: Neural text generation from structured data with application to the biography domain. In: EMNLP, pp. 1203–1213 (2016)
Li, J., Monroe, W., Ritter, A., Jurafsky, D., Galley, M., Gao, J.: Deep reinforcement learning for dialogue generation. In: EMNLP (2016)
Liu, T., Wang, K., Sha, L., Chang, B., Sui, Z.: Table-to-text generation by structure-aware seq2seq learning. In: AAAI (2018)
Mei, H., Bansal, M., Walter, M.R.: What to talk about and how? Selective generation using LSTMs with coarse-to-fine alignment. In: NAACL (2016)
Papineni, K., Roukos, S., Ward, T., Zhu, W.: BLEU: a method for automatic evaluation of machine translation. In: ACL, pp. 311–318 (2002)
Paulus, R., Xiong, C., Socher, R.: A deep reinforced model for abstractive summarization. CoRR abs/1705.04304 (2017)
Reiter, E., Dale, R.: Building applied natural language generation systems. Nat. Lang. Eng. 3, 57–87 (1997)
Ren, Z., Wang, X., Zhang, N., Lv, X., Li, L.: Deep reinforcement learning-based image captioning with embedding reward. In: CVPR (2017)
See, A., Liu, P.J., Manning, C.D.: Get to the point: summarization with pointer-generator networks. In: ACL, pp. 1073–1083. Association for Computational Linguistics, July 2017
Soricut, R., Marcu, D.: Stochastic language generation using WIDL-expressions and its application in machine translation and summarization. In: ACL (2006)
Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. In: NIPS (2014)
Sutton, R.S., Mcallester, D., Singh, S., Mansour, Y.: Policy gradient methods for reinforcement learning with function approximation. In: NIPS, pp. 1057–1063. MIT Press (2000)
Williams, R.J.: Simple statistical gradient-following algorithms for connectionist reinforcement learning. Mach. Learn. 8, 229–256 (1992)
Wiseman, S., Shieber, S.M., Rush, A.M.: Challenges in data-to-document generation. In: EMNLP (2017)
Yang, Z., Blunsom, P., Dyer, C., Ling, W.: Reference-aware language models. In: EMNLP, pp. 1850–1859 (2017)
Zaremba, W., Sutskever, I.: Reinforcement learning neural turing machines. CoRR abs/1505.00521 (2015)
Zhang, X., Lapata, M.: Sentence simplification with deep reinforcement learning. In: EMNLP (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Nie, F., Chen, H., Wang, J., Pan, R., Lin, CY. (2019). Beyond Word for Word: Fact Guided Training for Neural Data-to-Document Generation. In: Tang, J., Kan, MY., Zhao, D., Li, S., Zan, H. (eds) Natural Language Processing and Chinese Computing. NLPCC 2019. Lecture Notes in Computer Science(), vol 11838. Springer, Cham. https://doi.org/10.1007/978-3-030-32233-5_41
Download citation
DOI: https://doi.org/10.1007/978-3-030-32233-5_41
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-32232-8
Online ISBN: 978-3-030-32233-5
eBook Packages: Computer ScienceComputer Science (R0)
