Abstract
Generating a coherent story is a challenging task in natural language processing, which requires maintaining the coherence of plots and inter-sentence semantics throughout the generated text. While existing generation models can generate texts with good intra-sentence coherence, it is still difficult to plan a coherent plot and inter-sentence semantics throughout the text. In this paper, we propose a novel multi-granularity constraints text generation model, which constrains at the token level and sentence level, respectively. For the plot incoherence issue, the token-level constraint is added, which is a new plot guidance method to maintain the coherence of the plot while avoiding the introduction of extra exposure bias. For the problem of semantic incoherence, an auxiliary task of modelling the semantic relations between sentences is designed on the sentence-level constraint. Extensive experiments have shown that our model can generate more coherent stories than the baselines.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
SentenceBERT calculates the similarity based on the cosine between the sentence embedding vectors, and we normalize the result to [0, 1].
References
Fan, A., Lewis, M., Dauphin, Y.: Hierarchical neural story generation. arXiv preprint arXiv:1805.04833 (2018)
Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al.: Language models are unsupervised multitask learners. OpenAI Blog 1(8), 9 (2019)
Li, J., Luong, M.T., Jurafsky, D.: A hierarchical neural autoencoder for paragraphs and documents. arXiv preprint arXiv:1506.01057 (2015)
Shen, D., et al.: Towards generating long and coherent text with multi-level latent variable models. arXiv preprint arXiv:1902.00154 (2019)
Guan, J., Huang, F., Zhao, Z., Zhu, X., Huang, M.: A knowledge-enhanced pretraining model for commonsense story generation. Trans. Assoc. Comput. Linguist. 8, 93–108 (2020)
Xu, P., et al.: MEGATRON-CNTRL: controllable story generation with external knowledge using large-scale language models. arXiv preprint arXiv:2010.00840 (2020)
Yao, L., Peng, N., Weischedel, R., Knight, K., Zhao, D., Yan, R.: Plan-and-write: towards better automatic storytelling. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 7378–7385 (2019)
Tan, B., Yang, Z., AI-Shedivat, M., Xing, E.P., Hu, Z.: Progressive generation of long text with pretrained language models. arXiv preprint arXiv:2006.15720 (2020)
Ribeiro, M.T., Wu, T., Guestrin, C., Singh, S.: Beyond accuracy: behavioral testing of NLP models with checklist. arXiv preprint arXiv:2005.04118 (2020)
Radford, A., Narasimhan, K., Salimans, T., Sutskever, I.: Improving language understanding by generative pre-training (2018)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Zhu, C., Xu, R., Zeng, M., Huang, X.: A hierarchical network for abstractive meeting summarization with cross-domain pretraining. arXiv preprint arXiv:2004.02016 (2020)
Liu, C., Wang, P., Xu, J., Li, Z., Ye, J.: Automatic dialogue summary generation for customer service. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1957–1965 (2019)
Lewis, M., et al.: BART: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv preprint arXiv:1910.13461 (2019)
Raffel, C., et al.: Exploring the limits of transfer learning with a unified text-to-text transformer. arXiv preprint arXiv:1910.10683 (2019)
Zhang, J., Zhao, Y., Saleh, M., Liu, P.: PEGASUS: pre-training with extracted gap-sentences for abstractive summarization. In: International Conference on Machine Learning, pp. 11328–11339. PMLR (2020)
Xu, J., Ren, X., Zhang, Y., Zeng, Q., Cai, X., Sun, X.: A skeleton-based model for promoting coherence among sentences in narrative story generation. arXiv preprint arXiv:1808.06945 (2018)
Fan, A., Lewis, M., Dauphin, Y.: Strategies for structuring story generation. arXiv preprint arXiv:1902.01109 (2019)
Goldfarb-Tarrant, S., Chakrabarty, T., Weischedel, R., Peng, N.: Content planning for neural story generation with aristotelian rescoring. arXiv preprint arXiv:2009.09870 (2020)
Peng, N., Ghazvininejad, M., May, J., Knight, K.: Towards controllable story generation. In: Proceedings of the First Workshop on Storytelling, pp. 43–49 (2018)
Chandu, K., Prabhumoye, S., Salakhutdinov, R., Black, A.W.: “My way of telling a story”: persona based grounded story generation. In: Proceedings of the Second Workshop on Storytelling, pp. 11–21 (2019)
Huang, Q., Gan, Z., Celikyilmaz, A., Wu, D., Wang, J., He, X.: Hierarchically structured reinforcement learning for topically coherent visual story generation. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 8465–8472 (2019)
Rose, S., Engel, D., Cramer, N., Cowley, W.: Automatic keyword extraction from individual documents. In: Text Mining: Applications and Theory, vol. 1, pp. 1–20 (2010)
Reimers, N., Gurevych, I.: Sentence-BERT: sentence embeddings using siamese BERT-networks. arXiv preprint arXiv:1908.10084 (2019)
Mostafazadeh, N., et al.: A corpus and cloze evaluation for deeper understanding of commonsense stories. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 839–849 (2016)
Papineni, K., Roukos, S., Ward, T., Zhu, W.J.: BLEU: a method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pp. 311–318 (2002)
Acknowledgments
This work was funded by Chongqing Technology Innovation and Application Development (Major Theme Special) Project, Development and Application of Cloud Service Platform for Intelligent Detection and Monitoring of Industrial Equipment, NO. cstc2019jscx-zdztzx0043.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Guo, Z., Wan, J., Tang, H., Lu, Y. (2022). Story Generation Based on Multi-granularity Constraints. In: Fang, L., Povey, D., Zhai, G., Mei, T., Wang, R. (eds) Artificial Intelligence. CICAI 2022. Lecture Notes in Computer Science(), vol 13606. Springer, Cham. https://doi.org/10.1007/978-3-031-20503-3_18
Download citation
DOI: https://doi.org/10.1007/978-3-031-20503-3_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-20502-6
Online ISBN: 978-3-031-20503-3
eBook Packages: Computer ScienceComputer Science (R0)