Abstract
Generative question answering tasks usually suffer from the challenge of the lack of external knowledge. The generative question answering model cannot understand the intention of questions effectively because the questions asked by users are short and the amount of information is insufficient. Therefore, it needs to be supplemented by external knowledge. In this paper, we propose a generative question answering model combined with knowledge graph (KG-Transformer), which can solve the problem of inaccurate generation caused by the above challenge. The advantage of KG-Transformer is that it designs a knowledge retrieval module which can obtain external knowledge from the knowledge graph as a supplement to the intention of question. Besides, compared with the traditional sentence similarity method and hard fusion, it uses a soft switching mechanism, which can switch between the knowledge vector and the question vector, effectively extracting knowledge information and questions information and then fusion. Experiments on a benchmark dataset demonstrate that our model has robust superiority over compared methods in generating informative and accurate answer.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Vinyals, O., Le, Q.: A neural conversational model. arXiv preprint arXiv:1506.05869 (2015)
Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473 (2014)
Cao, K., Clark, S.: Latent variable dialogue models and their diversity. arXiv preprint arXiv:1702.05962 (2017)
Li, J., Monroe, W., Shi, T., Jean, S., Ritter, A., Jurafsky, D.: Adversarial learning for neural dialogue generation. arXiv preprint arXiv:1701.06547 (2017)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 5998–6008 (2017)
He, S., Liu, C., Liu, K., Zhao, J.: Generating natural answers by incorporating copying and retrieving mechanisms in sequence-to-sequence learning. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 199–208 (2017)
Annervaz, K., Chowdhury, S.B.R., Dukkipati, A.: Learning beyond datasets: knowledge graph augmented neural networks for natural language processing. arXiv preprint arXiv:1802.05930 (2018)
Madotto, A., Wu, C.-S., Fung, P.: Mem2Seq: effectively incorporating knowledge bases into end-to-end task-oriented dialog systems. arXiv preprint arXiv:1804.08217 (2018)
See, A., Liu, P.J., Manning, C.D.: Get to the point: summarization with pointer-generator networks. arXiv preprint arXiv:1704.04368 (2017)
Yang, B., Mitchell, T.: Leveraging knowledge bases in LSTMs for improving machine reading. arXiv preprint arXiv:1902.09091 (2019)
An, W., Chen, Q., Yang, Y., He, L.: Knowledge memory based LSTM model for answer selection. In: Liu, D., Xie, S., Li, Y., Zhao, D., El-Alfy, E.S. (eds.) Neural Information Processing. International Conference on Neural Information Processing, pp. 34–42. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-70096-0_4
Tu, Z., Jiang, Y., Liu, X., Shu, L., Shi, S.: Generative stock question answering. arXiv preprint arXiv:1804.07942 (2018)
Papineni, K., Roukos, S., Ward, T., Zhu, W.-J.: BLEU: a method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pp. 311–318 (2002)
Liu, C.-W., Lowe, R., Serban, I.V., Noseworthy, M., Charlin, L., Pineau, J.: How not to evaluate your dialogue system: an empirical study of unsupervised evaluation metrics for dialogue response generation. arXiv preprint arXiv:1603.08023 (2016)
Shen, X., Su, H., Niu, S., Demberg, V.: Improving variational encoder-decoders in dialogue generation. arXiv preprint arXiv:1802.02032 (2018)
Serban, I., et al.: A hierarchical latent variable encoder-decoder model for generating dialogues. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 31 (2017)
Li, J., Monroe, W., Ritter, A., Galley, M., Gao, J., Jurafsky, D.: Deep reinforcement learning for dialogue generation. arXiv preprint arXiv:1606.01541 (2016)
Sukhbaatar, S., Szlam, A., Weston, J., Fergus, R.: End-to-end memory networks. arXiv preprint arXiv:1503.08895 (2015)
Berant, J., Liang, P.: Semantic parsing via paraphrasing. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1415–1425 (2014)
Aho, A.V., Corasick, M.J.: Efficient string matching: an aid to bibliographic search. Commun. ACM 18(6), 333–340 (1975)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Ba, J.L., Kiros, J.R., Hinton, G.E.: Layer normalization. arXiv preprint arXiv:1607.06450 (2016)
Yan, G., Li, J.: Mobile medical question and answer system with improved char-level based convolution neural network and sparse auto encoder. In: Proceedings of the 2019 Asia Pacific Information Technology Conference, pp. 20–24 (2019)
Luong, M.-T., Sutskever, I., Le, Q.V., Vinyals, O., Zaremba, W.: Addressing the rare word problem in neural machine translation. arXiv preprint arXiv:1410.8206 (2014)
Acknowledgements
This research is supported by the Scientific Research Platforms and Projects in Universities in Guangdong Province under Grants 2019KTSCX204 and the Stable Support Projects for Shenzhen Higher Education Institutions under Grants SZWD2021011.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Liang, C., Yang, J., Fu, X. (2021). Knowledge Graph Enhanced Transformer for Generative Question Answering Tasks. In: Farkaš, I., Masulli, P., Otte, S., Wermter, S. (eds) Artificial Neural Networks and Machine Learning – ICANN 2021. ICANN 2021. Lecture Notes in Computer Science(), vol 12891. Springer, Cham. https://doi.org/10.1007/978-3-030-86362-3_22
Download citation
DOI: https://doi.org/10.1007/978-3-030-86362-3_22
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-86361-6
Online ISBN: 978-3-030-86362-3
eBook Packages: Computer ScienceComputer Science (R0)