Abstract
Pre-trained Relation Extraction (RE) models are widely employed in relation prediction in Knowledge Base Question Answering (KBQA). However, pre-trained models are usually optimized and evaluated on datasets (e.g. GLUE) which contain various Natural Language Processing (NLP) tasks except a RE task. As a result, it is difficult to select a best pre-trained model for relation prediction unless we evaluate all available pre-trained models on a relation prediction dataset. As the Semantic Similarity (SS) task in GLUE is similar to a RE task, a Semantic Similarity Model for Relation Prediction (SSMFRP) is proposed in this paper to convert a RE task in relation prediction to a SS task. In our model, a relation candidate in a RE model is converted into the corresponding question which contains a relation candidate. Then a modified SS model is employed to find the best-matched relation. Experimental results show that the effectiveness of our proposed model in relation prediction is related to the effectiveness of the original pre-trained model in SS and GLUE. Our model achieves an average accuracy of 91.2% with various pre-trained models and outperforms original models by an average margin of 1.8% with the similar training cost. In addition, further experiments show that our model is robust to abnormal input and outperforms original models by an average margin of 1.0% on datasets of abnormal input.
Supported by the Natural Science Foundation of China under Grant No. U21A20491, No. U1936109, No. U1908214.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bollacker, K., Evans, C., Paritosh, P., Sturge, T., Taylor, J.: Freebase: a collaboratively created graph database for structuring human knowledge. In: Proceedings of the 2008 ACM SIGMOD International conference on Management of Data, pp. 1247–1250 (2008)
Bordes, A., Usunier, N., Chopra, S., Weston, J.: Large-scale simple question answering with memory networks. arXiv preprint arXiv:1506.02075 (2015)
Clark, K., Luong, M.T., Le, Q.V., Manning, C.D.: Electra: pre-training text encoders as discriminators rather than generators. In: Proceedings of The International Conference on Learning Representations (2020)
Dai, Z., Li, L., Xu, W.: Cfo: conditional focused neural question answering with large-scale knowledge bases. In: Proceedings of Annual Meeting of the Association for Computational Linguistics, pp. 800–810 (2016)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 4171–4186 (2019)
Fikri, F.B., Oflazer, K., Yanikoglu, B.: Semantic similarity based evaluation for abstractive news summarization. In: Proceedings of the 1st Workshop on Natural Language Generation, Evaluation, and Metrics, pp. 24–33 (2021)
Hu, Z., Cao, Y., Huang, L., Chua, T.S.: How knowledge graph and attention help? a qualitative analysis into bag-level relation extraction. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics. p. 4662–4671 (2021)
Huang, B., Bai, Y., Zhou, X.: Hub at semeval-2021 task 2: word meaning similarity prediction model based on roberta and word frequency. In: Proceedings of the 15th International Workshop on Semantic Evaluation, pp. 719–723 (2021)
Huang, K., Qi, P., Wang, G., et al.: Entity and evidence guided document-level relation extraction. In: Proceedings of the 6th Workshop on Representation Learning for NLP, pp. 307–315 (2021)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: Proceedings of the 3rd International Conference on Learning Representations (2015)
Lan, Z., Chen, M., Goodman, S., et al.: Albert: a lite bert for self-supervised learning of language representations. arXiv preprint arXiv:1909.11942 (2019)
Luo, D., Su, J., Yu, S.: A bert-based approach with relation-aware attention for knowledge base question answering. In: Proceedings of International Joint Conference on Neural Networks, pp. 1–8 (2020)
Ma, R., Gui, T., Li, L., et al.: Sent: Sentence-level distant relation extraction via negative training. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics, pp. 6201–6213 (2021)
Petrochuk, M., Zettlemoyer, L.: Simplequestions nearly solved: a new upperbound and baseline approach. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 554–558 (2018)
Risch, J., Möller, T., Gutsch, J.: Semantic answer similarity for evaluating question answering models. In: Proceedings of the 3rd Workshop on Machine Reading for Question Answering, pp. 149–157 (2021)
Wang, A., Singh, A., Michael, J., et al.: Glue: A multi-task benchmark and analysis platform for natural language understanding. In: Proceedings of the 7th International Conference on Learning Representations (2019)
Wang, R., Ling, Z., Hu, Y.: Knowledge base question answering with attentive pooling for question representation. IEEE Access 7, 46773–46784 (2019)
Yan, H., Gui, T., Dai, J., et al.: A unified generative framework for various NER subtasks. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics, pp. 5808–5822 (2021)
Zhang, X., Zong, B., Cheng, W., et al.: Unsupervised concept representation learning for length-varying text similarity. In: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 5611–5620 (2021)
Zhu, W.: Autorc: improving BERT based relation classification models via architecture search. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing: Student Research Workshop, pp. 33–43 (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Wang, Z., Xu, X., Li, X., Song, X., Wei, X., Huang, D. (2022). SSMFRP: Semantic Similarity Model for Relation Prediction in KBQA Based on Pre-trained Models. In: Pimenidis, E., Angelov, P., Jayne, C., Papaleonidas, A., Aydin, M. (eds) Artificial Neural Networks and Machine Learning – ICANN 2022. ICANN 2022. Lecture Notes in Computer Science, vol 13530. Springer, Cham. https://doi.org/10.1007/978-3-031-15931-2_25
Download citation
DOI: https://doi.org/10.1007/978-3-031-15931-2_25
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-15930-5
Online ISBN: 978-3-031-15931-2
eBook Packages: Computer ScienceComputer Science (R0)