Abstract
Candidate ranking is the process of selecting the candidate with the best matching probability to the question after generating candidates in the knowledge base question answering (KBQA) task. It is a representative problem in mining matching relationships between candidates and questions. Previous research works always model questions and candidate representations separately, ignoring their impact on each other. The text information is too short to capture rich features in the KBQA task. Therefore, our work presents an attention neural network (ANN) fused with structural features (SF-ANN) to rank candidate facts jointly. First, two types of attention mechanisms are used to capture the correlation between the question and the candidate fact: a mutual-attention mechanism that captures the correspondence between the sentence components of a question and each part of a candidate and an intra-attention mechanism that captures the self-dependency of the concatenation between a question and a candidate fact. Second, an ANN is designed for fusing these two types of attention mechanisms to deeply couple interactive information of the input. Finally, knowledge base structural features are introduced to supplement the semantic information to increase the richness of the information. Three mutual attention mechanisms are applied for fusing them into the ANN, resulting in higher information gain. The experimental results on the SimpleQuestions (SimpleQ) benchmark demonstrate that the proposed model achieves a higher ranking accuracy (82.9%) than the state-of-the-art models. Moreover, the ablation study on SimpleQ and WebQuestionsSP (WebQSP) shows that leveraged features and the propoesd ANN both contribute to performance improvement.
Similar content being viewed by others
Notes
The experimental result is from Table 3 of [36]
References
Bao J, Duan N, Yan Z, Zhou M, Zhao T (2016) Constraint-based question answering with knowledge graph. In: Calzolari N, Matsumoto Y, Prasad R (eds) COLING 2016, 26th International Conference on Computational Linguistics, Proceedings of the Conference: Technical Papers. ACL, Osaka, pp 2503–2514
Berant J, Chou A, Frostig R, Liang P (2013) Semantic parsing on Freebase from question-answer pairs. In: Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, Seattle, pp 1533–1544
Bollacker K, Evans C, Paritosh P, Sturge T, Taylor J (2008) Freebase: a collaboratively created graph database for structuring human knowledge. In: In SIGMOD Conference, pp 1247–1250
Bordes A, Usunier N, Chopra S, Weston J (2015) Large-scale simple question answering with memory networks. CoRR arXiv:1506.02075
Chen Y, Li H (2020) Dam: Transformer-based relation detection for question answering over knowledge base. Knowl-Based Syst 106077:201–202. https://doi.org/10.1016/j.knosys.2020.106077
Chen Y, Wu L, Zaki MJ (2019) Bidirectional attentive memory networks for question answering over knowledge bases. In: Burstein J, Doran C, Solorio T (eds) Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019. https://doi.org/10.18653/v1/n19-1299, vol 1. Long and Short Papers, Association for Computational Linguistics, Minneapolis, pp 2913–2923
Devlin J, Chang M, Lee K, Toutanova K (2019) BERT: pre-training of deep bidirectional transformers for language understanding. In: Burstein J, Doran C, Solorio T (eds) Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019. https://doi.org/10.18653/v1/n19-1423. Long and Short Papers, Association for Computational Linguistics, Minneapolis, pp 4171–4186
Gupta V, Chinnakotla M, Shrivastava M (2018) Retrieve and re-rank: A simple and effective IR approach to simple question answering over knowledge graphs. In: Proceedings of the First Workshop on Fact Extraction and VERification (FEVER), Association for Computational Linguistics. https://doi.org/10.18653/v1/W18-5504. Brussels, pp 22–27
Hao Y, Zhang Y, Liu K, He S, Liu Z, Wu H, Zhao J (2017) An end-to-end model for question answering over knowledge base with cross-attention combining global knowledge. In: Barzilay R, Kan M (eds) Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017. https://doi.org/10.18653/v1/P17-1021, vol 1. Long Papers, Association for Computational Linguistics, Vancouver, pp 221–231
Hao Y, Liu H, He S, Liu K, Zhao J (2018) Pattern-revising enhanced simple question answering over knowledge bases. In: Bender E M, Derczynski L, Isabelle P (eds) Proceedings of the 27th International Conference on Computational Linguistics, COLING 2018. Association for Computational Linguistics, Santa Fe, pp 3272–3282
Jin ZX, Zhang BW, Zhou F, Qin J, Yin XC (2020) Ranking via partial ordering for answer selection. Inf Sci 538:358–371. https://doi.org/10.1016/j.ins.2020.05.110
Kingma DP, Ba J (2015) Adam: A method for stochastic optimization. In: Bengio Y, LeCun Y (eds) 3rd International Conference on Learning Representations, ICLR 2015. Conference Track Proceedings, San Diego
Lan Y, Jiang J (2020) Query graph generation for answering multi-hop complex questions from knowledge bases. In: Jurafsky D, Chai J, Schluter N, Tetreault J R (eds) Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online. Association for Computational Linguistics, pp 969–974
Liang C, Berant J, Le Q, Forbus KD, Lao N (2017) Neural symbolic machines: Learning semantic parsers on Freebase with weak supervision. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. https://doi.org/10.18653/v1/P17-1003, vol 1. (Long Papers), Association for Computational Linguistics, Vancouver, Canada, pp 23–33
Lukovnikov D, Fischer A, Lehmann J, Auer S (2017) Neural network-based question answering over knowledge graphs on word and character level. In: Barrett R, Cummings R, Agichtein E, Gabrilovich E (eds) Proceedings of the 26th International Conference on World Wide Web, WWW 2017. https://doi.org/10.1145/3038912.3052675. ACM, Perth, pp 1211–1220
Luo K, Lin F, Luo X, Zhu KQ (2018) Knowledge base question answering via encoding of complex query graphs. In: Riloff E, Chiang D, Hockenmaier J, Tsujii J (eds) Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. https://doi.org/10.18653/v1/d18-1242. Association for Computational Linguistics, Brussels, pp 2185–2194
Ma Y, Peng H, Cambria E (2018) Targeted aspect-based sentiment analysis via embedding commonsense knowledge into an attentive LSTM. In: McIlraith S A, Weinberger K Q (eds) Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18). AAAI Press, New Orleans, pp 5876–5883
Maruf S, Martins AFT, Haffari G (2019) Selective attention for context-aware neural machine translation. In: Burstein J, Doran C, Solorio T (eds) Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019. https://doi.org/10.18653/v1/n19-1313, vol 1. Long and Short Papers, Association for Computational Linguistics, Minneapolis, pp 3092–3102
Meng F, Lu Z, Li H, Liu Q (2016) Interactive attention for neural machine translation. In: Calzolari N, Matsumoto Y, Prasad R (eds) COLING 2016, 26th International Conference on Computational Linguistics, Proceedings of the Conference: Technical Papers. ACL, Osaka, pp 2174–2185
Miller AH, Fisch A, Dodge J, Karimi A, Bordes A, Weston J (2016) Key-value memory networks for directly reading documents. In: Su J, Carreras X, Duh K (eds) Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016. https://doi.org/10.18653/v1/d16-1147. The Association for Computational Linguistics, Austin, pp 1400–1409
Pennington J, Socher R, Manning CD (2014) Glove: Global vectors for word representation. In: Moschitti A, Pang B, Daelemans W (eds) Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014. https://doi.org/10.3115/v1/d14-1162. A meeting of SIGDAT, a Special Interest Group of the ACL, ACL, Doha, pp 1532–1543
Sorokin D, Gurevych I (2018) Modeling semantics with gated graph neural networks for knowledge base question answering. In: Bender E M, Derczynski L, Isabelle P (eds) Proceedings of the 27th International Conference on Computational Linguistics, COLING 2018. Association for Computational Linguistics, Santa Fe, pp 3306–3317
Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I (2017) Attention is all you need. In: Guyon I, von Luxburg U, Bengio S, Wallach H M, Fergus R, Vishwanathan S V N, Garnett R (eds) Advances in neural information processing systems 30: Annual conference on neural information processing systems 2017, Long Beach, pp 5998–6008
Wang C, Zhao R (2019) Multi-candidate ranking algorithm based spell correction. In: Degenhardt J, Kallumadi S, Porwal U, Trotman A (eds) Proceedings of the SIGIR 2019 Workshop on eCommerce, co-located with the 42st International ACM SIGIR Conference on Research and Development in Information Retrieval, eCom@SIGIR 2019, vol 2410. CEUR-WS.org, CEUR Workshop Proceedings, Paris
Wang R, Ling Z, Hu Y (2019) Knowledge base question answering with attentive pooling for question representation. IEEE Access 7:46773–46784. https://doi.org/10.1109/ACCESS.2019.2909826
Wang W, Pan SJ, Dahlmeier D, Xiao X (2017) Coupled multi-layer attentions for co-extraction of aspect and opinion terms. In: Singh S P, Markovitch S (eds) Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, vol 2017. AAAI Press, San Francisco, pp 3316–3322
Wang Y, Huang M, Zhu X, Zhao L (2016) Attention-based LSTM for aspect-level sentiment classification. In: Su J, Carreras X, Duh K (eds) Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016. https://doi.org/10.18653/v1/d16-1058. The Association for Computational Linguistics, Austin, pp 606–615
Wang Y, Zhang R, Xu C, Mao Y (2018) The APVA-TURBO approach to question answering in knowledge base. In: Bender E M, Derczynski L, Isabelle P (eds) Proceedings of the 27th International Conference on Computational Linguistics, COLING 2018. Association for Computational Linguistics, Santa Fe, pp 1998–2009
Yan Y, Zhang BW, Li XF, Liu Z (2020) List-wise learning to rank biomedical question-answer pairs with deep ranking recursive autoencoders. PloS one 15(11):e0242061
Yang Y, Chang MW (2015) S-MART: Novel Tree-based structured learning algorithms applied to tweet entity linking. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), Association for Computational Linguistics, Beijing, pp 504–513. https://doi.org/10.3115/v1/P15-1049
Yang Z, Yang D, Dyer C, He X, Smola AJ, Hovy EH (2016) Hierarchical attention networks for document classification. In: Knight K, Nenkova A, Rambow O (eds) NAACL HLT 2016, The 2016 conference of the north american chapter of the association for computational linguistics: Human language technologies. https://doi.org/10.18653/v1/n16-1174. The Association for Computational Linguistics, San Diego, pp 1480–1489
Yih W, Chang M, He X, Gao J (2015) Semantic parsing via staged query graph generation: Question answering with knowledge base. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, ACL 2015. https://doi.org/10.3115/v1/p15-1128, vol 1. Long Papers, The Association for Computer Linguistics, Beijing, pp 1321–1331
Wt Yih, Richardson M, Meek C, Chang MW, Suh J (2016) The value of semantic parse labeling for knowledge base question answering. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. https://doi.org/10.18653/v1/P16-2033, vol 2. Short Papers, Association for Computational Linguistics, Berlin, pp 201–206
Yin W, Schütze H, Xiang B, Zhou B (2016a) ABCNN: attention-based convolutional neural network for modeling sentence pairs. Trans Assoc Comput Linguist 4:259–272
Yin W, Yu M, Xiang B, Zhou B, Schu̇tze H (2016b) Simple question answering by attentive convolutional neural network. In: Calzolari N, Matsumoto Y, Prasad R (eds) COLING 2016, 26th International Conference on Computational Linguistics, Proceedings of the Conference: Technical Papers. ACL, Osaka, pp 1746–1756
Yu M, Yin W, Hasan KS, dos Santos CN, Xiang B, Zhou B (2017) Improved neural relation detection for knowledge base question answering. In: Barzilay R, Kan M (eds) Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017. https://doi.org/10.18653/v1/P17-1053, vol 1. Long Papers, Association for Computational Linguistics, Vancouver, pp 571–581
Zhao S, Zhang Z (2018) Attention-via-attention neural machine translation. In: McIlraith S A, Weinberger K Q (eds) Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18). AAAI Press, New Orleans, pp 563–570
Zhao W, Chung T, Goyal AK, Metallinou A (2019) Simple question answering with subgraph ranking and joint-scoring. In: Burstein J, Doran C, Solorio T (eds) Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019. https://doi.org/10.18653/v1/n19-1029, vol 1. Long and Short Papers, Association for Computational Linguistics, Minneapolis, pp 324–334
Zhou G, Xie Z, Yu Z, Huang JX (2021) Dfm: a parameter-shared deep fused model for knowledge base question answering. Inf Sci 547:103–118. https://doi.org/10.1016/j.ins.2020.08.037
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Zhang, Y., Jin, L., Zhang, Z. et al. SF-ANN: leveraging structural features with an attention neural network for candidate fact ranking. Appl Intell 52, 5841–5856 (2022). https://doi.org/10.1007/s10489-021-02739-y
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-021-02739-y