Skip to main content

Advertisement

Springer Nature Link
Log in

A study of approaches to answering complex questions over knowledge bases

  • Survey Paper
  • Published:
Knowledge and Information Systems Aims and scope Submit manuscript

Abstract

Question answering (QA) systems retrieve the most relevant answer to a natural language question. Knowledge base question answering (KBQA) systems explore entities and relations from knowledge bases to generate answers. Currently, QA systems achieve better results when answering simple questions, but complex QA systems are receiving great attention nowadays. However, there is a lack of studies that analyzes complex questions inside the KBQA field and how it has been addressed. This work aims to fill this gap, presenting a systematic mapping on the complex knowledge base question answering (C-KBQA). The main contributions of this work are: (i) the use of a systematic method to provide an overview of C-KBQA; (ii) a collection of 54 papers systematically selected from 894 papers; (iii) the identification of the most frequent venues, domains, and knowledge bases used in the literature; (iv) a mapping of methods, datasets, and metrics used in the C-KBQA scenario; (v) future directions and the main gaps in the C-KBQA field. The authors show that the C-KBQA system aims to solve two question types: multi-hop and constraint questions. Also, it was possible to identify three main steps to construct a C-KBQA system and the use of two main approaches in this process. It was also noticed that datasets for C-KBQA are still an open challenge.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. http://qald.aksw.org/.

  2. As Google Scholar does not have native metadata feature filtering, we created a script that does it using the HTML of the pages and Regular Expressions. The script is freely available and can be accessed at https://github.com/lapic-ufjf/gscholar-review-filter.

  3. http://parsif.al.

  4. https://github.com/lapic-ufjf/CKBQA-systematic-mapping-2021.

  5. We used the web API of Stanford CoreNLP package in version 4.0.0 (updated 2020-04-16). It can be accessed at https://corenlp.run/.

  6. https://universaldependencies.org/u/dep/advmod.html.

  7. https://nlp.stanford.edu/software/CRF-NER.html.

  8. TSV files available at https://github.com/lapic-ufjf/CKBQA-systematic-mapping-2021.

  9. https://www.w3.org/TR/rdfa-primer/.

References

  1. Abdelkawi A, Zafar H, Maleshkova M, Lehmann J (2019) Complex query augmentation for question answering over knowledge graphs. In: OTM confederated international conferences” On the move to meaningful internet systems”. Springer, pp 571–587

  2. Abujabal A, Yahya M, Riedewald M, Weikum G (2017) Automated template generation for question answering over knowledge graphs. In: Proceedings of the 26th international conference on world wide web, pp 1191–1200

  3. Agarwal P, Ramanath M, Shroff G (2019) Retrieving relationships from a knowledge graph for question answering. In: Azzopardi L, Stein B, Fuhr N, Mayr P, Hauff C, Hiemstra D (eds) Advances in information retrieval. Springer, Cham, pp 35–50

    Chapter  Google Scholar 

  4. Anderson M, Gómez-Rodríguez C (2020) Distilling neural networks for greener and faster dependency parsing

  5. Auer S, Bizer C, Kobilarov G, Lehmann J, Cyganiak R, Ives Z (2007) Dbpedia: a nucleus for a web of open data. In: The semantic web. Springer, pp 722–735

  6. Bakhshi M, Nematbakhsh M, Mohsenzadeh M, Rahmani AM (2020) Data-driven construction of sparql queries by approximate question graph alignment in question answering over knowledge graphs. Expert Syst Appl 146:113205

    Article  Google Scholar 

  7. Bao J, Duan N, Yan Z, Zhou M, Zhao T (2016) Constraint-based question answering with knowledge graph. In: Proceedings of COLING 2016, the 26th international conference on computational linguistics: technical papers. The COLING 2016 Organizing Committee, Osaka, Japan, pp 2503–2514. https://www.aclweb.org/anthology/C16-1236

  8. Bast H, Haussmann E (2015) More accurate question answering on freebase. In: Proceedings of the 24th ACM international on conference on information and knowledge management, pp 1431–1440

  9. Berant J, Chou A, Frostig R, Liang P (2013) Semantic parsing on freebase from question-answer pairs. In: Proceedings of the 2013 conference on empirical methods in natural language processing, pp 1533–1544

  10. Berners-Lee T, Hendler J, Lassila O (2001) The semantic web. Sci Am 284(5):34–43

    Article  Google Scholar 

  11. Bhutani N, Zheng X, Jagadish HV (2019) Learning to answer complex questions over knowledge bases with query composition. In: Proceedings of the 28th ACM international conference on information and knowledge management, CIKM ’19. Association for Computing Machinery, New York, NY, USA, pp 739–748. https://doi.org/10.1145/3357384.3358033

  12. Bhutani N, Zheng X, Qian K, Li Y, Jagadish H (2020) Answering complex questions by combining information from curated and extracted knowledge bases. In: Proceedings of the first workshop on natural language interfaces. Association for Computational Linguistics, Online, pp 1–10. https://doi.org/10.18653/v1/2020.nli-1.1

  13. Bizer C, Heath T, Berners-Lee T, Sheth A (2011) Linked data-the story so far. Semantic services, interoperability and web applications: emerging concepts, vol 1, no. 1, pp 205–227

  14. Bojanowski P, Grave E, Joulin A, Mikolov T (2017) Enriching word vectors with subword information. Trans Assoc Comput Linguist 5:135–146

    Article  Google Scholar 

  15. Bollacker K, Cook R, Tufts P (2007) Freebase: a shared database of structured general human knowledge. In: AAAI, vol 7, pp 1962–1963 (2007)

  16. Bollacker K, Evans C, Paritosh P, Sturge T, Taylor J (2008) Freebase: a collaboratively created graph database for structuring human knowledge. In: Proceedings of the 2008 ACM SIGMOD international conference on Management of data, pp 1247–1250

  17. Bordes A, Usunier N, Chopra S, Weston J (2015) Large-scale simple question answering with memory networks. arXiv preprint arXiv:1506.02075

  18. Bosselut A, Bras RL, Choi Y (2020) Dynamic neuro-symbolic knowledge graph construction for zero-shot commonsense question answering. In: Proceedings of the AAAI conference on Artificial Intelligence, vol 35, pp 4923–4931. https://ojs.aaai.org/index.php/AAAI/article/view/16625

  19. Cai Q, Yates A (2013) Large-scale semantic parsing via schema matching and lexicon extension. In: Proceedings of the 51st annual meeting of the association for computational linguistics, vol 1. Long Papers, pp 423–433

  20. Chen A, Stanovsky G, Singh S, Gardner M (2019) Evaluating question answering evaluation. In: Proceedings of the 2nd workshop on machine reading for question answering. Association for Computational Linguistics, Hong Kong, China, p. 119–124. https://doi.org/10.18653/v1/D19-5817

  21. Chen X, Jia S, Xiang Y (2020) A review: knowledge reasoning over knowledge graph. Expert Syst Appl 141:112948. https://doi.org/10.1016/j.eswa.2019.112948

    Article  Google Scholar 

  22. Cooper ID (2016) What is a mapping study? J Med Lib Assoc JMLA 104(1):76

    Article  Google Scholar 

  23. Costa C, Murta L (2013) Version control in distributed software development: a systematic mapping study. In: 8th international conference on global software engineering (ICGSE), pp 90–99. IEEE (2013)

  24. Croft WB, Metzler D, Strohman T (2010) Search engines: information retrieval in practice, vol 520. Addison-Wesley Publishing, Boston

    Google Scholar 

  25. Cui W, Xiao Y, Wang H, Song Y, Hwang S, Wang W (2019) KBQA: learning question answering over QA corpora and knowledge bases. CoRR. arXiv:1903.02419

  26. Cui W, Xiao Y, Wang H, Song Y, Hwang SW, Wang W (2017) KBQA: learning question answering over QA corpora and knowledge bases. Proc VLDB Endow 10(5):565–576. https://doi.org/10.14778/3055540.3055549

    Article  Google Scholar 

  27. Dettmers T, Minervini P, Stenetorp P, Riedel S (2018) Convolutional 2d knowledge graph embeddings. In: McIlraith SA, Weinberger KQ (eds) Proceedings of the thirty-second AAAI conference on artificial intelligence, (AAAI-18), the 30th innovative applications of artificial intelligence (IAAI-18), and the 8th AAAI symposium on educational advances in artificial intelligence (EAAI-18), New Orleans, Louisiana, USA, February 2–7, 2018. AAAI Press, New Orleans, pp 1811–1818

  28. Diefenbach D, Both A, Singh K, Maret P (2020) Towards a question answering system over the semantic web. Semant Web 11(3):421–439. https://doi.org/10.3233/SW-190343

  29. Diefenbach D, Lopez V, Singh K, Maret P (2018) Core techniques of question answering systems over knowledge bases: a survey. Knowl Inf Syst 55(3):529–569

    Article  Google Scholar 

  30. Dimitrakis E, Sgontzos K, Tzitzikas Y (2019) A survey on question answering systems over linked data and documents. J Intell Inf Syst 55:1–27

    Google Scholar 

  31. Ding J, Hu W, Xu Q, Qu Y (2019) Leveraging frequent query substructures to generate formal queries for complex question answering. In: Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP). Association for Computational Linguistics, Hong Kong, China, pp 2614–2622. https://doi.org/10.18653/v1/D19-1263

  32. Dubey M, Banerjee D, Abdelkawi A, Lehmann J (2019) LC-quad 2.0: a large dataset for complex question answering over wikidata and dbpedia. In: International semantic web conference. Springer, pp 69–78 (2019)

  33. Evseev DA, Arkhipov MY (2020) Sparql query generation for complex question answering with bert and bilstm-based model. In: Computational linguistics and intellectual technologies: proceedings of the international conference dialogue 2020, vol 2020-June, pp 270–282

  34. Fernandez-Sotos P, Torio I, Fernandez-Caballero A, Navarro E, Gonzalez P, Dompablo M, Rodriguez-Jimenez R (2019) Social cognition remediation interventions: a systematic mapping review. PloS One 14(6):1–20. https://doi.org/10.1371/journal.pone.0218720

  35. Greenhalgh T, Peacock R (2005) Effectiveness and efficiency of search methods in systematic reviews of complex evidence: audit of primary sources. BMJ 331(7524):1064–1065. https://doi.org/10.1136/bmj.38636.593461.68

    Article  Google Scholar 

  36. Gu G, Li B, Gao H, Wang M (2020) Learning to answer complex questions with evidence graph. In: Asia-Pacific Web (APWeb) and Web-Age Information Management (WAIM) joint international conference on web and big data. Springer, pp 257–269

  37. Gusenbauer M, Haddaway NR (2020) Which academic search systems are suitable for systematic reviews or meta-analyses? evaluating retrieval qualities of google scholar, pubmed, and 26 other resources. Res Synth Methods 11(2):181–217

    Article  Google Scholar 

  38. Haddaway NR, Bernes C, Jonsson BG, Hedlund K (2016) The benefits of systematic mapping to evidence-based environmental management. Ambio 45(5):613–620

    Article  Google Scholar 

  39. Han W, Jiang Y, Ng HT, Tu K (2020) A survey of unsupervised dependency parsing. In: Proceedings of the 28th International Conference on Computational Linguistics, pp 2522–2533, Barcelona, Spain (Online). International Committee on Computational Linguistics. https://doi.org/10.18653/v1/2020.coling-main.227

  40. Hao Z, Wu B, Wen W, Cai R (2019) A subgraph-representation-based method for answering complex questions over knowledge bases. Neural Netw 119:57–65

    Article  Google Scholar 

  41. Höffner K, Walter S, Marx E, Usbeck R, Lehmann J, Ngonga Ngomo AC (2017) Survey on challenges of question answering in the semantic web. Semant Web 8(6):895–920

    Article  Google Scholar 

  42. Hong MD, Oh KJ, Go SH, Jo GS (2016) Temporal ontology representation and reasoning using ordinals and sets for historical events. In: Asian conference on intelligent information and database systems. Springer, pp 75–85

  43. Hu S, Zou L, Yu JX, Wang H, Zhao D (2017) Answering natural language questions by subgraph matching over knowledge graphs. IEEE Trans Knowl Data Eng 30(5):824–837

    Article  Google Scholar 

  44. Hu S, Zou L, Zhang X (2018) A state-transition framework to answer complex questions over knowledge base. In: Proceedings of the 2018 conference on empirical methods in natural language processing. Association for Computational Linguistics, Brussels, Belgium, pp 2098–2108. https://doi.org/10.18653/v1/D18-1234

  45. Hua Y, Li YF, Haffari G, Qi G, Wu T (2020) Few-shot complex knowledge base question answering via meta reinforcement learning. In: Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP). Association for Computational Linguistics, Online, pp 5827–5837. https://doi.org/10.18653/v1/2020.emnlp-main.469

  46. Hua Y, Li YF, Haffari G, Qi G, Wu W (2020) Retrieve, program, repeat: complex knowledge base question answering via alternate meta-learning. In: Bessiere C (ed) Proceedings of the twenty-ninth international joint conference on artificial intelligence, IJCAI-20. International Joint Conferences on Artificial Intelligence Organization, Virtual, Japan, pp 3679–3686. https://doi.org/10.24963/ijcai.2020/509

  47. Hua Y, Li YF, Qi G, Wu W, Zhang J, Qi D (2020) Less is more: data-efficient complex question answering over knowledge bases. J Web Semant 100612. https://doi.org/10.1016/j.websem.2020.100612. https://www.sciencedirect.com/science/article/pii/S1570826820300470

  48. Imam M et al (2011) Improvements to the complex question answering models. Ph.D. thesis, Lethbridge, Alta.: University of Lethbridge, c2011

  49. Ji S, Pan S, Cambria E, Marttinen P, Yu PS (2020) A survey on knowledge graphs: representation, acquisition and applications. arXiv preprint arXiv:2002.00388

  50. Jia Z, Abujabal A, Saha Roy R, Strötgen J, Weikum G (2018) Tequila: temporal question answering over knowledge bases. In: Proceedings of the 27th ACM international conference on information and knowledge management, pp 1807–1810

  51. Jin H, Luo Y, Gao C, Tang X, Yuan P (2019) ComQA: question answering over knowledge base via semantic matching. IEEE Access 7:75235–75246

    Article  Google Scholar 

  52. Joulin A, Grave E, Bojanowski P, Nickel M, Mikolov T (2017) Fast linear model for knowledge graph embeddings (2017). https://doi.org/10.48550/ARXIV.1710.10881. arXiv:1710.10881

  53. Kartsaklis D, Pilehvar MT, Collier N (2018) Mapping text to knowledge graph entities using multi-sense LSTMs. In: Proceedings of the 2018 conference on empirical methods in natural language processing. Association for Computational Linguistics, Brussels, Belgium, pp 1959–1970. https://doi.org/10.18653/v1/D18-1221

  54. Kitchenham B (2004) Procedures for performing systematic reviews. Keele, UK, Keele University, vol 33, pp 1–26

  55. Lehmann J, Isele R, Jakob M, Jentzsch A, Kontokostas D, Mendes PN, Hellmann S, Morsey M, Van Kleef P, Auer S et al (2015) Dbpedia—a large-scale, multilingual knowledge base extracted from wikipedia. Semant Web 6(2):167–195

    Article  Google Scholar 

  56. Li X, Hu S, Zou L (2020) Natural answer generation via graph transformer. In: Asia-Pacific Web (APWeb) and Web-Age Information Management (WAIM) joint international conference on web and big data. Springer, pp 302–318

  57. Liang C, Berant J, Le Q, Forbus KD, Lao N (2017) Neural symbolic machines: Learning semantic parsers on freebase with weak supervision. In: 55th annual meeting of the association for computational linguistics, ACL 2017. Association for Computational Linguistics (ACL), pp 23–33

  58. Lin BY, Chen X, Chen J, Ren X (2019) Kagnet: knowledge-aware graph networks for commonsense reasoning. In: Inui K, Jiang J, Ng V, Wan X (eds) Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3–7, 2019. Association for Computational Linguistics, pp 2829–2839. https://doi.org/10.18653/v1/D19-1282

  59. Liu Y, Wan Y, He L, Peng H, Yu PS (2020) Kg-bart: Knowledge graph-augmented bart for generative commonsense reasoning

  60. Lu X, Pramanik S, Saha Roy R, Abujabal A, Wang Y, Weikum G (2019) Answering complex questions by joining multi-document evidence with quasi knowledge graphs. In: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, pp 105–114

  61. Lukovnikov D, Fischer A, Lehmann J (2019) Pretrained transformers for simple question answering over knowledge graphs. In: International semantic web conference, vol. abs/2001.11985. Springer, pp 470–486. arxiv:2001.11985

  62. Luo K, Lin F, Luo X, Zhu K (2018) Knowledge base question answering via encoding of complex query graphs. In: Proceedings of the 2018 conference on empirical methods in natural language processing. Association for Computational Linguistics, Brussels, Belgium, pp 2185–2194

  63. Maheshwari G, Trivedi P, Lukovnikov D, Chakraborty N, Fischer A, Lehmann J (2019) Learning to rank query graphs for complex question answering over knowledge graphs. In: International semantic web conference. Springer, pp 487–504

  64. Mendes PN, Jakob M, García-Silva A, Bizer C (2011) Dbpedia spotlight: shedding light on the web of documents. In: Proceedings of the 7th international conference on semantic systems. Association for Computing Machinery, New York, NY, USA, pp 1–8

  65. Meyer B, Choppy C, Staunstrup J, van Leeuwen J (2009) Viewpoint research evaluation for computer science. Commun ACM 52(4):31–34

    Article  Google Scholar 

  66. Mikolov T, Chen K, Corrado G, Dean J (2013) Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781

  67. Miller A, Fisch A, Dodge J, Karimi AH, Bordes A, Weston J (2016) Key-value memory networks for directly reading documents. In: Proceedings of the 2016 conference on empirical methods in natural language processing. Association for Computational Linguistics, Austin, Texas, pp 1400–1409

  68. Nagarhalli TP, Vaze V, Rana NK (2020) A review of current trends in the development of chatbot systems. In: 2020 6th international conference on advanced computing and communication systems (ICACCS). IEEE, Coimbatore, India, pp. 706–710. https://doi.org/10.1109/ICACCS48705.2020.9074420

  69. Namaki MH, Chowdhury FA, Islam M, Doppa J, Wu Y (2017) Learning to speed up query planning in graph databases. In: Proceedings of the international conference on automated planning and scheduling, vol 27, no. 1, p 9. https://ojs.aaai.org/index.php/ICAPS/article/view/13849

  70. Neiva FW, David JMN, Braga R, Campos F (2016) Towards pragmatic interoperability to support collaboration: a systematic review and mapping of the literature. Inf Softw Technol 72:137–150

    Article  Google Scholar 

  71. Nouar W, Boufaida Z (2018) Deep semantic parsing method to capture the structure of multi-relation questions over linked data. In: CEUR workshop proceedings, vol 2379, pp 5–14. https://www.scopus.com

  72. Pennington J, Socher R, Manning CD (2014) Glove: global vectors for word representation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP). Association for Computational Linguistics, Doha, Qatar, pp 1532–1543

  73. Perryman CL (2016) Mapping studies. J Med Lib Assoc JMLA 104(1):79

    Article  Google Scholar 

  74. Petersen K, Feldt R, Mujtaba S, Mattsson M (2008) Systematic mapping studies in software engineering. In: 12th international conference on evaluation and assessment in software engineering (EASE) 12, pp 1–10

  75. Petersen K, Vakkalanka S, Kuzniarz L (2015) Guidelines for conducting systematic mapping studies in software engineering: an update. Inf Softw Technol 64:1–18

    Article  Google Scholar 

  76. Petticrew M, Roberts H (2006) Systematic reviews in the social sciences: a practical guide, vol 6 : Blackwell Publishing CrossRef Google Scholar, Malden, pp 304–305

  77. Popping R (2003) Knowledge graphs and network text analysis. Soc Sci Inf 42(1):91–106

    Article  Google Scholar 

  78. Qiu Y, Wang Y, Jin X, Zhang K (2020) Stepwise reasoning for multi-relation question answering over knowledge graph with weak supervision. In: Proceedings of the 13th international conference on web search and data mining, pp 474–482

  79. Qiu Y, Zhang K, Wang Y, Jin X, Bai L, Guan S, Cheng X (2020) Hierarchical query graph generation for complex question answering over knowledge graph. In: Proceedings of the 29th ACM international conference on information & knowledge management, pp 1285–1294

  80. Radoev N, Zouaq A, Tremblay M, Gagnon M (2018) A language adaptive method for question answering on French and English. In: Semantic web evaluation challenge. Springer, pp 98–113

  81. Rajabi E, Sanchez-Alonso S (2019) Knowledge discovery using SPARQL property path: the case of disease data set. J Inf Sci. https://doi.org/10.1177/0165551519865495

  82. Ramesh K, Ravishankaran S, Joshi A, Chandrasekaran K (2017) A survey of design techniques for conversational agents. In: Kaushik S, Gupta D, Kharb L, Chahal D (eds) Information, communication and computing technology. Springer, Singapore, pp 336–350

    Chapter  Google Scholar 

  83. Reddy ACO, Madhavi K (2020) Convolutional recurrent neural network with template based representation for complex question answering. Int J Electr Comput Eng 10(3):2710

    Google Scholar 

  84. Rodrigo A, Penas A (2017) A study about the future evaluation of question-answering systems. Knowl Based Syst 137:83–93

    Article  Google Scholar 

  85. Saha A, Ansari GA, Laddha A, Sankaranarayanan K, Chakrabarti S (2019) Complex program induction for querying knowledge bases in the absence of gold programs. Trans Assos Comput Linguist 7:185–200

    Article  Google Scholar 

  86. Saha A, Pahuja V, Khapra M, Sankaranarayanan K, Chandar S (2018) Complex sequential question answering: Towards learning to converse over linked question answer pairs with a knowledge graph. In: Proceedings of the AAAI conference on artificial intelligence, vol 32, no. 1, p 9

  87. Sakor A, Singh K, Patel A, Vidal ME (2020) Falcon 2.0: an entity and relation linking tool over wikidata. In: Proceedings of the 29th ACM international conference on information & knowledge management, CIKM ’20. Association for Computing Machinery, New York, NY, USA, pp 3141–3148. https://doi.org/10.1145/3340531.3412777

  88. Sakor A, Singh K, Vidal ME (2019) Falcon: an entity and relation linking framework over dbpedia. In: CEUR workshop proceedings 2456 (2019), vol 2456. RWTH, Aachen, pp 265–268

  89. Schumacher E, Mayfield J, Dredze M (2020) Cross-lingual transfer in zero-shot cross-language entity linking

  90. Shen T, Geng X, Long G, Jiang J, Zhang C, Jiang D (2020) Effective search of logical forms for weakly supervised knowledge-based question answering. In: Bessiere C (ed) Proceedings of the twenty-ninth international joint conference on artificial intelligence, IJCAI-20. International Joint Conferences on Artificial Intelligence Organization, pp 2227–2233. https://doi.org/10.24963/ijcai.2020/308

  91. Shi P, Lin J (2019) Simple bert models for relation extraction and semantic role labeling

  92. Shin S, Lee KH (2020) Processing knowledge graph-based complex questions through question decomposition and recomposition. Inf Sci 523:234–244

    Article  MathSciNet  Google Scholar 

  93. da Silva JWF, Venceslau ADP, Sales JE, Maia JGR, Pinheiro VCM, Vidal VMP (2020) A short survey on end-to-end simple question answering systems. Artif Intell Rev 53(7):5429–5453

    Article  Google Scholar 

  94. Singh K, Radhakrishna AS, Both A, Shekarpour S, Lytra I, Usbeck R, Vyas A, Khikmatullaev A, Punjani D, Lange C, Vidal ME, Lehmann J, Auer S (2018) Why reinvent the wheel: let’s build question answering systems together. In: Proceedings of the 2018 world wide web conference, WWW ’18. International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, pp 1247–1256. https://doi.org/10.1145/3178876.3186023

  95. Talmor A, Berant J (2018) The web as a knowledge-base for answering complex questions. In: Proceedings of the 2018 conference of the North American chapter of the association for computational linguistics: human language technologies, vol 1 (Long Papers), pp 641–651

  96. Tong P, Zhang Q, Yao J (2019) Leveraging domain context for question answering over knowledge graph. Data Sci Eng 4(4):323–335

    Article  Google Scholar 

  97. Tran TK, Gad-Elrab MH, Stepanova D, Kharlamov E, Strötgen J (2020) Fast computation of explanations for inconsistency in large-scale knowledge graphs. In: Proceedings of the web conference 2020, WWW ’20. Association for Computing Machinery, New York, NY, USA, pp 2613–2619. https://doi.org/10.1145/3366423.3380014

  98. Trivedi P, Maheshwari G, Dubey M, Lehmann J (2017) Lc-quad: a corpus for complex question answering over knowledge graphs. In: International semantic web conference. Springer, Cham, pp 210–218

  99. Vakulenko S, Fernandez Garcia JD, Polleres A, de Rijke M, Cochez M (2019) Message passing for complex question answering over knowledge graphs. In: Proceedings of the 28th ACM international conference on information and knowledge management, pp 1431–1440

  100. Van Pinxteren MM, Pluymaekers M, Lemmink JG (2020) Human-like communication in conversational agents: a literature review and research agenda. J Serv Manag 203–225 ahead-of-print

  101. Vrandečić D, Krötzsch M (2014) Wikidata: a free collaborative knowledgebase. Commun ACM 57(10):78–85

    Article  Google Scholar 

  102. Wang R, Wang M, Liu J, Cochez M, Decker S (2019) Structured query construction via knowledge graph embedding. Knowl Inf Syst 1–28

  103. Wohlin C (2014) Guidelines for snowballing in systematic literature studies and a replication in software engineering. In: Proceedings of the 18th international conference on evaluation and assessment in software engineering, EASE ’14. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/2601248.2601268

  104. Wu L, Wu P, Zhang X (2019) A seq2seq-based approach to question answering over knowledge bases. In: Joint international semantic technology conference. Springer, pp 170–181

  105. Wu P, Zhang X, Feng Z (2019) A survey of question answering over knowledge base. In: China conference on knowledge graph and semantic computing. Springer, Singapore, pp 86–97

  106. Xu K, Lai Y, Feng Y, Wang Z (2019) Enhancing key-value memory neural networks for knowledge based question answering. In: Proceedings of the 2019 conference of the north american chapter of the association for computational linguistics: human language technologies, volume 1 (Long and Short Papers). Association for Computational Linguistics, Minneapolis, Minnesota, pp 2937–2947

  107. Xu K, Reddy S, Feng Y, Huang S, Zhao D (2016) Question answering on freebase via relation extraction and textual evidence. In: ACL, no. 1, 2016. http://aclweb.org/anthology/P/P16/P16-1220.pdf

  108. Yang S, Han F, Wu Y, Yan X (2016) Fast top-k search in knowledge graphs. In: 2016 IEEE 32nd international conference on data engineering (ICDE). IEEE Computer Society, Los Alamitos, CA, USA, pp 990–1001. https://doi.org/10.1109/ICDE.2016.7498307

  109. Yang S, Jiang Y, Han W, Tu K (2020) Second-order unsupervised neural dependency parsing

  110. Yang Y, Chang MW (2015) S-MART: novel tree-based structured learning algorithms applied to tweet entity linking. In: Proceedings of the 53rd annual meeting of the association for computational linguistics and the 7th international joint conference on natural language processing (volume 1: Long Papers). Association for Computational Linguistics, Beijing, China, pp 504–513. https://doi.org/10.3115/v1/P15-1049

  111. Yih Wt, Chang MW, He X, Gao J (2015) Semantic parsing via staged query graph generation: question answering with knowledge base. In: Proceedings of the 53rd annual meeting of the association for computational linguistics and the 7th international joint conference on natural language processing (volume 1: long papers), pp 1321–1331

  112. Yih Wt, Richardson M, Meek C, Chang MW, Suh J (2016) The value of semantic parse labeling for knowledge base question answering. In: Proceedings of the 54th annual meeting of the association for computational linguistics (volume 2: short papers), pp 201–206

  113. Yin P, Duan N, Kao B, Bao J, Zhou M (2015) Answering questions with complex semantic constraints on open knowledge bases. In: Proceedings of the 24th acm international on conference on information and knowledge management, pp 1301–1310

  114. Yin W, Ge W, Wang H (2014) Cdqa: an ontology-based question answering system for Chinese delicacy. In: 2014 IEEE 3rd international conference on cloud computing and intelligence systems. IEEE, Shenzhen, China, pp 1–7. https://doi.org/10.1109/CCIS.2014.7175693

  115. Yu M, Yin W, Hasan KS, dos Santos CN, Xiang B, Zhou B (2017) Improved neural relation detection for knowledge base question answering. In: ACL, no. 1, pp. 571–581. https://doi.org/10.18653/v1/P17-1053

  116. Zafar H, Napolitano G, Lehmann J (2018) Formal query generation for question answering over knowledge bases. In: European semantic web conference. Springer, pp 714–728

  117. Zhang J, Hua Y, Qi G, Qi D (2018) Semantic parsing for multiple-relation Chinese question answering. In: CEUR workshop proceedings, vol 2242, pp 101–106. https://www.scopus.com

  118. Zhang L, Winn J, Tomioka R (2016) Gaussian attention model and its application to knowledge base embedding and question answering. arXiv preprint arXiv:1611.02266

  119. Zhang Q, Tong P, Yao J, Wang X (2020) Mutual relation detection for complex question answering over knowledge graph. In: International conference on database systems for advanced applications. Springer, pp 623–631

  120. Zhang Y, Dai H, Kozareva Z, Smola A, Song L (2018) Variational reasoning for question answering with knowledge graph. In: Proceedings of the AAAI conference on artificial intelligence, vol 32, no. 1, pp 6069–6076. https://ojs.aaai.org/index.php/AAAI/article/view/12057

  121. Zheng C, Kordjamshidi P (2020) SRLGRN: semantic role labeling graph reasoning network. In: Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP). Association for Computational Linguistics, pp 8881–8891, Online

  122. Zheng W, Yu JX, Zou L, Cheng H (2018) Question answering over knowledge graphs: question understanding via template decomposition. Proc VLDB Endow 11(11):1373–1386

  123. Zhou M, Huang M, Zhu X (2018) An interpretable reasoning network for multi-relation question answering. In: Proceedings of the 27th international conference on computational linguistics, pp 2010–2022

Download references

Acknowledgements

The authors thank the financial support provided. This study was financed in part by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior—Brazil (CAPES) Finance Code 001. Also, the authors thank the financial support of the National Education and Research Network (RNP).

Author information

Authors and Affiliations

  1. Department of Information Systems and Operations Management, Vienna University of Economics and Business, Welthandelsplatz 1, 1020, Vienna, Austria

    Jorão Gomes Jr.

  2. LApIC Research Group, Graduate Program in Computer Science, Federal University of Juiz de Fora (UFJF), Juiz de Fora, 36.060-900, Brazil

    Jorão Gomes Jr.

  3. Department of Computer Science, Federal University of Juiz de Fora (UFJF), Juiz de Fora, 36.060-900, Brazil

    Rômulo Chrispim de Mello & Victor Ströele

  4. LApIC Research Group, Department of Computer Science, Federal University of Juiz de Fora (UFJF), Juiz de Fora, 36.060-900, Brazil

    Jairo Francisco de Souza

Authors
  1. Jorão Gomes Jr.
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Rômulo Chrispim de Mello
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Victor Ströele
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Jairo Francisco de Souza
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Jairo Francisco de Souza.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

See Table 12.

Table 12 Full list of papers
Full size table

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gomes, J., de Mello, R.C., Ströele, V. et al. A study of approaches to answering complex questions over knowledge bases. Knowl Inf Syst 64, 2849–2881 (2022). https://doi.org/10.1007/s10115-022-01737-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10115-022-01737-x

Keywords