Abstract
The task of factoid question answering (QA) faces new challenges when applied in scenarios with rapidly changing context information, for example on smartphones. Instead of asking who the architect of the “Holocaust Memorial” in Berlin was, the same question could be phrased as “Who was the architect of the many stelae in front of me?” presuming the user is standing in front of it. While traditional QA systems rely on static information from knowledge bases and the analysis of named entities and predicates in the input, question answering for temporal and spatial questions imposes new challenges to the underlying methods. To tackle these challenges, we present the Context-aware Spatial QA Dataset (CASQAD) with over 5,000 annotated questions containing visual and spatial references that require information about the user’s location and moving direction to compose a suitable query. These questions were collected in a large scale user study and annotated semi-automatically, with appropriate measures to ensure the quality.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
He designed the Memorial to the Murdered Jews of Europe https://www.visitberlin.de/en/memorial-murdered-jews-europe.
- 2.
A visibility engine computes, which objects are visible from the user’s point of view.
- 3.
- 4.
A HIT describes the micro tasks a requester posts to the workers on Amazon’s platform, also known as a “project”.
- 5.
Using state-of-the-art models from https://spacy.io/.
- 6.
- 7.
- 8.
- 9.
- 10.
All meta information is provided by Google’s Street View API https://developers.google.com/maps/documentation/streetview/.
- 11.
We removed manually questions such as “Who am I?”.
- 12.
- 13.
- 14.
- 15.
Even though we instructed the turkers to phrase only one question per input frame, not all followed the instruction.
- 16.
References
Auer, S., et al.: DBpedia: a nucleus for a web of open data. In: Aberer, K., et al. (eds.) ASWC/ISWC -2007. LNCS, vol. 4825, pp. 722–735. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-76298-0_52
Bao, J., Duan, N., Yan, Z., Zhou, M., Zhao, T.: Constraint-based question answering with knowledge graph. In: Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pp. 2503–2514. The COLING 2016 Organizing Committee, Osaka, Japan (2016)
Berant, J., Chou, A., Frostig, R., Liang, P.: semantic parsing on freebase from question-answer pairs. In: Proceedings of EMNLP (October), pp. 1533–1544 (2013)
Bollacker, K., Cook, R., Tufts, P.: Freebase: a shared database of structured general human knowledge. In: Proceedings of the National Conference on Artificial Intelligence, vol. 22, no 2, p. 1962 (2007)
Bordes, A., Chopra, S., Weston, J.: Question answering with subgraph embeddings. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), 25–29 October, pp. 615–620 (2014). https://doi.org/10.3115/v1/D14-1067
Bordes, A., Usunier, N., Chopra, S., Weston, J.: Large-scale simple question answering with memory networks (2015). https://doi.org/10.1016/j.geomphys.2016.04.013
Buhrmester, M.D., Talaifar, S., Gosling, S.D.: An evaluation of amazon’s mechanical turk, its rapid rise, and its effective use. Perspect. Psychol. Sci. 13(2), 149–154 (2018). https://doi.org/10.1177/1745691617706516
Bulcaen, C.: Rethinking context: language as an interactive phenomenon. Lang. Lit. 4(1), 61–64 (1995). https://doi.org/10.1177/096394709500400105
Cai, Q., Yates, A.: Large-scale semantic parsing via schema matching and lexicon extension. In: Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 423–433. Association for Computational Linguistics, Sofia, Bulgaria, August 2013
Cheung, J.H., Burns, D.K., Sinclair, R.R., Sliter, M.: Amazon mechanical turk in organizational psychology: an evaluation and practical recommendations. J. Bus. Psychol. 32(4), 347–361 (2016). https://doi.org/10.1007/s10869-016-9458-5
Dhingra, B., Danish, D., Rajagopal, D.: Simple and effective semi-supervised question answering. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pp. 582–587. Association for Computational Linguistics, Stroudsburg, PA, USA (2018). https://doi.org/10.18653/v1/N18-2092
Diefenbach, D., Lopez, V., Singh, K., Maret, P.: Core techniques of question answering systems over knowledge bases: a survey. Knowl. Inf. Syst. 55(3), 529–569 (2017). https://doi.org/10.1007/s10115-017-1100-y
Diefenbach, D., Tanon, T.P., Singh, K., Maret, P.: Question answering benchmarks for Wikidata. In: CEUR Workshop Proceedings, vol. 1963, pp. 3–6 (2017)
Dubey, M., Banerjee, D., Abdelkawi, A., Lehmann, J.: LC-QuAD 2.0: a large dataset for complex question answering over Wikidata and DBpedia. In: Ghidini, C., et al. (eds.) ISWC 2019. LNCS, vol. 11779, pp. 69–78. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-30796-7_5
Dunn, M., Sagun, L., Higgins, M., Guney, V.U., Cirik, V., Cho, K.: SearchQA: a new q&a dataset augmented with context from a search engine (2017)
Hara, K., Le, V., Froehlich, J.: Combining crowdsourcing and google street view to identify street-level accessibility problems. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems - CHI ’2013, p. 631 (2013). https://doi.org/10.1145/2470654.2470744
Höffner, K., Walter, S., Marx, E., Usbeck, R., Lehmann, J., Ngonga Ngomo, A.C.: Survey on challenges of question answering in the semantic web. Semant. Web 8(6), 895–920 (2017). https://doi.org/10.3233/SW-160247
Janarthanam, S., et al.: Evaluating a city exploration dialogue system combining question-answering and pedestrian navigation. In: 51st Annual Meeting of the Association of Computational Linguistics (October 2015), pp. 1660–1668 (2013). https://doi.org/10.18411/a-2017-023
Janarthanam, S., et al.: Integrating location, visibility, and question-answering in a spoken dialogue system for pedestrian city exploration. In: Proceedings of the 13th Annual Meeting of the Special Interest Group on Discourse and Dialogue (July), pp. 134–136 (2012)
Jia, Z., Abujabal, A., Roy, R.S., Strötgen, J., Weikum, G.: TempQuestions: a benchmark for temporal question answering. In: WWW (Companion Volume), vol. 2, pp. 1057–1062 (2018)
Kočiský, T., et al.: The narrativeQA reading comprehension challenge. Trans. Assoc. Comput. Linguist. 6, 317–328 (2018). https://doi.org/10.1162/tacl_a_00023
Liu, Y., Alexandrova, T., Nakajima, T.: Using stranger [sic] as sensors: temporal and geo-sensitive question answering via social media. In: WWW ’13: Proceedings of the 22nd international conference on World Wide Web, pp. 803–813 (2013). https://doi.org/10.1145/2488388.2488458
Petrochuk, M., Zettlemoyer, L.: SimpleQuestions nearly solved: a new upperbound and baseline approach. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 554–558. Association for Computational Linguistics, Stroudsburg, PA, USA (2018). https://doi.org/10.18653/v1/D18-1051
Rajpurkar, P., Jia, R., Liang, P.: Know what you don’t know: unanswerable questions for SQuAD. In: ACL 2018–56th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Long Papers) 2, pp. 784–789 (2018)
Rajpurkar, P., Zhang, J., Lopyrev, K., Liang, P.: SQuAD: 100,000+ questions for machine comprehension of text. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (ii), pp. 2383–2392 (2016). https://doi.org/10.18653/v1/D16-1264
Salmen, J., Houben, S., Schlipsing, M.: Google street view images support the development of vision-based driver assistance systems. In: Proceedings of the IEEE Intelligent Vehicles Symposium (June 2012), pp. 891–895 (2012). https://doi.org/10.1109/IVS.2012.6232195
Suchanek, F.M., Kasneci, G., Weikum, G.: Yago: a core of semantic knowledge. In: Proceedings of the 16th International Conference on World Wide Web (2007). https://doi.org/10.1145/1242572.1242667
Trivedi, P., Maheshwari, G., Dubey, M., Lehmann, J.: LC-QuAD: a corpus for complex question answering over knowledge graphs. In: d’Amato, C., et al. (eds.) ISWC 2017. LNCS, vol. 10588, pp. 210–218. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-68204-4_22
Unger, C., et al.: Question answering over linked data (QALD-4). In: CEUR Workshop Proceedings, vol. 1180, pp. 1172–1180 (2014)
Unger, C., et al.: Question answering over linked data (QALD-5). In: CLEF, vol. 1180, pp. 1172–1180 (2015)
Usbeck, R., Ngomo, A.C.N., Haarmann, B., Krithara, A., Röder, M., Napolitano, G.: 7th open challenge on question answering over linked data (QALD-7). Commun. Comput. Inf. Sci. 769, 59–69 (2017). https://doi.org/10.1007/978-3-319-69146-6_6
Usbeck, R., et al.: Benchmarking question answering systems. Semant. Web 1, 1–5 (2016)
Yang, M.C., Duan, N., Zhou, M., Rim, H.C.: Joint relational embeddings for knowledge-based question answering. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 645–650 (2014). https://doi.org/10.3115/v1/D14-1071
Yang, Z., Hu, J., Salakhutdinov, R., Cohen, W.: Semi-supervised QA with generative domain-adaptive nets. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1040–1050. Association for Computational Linguistics, Stroudsburg, PA, USA (2017). https://doi.org/10.18653/v1/P17-1096
Yang, Z., et al.: HotpotQA: a dataset for diverse, explainable multi-hop question answering. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 2369–2380. Association for Computational Linguistics, Stroudsburg, PA, USA (2019). https://doi.org/10.18653/v1/D18-1259
Yatskar, M.: A qualitative comparison of CoQA, SQuAD 2.0 and QuAC. NAACL-HLT, September 2018
Yih, W.T., Chang, M.W., He, X., Gao, J.: Semantic parsing via staged query graph generation: question answering with knowledge base. In: ACL, pp. 1321–1331 (2015). https://doi.org/10.3115/v1/P15-1128
Yin, Z., Goldberg, D.W., Zhang, C., Prasad, S.: An NLP-based question answering framework for spatio-temporal analysis and visualization. In: ACM International Conference Proceeding Series Part, vol. F1482, pp. 61–65 (2019). https://doi.org/10.1145/3318236.3318240
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Rose, J., Lehmann, J. (2020). CASQAD – A New Dataset for Context-Aware Spatial Question Answering. In: Pan, J.Z., et al. The Semantic Web – ISWC 2020. ISWC 2020. Lecture Notes in Computer Science(), vol 12507. Springer, Cham. https://doi.org/10.1007/978-3-030-62466-8_1
Download citation
DOI: https://doi.org/10.1007/978-3-030-62466-8_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-62465-1
Online ISBN: 978-3-030-62466-8
eBook Packages: Computer ScienceComputer Science (R0)