Skip to main content
SpringerLink
Log in

TemporalFC: A Temporal Fact Checking Approach over Knowledge Graphs

  • Conference paper
  • First Online:
The Semantic Web – ISWC 2023 (ISWC 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14265))

Included in the following conference series:

  • 1319 Accesses

Abstract

Verifying assertions is an essential part of creating and maintaining knowledge graphs. Most often, this task cannot be carried out manually due to the sheer size of modern knowledge graphs. Hence, automatic fact-checking approaches have been proposed over the last decade. These approaches aim to compute automatically whether a given assertion is correct or incorrect. However, most fact-checking approaches are binary classifiers that fail to consider the volatility of some assertions, i.e., the fact that such assertions are only valid at certain times or for specific time intervals. Moreover, the few approaches able to predict when an assertion was valid (i.e., time-point prediction approaches) rely on manual feature engineering. This paper presents TemporalFC, a temporal fact-checking approach that uses multiple sources of background knowledge to assess the veracity and temporal validity of a given assertion. We evaluate TemporalFC on two datasets and compare it to the state of the art in fact-checking and time-point prediction. Our results suggest that TemporalFC outperforms the state of the art on the fact-checking task by 0.13 to 0.15 in terms of Area Under the Receiver Operating Characteristic curve and on the time-point prediction task by 0.25 to 0.27 in terms of Mean Reciprocal Rank. Our code is open-source and can be found at https://github.com/dice-group/TemporalFC.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    http://lodstats.aksw.org/.

  2. 2.

    From here on, we work with IRIs. The prefixes for these IRIs that we use are “xs” and “:”. The xmlns schema is identified by the URI-Reference http://www.w3.org/2001/XMLSchema/# and is associated with the prefix ’xs’. Furthermore, we use “:” prefix for literals.

  3. 3.

    Fair comparison could not be possible with missing entities, which constitute many assertions.

  4. 4.

    https://www.elastic.co/.

  5. 5.

    https://doi.org/10.5281/zenodo.7913193.

  6. 6.

    We report the parameters that were used to achieve the results reported in this study. Nevertheless, the user has the option to modify these parameters to suit her personal preferences. Visit the project home page to get the complete list of parameters.

  7. 7.

    Among all the available pre-training models from the SBert webpage (https://www.sbert.net/docs/pretrained_models.html), we select nq-distilbert-base-v1 for our approach (as suggested in [41]).

  8. 8.

    We ran experiments with other values of k, i.e., 1, 2, 3, and 5 and found that \(k=3\) worked best for our approach. We cannot present comprehensive results in this paper due to space limitations. However, they can be found in our extended, green open-access version of the paper.

  9. 9.

    Our results are also available on the GERBIL benchmarking platform [42]: 1. Using assertion-based negative examples: http://w3id.org/gerbil/kbc/experiment?id=202301180129, http://w3id.org/gerbil/kbc/experiment?id=202301180056, http://w3id.org/gerbil/kbc/experiment?id=202301180123, and http://w3id.org/gerbil/kbc/experiment?id=202301180125. 2. Using time-based negative examples: http://w3id.org/gerbil/kbc/experiment?id=202305020014, http://w3id.org/gerbil/kbc/experiment?id=202305020015, http://w3id.org/gerbil/kbc/experiment?id=202305020012, and http://w3id.org/gerbil/kbc/experiment?id=202305020013.

  10. 10.

    We use a Wilcoxon signed rank test with a significance threshold \(\alpha = 0.05\).

  11. 11.

    Source code: https://github.com/dice-group/TemporalFC.

References

  1. Athreya, R.G., Ngonga Ngomo, A.C., Usbeck, R.: Enhancing community interactions with data-driven chatbots-the dbpedia chatbot. In: Companion Proceedings of World Wide Web, pp. 143–146. International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE (2018). https://doi.org/10.1145/3184558.3186964

  2. Auer, S., Bizer, C., Kobilarov, G., Lehmann, J., Cyganiak, R., Ives, Z.: DBpedia: a nucleus for a web of open data. In: Aberer, K., et al. (eds.) ASWC/ISWC -2007. LNCS, vol. 4825, pp. 722–735. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-76298-0_52

    Chapter  Google Scholar 

  3. Balazevic, I., Allen, C., Hospedales, T.: TuckER: tensor factorization for knowledge graph completion. In: EMNLP-IJCNLP, pp. 5185–5194. Association for Computational Linguistics, Hong Kong (2019). https://doi.org/10.18653/v1/D19-1522. https://aclanthology.org/D19-1522

  4. Bengio, Y., Courville, A., Vincent, P.: Representation learning: a review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1798–1828 (2013)

    Article  Google Scholar 

  5. Bordes, A., Usunier, N., Garcia-Durán, A., Weston, J., Yakhnenko, O.: Translating embeddings for modeling multi-relational data. In: NIPS, pp. 2787–2795. Curran Associates Inc., Red Hook (2013)

    Google Scholar 

  6. Chekol, M.W.: Tensor decomposition for link prediction in temporal knowledge graphs. In: Proceedings of the 11th on Knowledge Capture Conference, pp. 253–256. ACM, New York (2021). https://doi.org/10.1145/3460210.3493558

  7. Chen, Y., Goldberg, S., Wang, D.Z., Johri, S.S.: Ontological pathfinding: mining first-order knowledge from large knowledge bases. In: ICMD, pp. 835–846. ACM, New York (2016). https://doi.org/10.1145/2882903.2882954

  8. Dasgupta, S.S., Ray, S.N., Talukdar, P.: HyTE: hyperplane-based temporally aware knowledge graph embedding. In: EMNLP, pp. 2001–2011. Association for Computational Linguistics, Brussels (2018). https://doi.org/10.18653/v1/D18-1225. https://aclanthology.org/D18-1225

  9. Demir, C., Moussallem, D., Heindorf, S., Ngomo, A.C.N.: Convolutional hypercomplex embeddings for link prediction. In: Asian Conference on Machine Learning, pp. 656–671. PMLR (2021)

    Google Scholar 

  10. Demir, C., Ngomo, A.-C.N.: Convolutional complex knowledge graph embeddings. In: Verborgh, R., et al. (eds.) ESWC 2021. LNCS, vol. 12731, pp. 409–424. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-77385-4_24

    Chapter  Google Scholar 

  11. Dong, X.L., et al.: Knowledge vault: a web-scale approach to probabilistic knowledge fusion. In: SIGKDD, pp. 601–610 (2014). http://www.cs.cmu.edu/nlao/publication/2014.kdd.pdf

  12. Drucker, P.F.: The Age of Discontinuity: Guidelines to Our Changing Society. Transaction Publishers, Piscataway (2011)

    Google Scholar 

  13. Ermilov, I., Lehmann, J., Martin, M., Auer, S.: LODStats: the data web census dataset. In: Groth, P., et al. (eds.) ISWC 2016. LNCS, vol. 9982, pp. 38–46. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46547-0_5

    Chapter  Google Scholar 

  14. Galárraga, L., Teflioudi, C., Hose, K., Suchanek, F.M.: Fast rule mining in ontological knowledge bases with amie+. VLDB J. 24(6), 707–730 (2015). https://doi.org/10.1007/s00778-015-0394-1

    Article  Google Scholar 

  15. Galárraga, L.A., Teflioudi, C., Hose, K., Suchanek, F.: Amie: association rule mining under incomplete evidence in ontological knowledge bases. In: World Wide Web, World Wide Web ’13, pp. 413–422. ACM, New York (2013). https://doi.org/10.1145/2488388.2488425

  16. García-Durán, A., Dumančić, S., Niepert, M.: Learning sequence encoders for temporal knowledge graph completion. In: EMNLP, pp. 4816–4821. Association for Computational Linguistics, Brussels (2018). https://doi.org/10.18653/v1/D18-1516. https://aclanthology.org/D18-1516

  17. Gardner, M., Mitchell, T.: Efficient and expressive knowledge base completion using subgraph feature extraction. In: EMNLP, pp. 1488–1498 (2015)

    Google Scholar 

  18. Gardner, M., Talukdar, P., Krishnamurthy, J., Mitchell, T.: Incorporating vector space similarity in random walk inference over knowledge bases. In: EMNLP, pp. 397–406. Association for Computational Linguistics, Doha (2014). https://doi.org/10.3115/v1/D14-1044

  19. Gerber, D., et al.: Defacto-temporal and multilingual deep fact validation. Web Semant. 35(P2), 85–101 (2015). https://doi.org/10.1016/j.websem.2015.08.001

  20. Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: ICML, p. 448–456. JMLR.org (2015)

    Google Scholar 

  21. Ji, G., He, S., Xu, L., Liu, K., Zhao, J.: Knowledge graph embedding via dynamic mapping matrix. In: IJCNLP, pp. 687–696. Association for Computational Linguistics, Beijing (2015). https://doi.org/10.3115/v1/P15-1067. https://www.aclweb.org/anthology/P15-1067

  22. Jin, W., Zhang, C., Szekely, P.A., Ren, X.: Recurrent event network for reasoning over temporal knowledge graphs. CoRR abs/1904.05530 (2019). http://arxiv.org/abs/1904.05530

  23. Kim, J., Choi, K.s.: Unsupervised fact checking by counter-weighted positive and negative evidential paths in a knowledge graph. In: CICLing, pp. 1677–1686. International Committee on Computational Linguistics, Barcelona (2020). https://doi.org/10.18653/v1/2020.coling-main.147. https://www.aclweb.org/anthology/2020.coling-main.147

  24. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. CoRR arXiv:1412.6980 (2014)

  25. Konstantinovskiy, L., Price, O., Babakar, M., Zubiaga, A.: Toward automated factchecking: developing an annotation schema and benchmark for consistent automated claim detection. Dig. Threats: Res. Pract. 2(2) (2021). https://doi.org/10.1145/3412869

  26. Koster, A., Bazzan, A., Souza, M.d.: Liar liar, pants on fire; or how to use subjective logic and argumentation to evaluate information from untrustworthy sources. Artif. Intell. Rev. 48 (2017). https://doi.org/10.1007/s10462-016-9499-1

  27. Kotonya, N., Toni, F.: Explainable automated fact-checking for public health claims. arXiv preprint arXiv:2010.09926 (2020)

  28. Lacroix, T., Obozinski, G., Usunier, N.: Tensor decompositions for temporal knowledge base completion (2020). https://doi.org/10.48550/ARXIV.2004.04926. https://arxiv.org/abs/2004.04926

  29. Lajus, J., Galárraga, L., Suchanek, F.: Fast and exact rule mining with AMIE 3. In: Harth, A., Kirrane, S., Ngonga Ngomo, A.-C., Paulheim, H., Rula, A., Gentile, A.L., Haase, P., Cochez, M. (eds.) ESWC 2020. LNCS, vol. 12123, pp. 36–52. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-49461-2_3

    Chapter  Google Scholar 

  30. Leblay, J., Chekol, M.W.: Deriving validity time in knowledge graph. In: Companion Proceedings of the The Web Conference 2018, pp. 1771–1776. International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE (2018). https://doi.org/10.1145/3184558.3191639

  31. Lin, Y., Liu, Z., Sun, M., Liu, Y., Zhu, X.: Learning entity and relation embeddings for knowledge graph completion. In: AAAI, vol. 29 (2015)

    Google Scholar 

  32. Long, Y., Lu, Q., Xiang, R., Li, M., Huang, C.R.: Fake news detection through multi-perspective speaker profiles. In: ICNLP, pp. 252–256. Asian Federation of Natural Language Processing, Taipei, Taiwan (2017). https://aclanthology.org/I17-2043

  33. Ma, Y., Tresp, V., Daxberger, E.A.: Embedding models for episodic knowledge graphs. J. Web Semant. 59, 100490 (2019). https://doi.org/10.1016/j.websem.2018.12.008. https://www.sciencedirect.com/science/article/pii/S1570826818300702

  34. Malyshev, S., Krötzsch, M., González, L., Gonsior, J., Bielefeldt, A.: Getting the most out of wikidata: semantic technology usage in wikipedia’s knowledge graph. In: Vrandečić, D., et al. (eds.) ISWC 2018. LNCS, vol. 11137, pp. 376–394. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00668-6_23

    Chapter  Google Scholar 

  35. Nayyeri, M., et al.: Dihedron algebraic embeddings for spatio-temporal knowledge graph completion. In: ESWC, pp. 253–269. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-06981-9_15

  36. Ngonga Ngomo, A.C., Röder, M., Syed, Z.H.: Semantic web challenge 2019. Website (2019). https://github.com/dice-group/semantic-web-challenge.github.io/. Accessed 30 Mar 2022

  37. Ortona, S., Meduri, V.V., Papotti, P.: Rudik: rule discovery in knowledge bases. Proc. VLDB Endow. 11(12), 1946–1949 (2018). https://doi.org/10.14778/3229863.3236231

  38. Pasternack, J., Roth, D.: Knowing what to believe (when you already know something). In: CICLing, pp. 877–885. Association for Computational Linguistics, USA (2010)

    Google Scholar 

  39. Pasternack, J., Roth, D.: Latent credibility analysis. In: World Wide Web, World Wide Web 2013, pp. 1009–1020. ACM, New York (2013). https://doi.org/10.1145/2488388.2488476

  40. Paulheim, H., Ngonga Ngomo, A.C., Bennett, D.: Semantic web challenge 2018. Website (2018). http://iswc2018.semanticweb.org/semantic-web-challenge-2018/index.html. Accessed 30 Mar 2022

  41. Qudus, U., Röder, M., Saleem, M., Ngomo, A.C.N.: Hybridfc: a hybrid fact-checking approach for knowledge graphs. In: International Semantic Web Conference, pp. 462–480. Springer International Publishing, Cham (2022). https://doi.org/10.1007/978-3-031-19433-7_27. https://papers.dice-research.org/2022/ISWC_HybridFC/public.pdf

  42. Röder, M., Usbeck, R., Ngomo, A.N.: GERBIL - benchmarking named entity recognition and linking consistently. Semant. Web 9(5), 605–625 (2018). https://doi.org/10.3233/SW-170286. http://www.semantic-web-journal.net/system/files/swj1671.pdf

  43. Rula, A., et al.: Tisco: temporal scoping of facts. Web Semant. 54(C), 72–86 (2019). https://doi.org/10.1016/j.websem.2018.09.002

  44. Shi, B., Weninger, T.: Discriminative predicate path mining for fact checking in knowledge graphs. Know.-Based Syst. 104(C), 123–133 (2016). https://doi.org/10.1016/j.knosys.2016.04.015

  45. Shiralkar, P., Flammini, A., Menczer, F., Ciampaglia, G.L.: Finding streams in knowledge graphs to support fact checking. In: ICDM, pp. 859–864 (2017). https://doi.org/10.1109/ICDM.2017.105

  46. da Silva, A.A.M., Röder, M., Ngomo, A.-C.N.: Using compositional embeddings for fact checking. In: Hotho, A., et al. (eds.) ISWC 2021. LNCS, vol. 12922, pp. 270–286. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-88361-4_16

    Chapter  Google Scholar 

  47. Strubell, E., Ganesh, A., McCallum, A.: Energy and policy considerations for deep learning in NLP. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 3645–3650. Association for Computational Linguistics, Florence (2019). https://doi.org/10.18653/v1/P19-1355. https://aclanthology.org/P19-1355

  48. Suchanek, F.M., Kasneci, G., Weikum, G.: Yago: a core of semantic knowledge. In: World Wide Web, pp. 697–706. ACM (2007)

    Google Scholar 

  49. Syed, Z.H., Röder, M., Ngomo, A.-C.N.: Unsupervised discovery of corroborative paths for fact validation. In: Ghidini, C., et al. (eds.) ISWC 2019. LNCS, vol. 11778, pp. 630–646. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-30793-6_36

    Chapter  Google Scholar 

  50. Syed, Z.H., Röder, M., Ngonga Ngomo, A.C.: Factcheck: Validating RDF triples using textual evidence. In: CIKM, CIKM 2018, pp. 1599–1602. ACM, New York (2018). https://doi.org/10.1145/3269206.3269308

  51. Syed, Z.H., Srivastava, N., Röder, M., Ngomo, A.C.N.: Copaal - an interface for explaining facts using corroborative paths. In: International Semantic Web Conference (2019)

    Google Scholar 

  52. Syed, Z.H., Srivastava, N., Röder, M., Ngomo, A.N.: COPAAL - an interface for explaining facts using corroborative paths. In: International Semantic Web Conference, vol. 2456, pp. 201–204. CEUR-WS.org (2019). http://ceur-ws.org/Vol-2456/paper52.pdf

  53. Trouillon, T., Welbl, J., Riedel, S., Gaussier, E., Bouchard, G.: Complex embeddings for simple link prediction. In: ICML, pp. 2071–2080 (2016)

    Google Scholar 

  54. Wang, Q., Mao, Z., Wang, B., Guo, L.: Knowledge graph embedding: a survey of approaches and applications. IEEE Trans. Knowl. Data Eng. 29(12), 2724–2743 (2017). https://doi.org/10.1109/TKDE.2017.2754499

    Article  Google Scholar 

  55. Watt, N., du Plessis, M.C.: Dropout algorithms for recurrent neural networks. In: SAICSIT, pp. 72–78. ACM, New York (2018). https://doi.org/10.1145/3278681.3278691

  56. Xu, C., Nayyeri, M., Alkhoury, F., Shariat Yazdi, H., Lehmann, J.: TeRo: a time-aware knowledge graph embedding via temporal rotation. In: CICLing, pp. 1583–1593. International Committee on Computational Linguistics, Barcelona (2020). https://doi.org/10.18653/v1/2020.coling-main.139. https://aclanthology.org/2020.coling-main.139

  57. Yang, B., Yih, W., He, X., Gao, J., Deng, L.: Embedding entities and relations for learning and inference in knowledge bases. In: ICLR (2015). http://arxiv.org/abs/1412.6575

Download references

Acknowledgments

This work is part of a project that has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 860801, the German Federal Ministry of Education and Research (BMBF) within the project NEBULA under the grant no 13N16364, the Ministry of Culture and Science of North Rhine-Westphalia (MKW NRW) within the project SAIL under the grant no NW21-059D, and the European Union’s Horizon Europe research and innovation programme under grant agreement No 101070305. This work is also funded by the FWF Austrian Science Fund and the Internet Foundation Austria under the FWF Elise Richter and netidee SCIENCE programmes as project number V 759-N.

Author information

Authors and Affiliations

  1. DICE Group, Department of Computer Science, Universität Paderborn, Paderborn, Germany

    Umair Qudus, Michael Röder & Axel-Cyrille Ngonga Ngomo

  2. Institute for Information Systems and New Media, Vienna University of Economics and Business, Vienna, Austria

    Sabrina Kirrane

Authors
  1. Umair Qudus
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michael Röder
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sabrina Kirrane
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Axel-Cyrille Ngonga Ngomo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Umair Qudus .

Editor information

Editors and Affiliations

  1. University of Liverpool, Liverpool, UK

    Terry R. Payne

  2. University of Bologna, Bologna, Italy

    Valentina Presutti

  3. Southeast University, Nanjing, China

    Guilin Qi

  4. Universidad Politécnica de Madrid, Madrid, Spain

    María Poveda-Villalón

  5. Huawei Technologies R&D UK, Edinburgh, UK

    Giorgos Stoilos

  6. Centrum Wiskunde and Informatica, Amsterdam, The Netherlands

    Laura Hollink

  7. IT University of Copenhagen, Copenhagen, Denmark

    Zoi Kaoudi

  8. Nanjing University, Nanjing, China

    Gong Cheng

  9. Tsinghua University, Beijing, Beijing, China

    Juanzi Li

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Qudus, U., Röder, M., Kirrane, S., Ngomo, AC.N. (2023). TemporalFC: A Temporal Fact Checking Approach over Knowledge Graphs. In: Payne, T.R., et al. The Semantic Web – ISWC 2023. ISWC 2023. Lecture Notes in Computer Science, vol 14265. Springer, Cham. https://doi.org/10.1007/978-3-031-47240-4_25

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-47240-4_25

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-47239-8

  • Online ISBN: 978-3-031-47240-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation