Skip to main content
SpringerLink
Log in
Book cover

European Semantic Web Conference

ESWC 2023: The Semantic Web pp 105–121Cite as

Transformer Based Semantic Relation Typing for Knowledge Graph Integration

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13870))

Abstract

More and more knowledge graphs (KGs) are generated in various domains. Applications using more than one KG require an integrated view of those KGs, which, in the first place, requires a common schema or ontology. Merging schemas requires not only equivalence mappings between classes but also other semantic relations, like subclass, superclass, etc. In this paper, we introduce TaSeR, a Transformer based model for Semantic Relation Typing, which is able to decide which type of relation holds between two given classes. The approach can differentiate between equivalent class, sub-/superclass, part of/has part, cohyponym, and no relation at all. With the latter outcome, it is not only possible to refine given class alignments, but also filter incorrect correspondences. The models are trained based on examples from general knowledge graphs as well as fine-tuned on the test case at hand. The former models can be directly used to predict a relation without further training. We show that those models are able to outperform other approaches which solve a similar task. For the evaluation, a new measure is introduced which credits for proximal matches.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    https://www.w3.org/2001/sw/BestPractices/OEP/SimplePartWhole/.

  2. 2.

    As stated in https://wordnet.princeton.edu/documentation/wn1wn.

  3. 3.

    http://mappings.dbpedia.org.

  4. 4.

    http://mappings.dbpedia.org/index.php/Mapping_en:Infobox_aircraft_type.

  5. 5.

    https://dbpedia.org/sparql.

  6. 6.

    The URI fragment is extracted by using the text after the last slash or hashtag.

  7. 7.

    https://en.wikipedia.org/wiki/Camel_case.

  8. 8.

    https://schema.org/version/latest/schemaorg-current-http-types.csv.

  9. 9.

    https://query.wikidata.org.

  10. 10.

    https://wikidata.demo.openlinksw.com/sparql.

  11. 11.

    The number of trained models is fixed to ten and the following hyperparameters are tuned: learning rate (loguniform between 1e-6 and 1e-4), train epochs (between 1 to 10), seed (uniform distribution from 1 to 40), batch size (choice of 4, 8, 16, 32, 64, 128 until the maximum possible batch size). The mutations of HPs are defined by: weight decay (uniform between 0.0 and 0.3), learning rate (uniform between 1e-5 and 5e-5), batch size (choice of 4, 8, 16, 32, 64, 128 until the maximum possible batch size).

  12. 12.

    https://huggingface.co/dwsunimannheim/TaSeR.

References

  1. Arnold, P., Rahm, E.: Enriching ontology mappings with semantic relations. Data Knowl. Eng. 93, 1–18 (2014). https://doi.org/10.1016/j.datak.2014.07.001

    Article  Google Scholar 

  2. Auer, S., Bizer, C., Kobilarov, G., Lehmann, J., Cyganiak, R., Ives, Z.: DBpedia: A Nucleus for a Web of Open Data. In: Aberer, K., et al. (eds.) ASWC/ISWC -2007. LNCS, vol. 4825, pp. 722–735. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-76298-0_52

    Chapter  Google Scholar 

  3. Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., Yakhnenko, O.: Translating embeddings for modeling multi-relational data. Adv. Neural Inf. Process. Syst. 26, 2787–2795 (2013)

    Google Scholar 

  4. Chen, J., He, Y., Geng, Y., Jimenez-Ruiz, E., Dong, H., Horrocks, I.: Contextual semantic embeddings for ontology subsumption prediction. arXiv preprint arXiv:2202.09791 (2022)

  5. Chu, C.X., Razniewski, S., Weikum, G.: Tifi: taxonomy induction for fictional domains. In: The World Wide Web Conference (WWW), pp. 2673–2679 (2019). https://doi.org/10.1145/3308558.3313519

  6. Devlin, J., Chang, M., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Burstein, J., Doran, C., Solorio, T. (eds.) NAACL-HLT, pp. 4171–4186. Association for Computational Linguistics (2019). https://doi.org/10.18653/v1/n19-1423

  7. Galárraga, L., Teflioudi, C., Hose, K., Suchanek, F.M.: Fast rule mining in ontological knowledge bases with AMIE+. VLDB J. 24(6), 707–730 (2015)

    Article  Google Scholar 

  8. Glavaš, G., Ponzetto, S.P.: Dual tensor model for detecting asymmetric lexico-semantic relations. In: Conference on Empirical Methods in Natural Language Processing, EMNLP. Association for Computational Linguistics (2017)

    Google Scholar 

  9. Guha, R.V., Brickley, D., Macbeth, S.: Schema. org: evolution of structured data on the web. Commun. ACM 59(2), 44–51 (2016)

    Google Scholar 

  10. Hertling, S.: TaSeR, March 2023. https://doi.org/10.6084/m9.figshare.21750338.v1, https://figshare.com/articles/dataset/TaSeR/21750338

  11. Hertling, S., Paulheim, H.: Webisalod: providing hypernymy relations extracted from the web as linked open data. In: ISWC, pp. 111–119 (2017)

    Google Scholar 

  12. Hertling, S., Paulheim, H.: Dbkwik: extracting and integrating knowledge from thousands of wikis. Knowl. Inf. Syst. 62(6), 2169–2190 (2020)

    Article  Google Scholar 

  13. Hertling, S., Paulheim, H.: The knowledge graph track at OAEI. In: Harth, A., et al. (eds.) ESWC 2020. LNCS, vol. 12123, pp. 343–359. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-49461-2_20

    Chapter  Google Scholar 

  14. Jaderberg, M., et al.: Population based training of neural networks. arXiv preprint arXiv:1711.09846 (2017)

  15. Jiménez-Ruiz, E., Grau, B.C., Zhou, Y., Horrocks, I.: Large-scale interactive ontology matching: algorithms and implementation. In: ECAI, vol. 242, pp. 444–449 (2012)

    Google Scholar 

  16. Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., Soricut, R.: Albert: a lite bert for self-supervised learning of language representations. arXiv preprint arXiv:1909.11942 (2019)

  17. Liu, Y., et al.: Roberta: a robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 (2019)

  18. Meilicke, C., Chekol, M.W., Ruffinelli, D., Stuckenschmidt, H.: Anytime bottom-up rule learning for knowledge graph completion. In: International Joint Conference on Artificial Intelligence (IJCAI), pp. 3137–3143 (2019). https://doi.org/10.24963/ijcai.2019/435

  19. Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. Adv. Neural Inf. Process. Syst. 26, 3111–3119 (2013)

    Google Scholar 

  20. Miller, G.A.: Wordnet: a lexical database for English. Commun. ACM 38(11), 39–41 (1995)

    Article  Google Scholar 

  21. Navigli, R., Ponzetto, S.P.: Babelnet: the automatic construction, evaluation and application of a wide-coverage multilingual semantic network. Artif. Intell. 193, 217–250 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  22. Nickel, M., Tresp, V., Kriegel, H.P.: A three-way model for collective learning on multi-relational data. In: ICML (2011)

    Google Scholar 

  23. Paulheim, H.: Knowledge graph refinement: a survey of approaches and evaluation methods. Semant. Web 8(3), 489–508 (2017)

    Article  Google Scholar 

  24. Pellissier Tanon, T., Weikum, G., Suchanek, F.: YAGO 4: a reason-able knowledge base. In: Harth, A., et al. (eds.) ESWC 2020. LNCS, vol. 12123, pp. 583–596. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-49461-2_34

    Chapter  Google Scholar 

  25. Pour, M., et al.: Results of the ontology alignment evaluation initiative 2021. In: CEUR Workshop Proceedings 2021, vol. 3063, pp. 62–108. CEUR (2021)

    Google Scholar 

  26. Santus, E., Gladkova, A., Evert, S., Lenci, A.: The CogALex-V shared task on the corpus-based identification of semantic relations. In: Proceedings of the 5th Workshop on Cognitive Aspects of the Lexicon (CogALex-V), pp. 69–79 (2016)

    Google Scholar 

  27. Sérasset, G.: DBnary: wiktionary as a lemon-based multilingual lexical resource in RDF. Semant. Web 6(4), 355–361 (2015)

    Article  Google Scholar 

  28. Suchanek, F.M., Kasneci, G., Weikum, G.: Yago: a core of semantic knowledge. In: Proceedings of the 16th International Conference on World Wide Web, pp. 697–706 (2007)

    Google Scholar 

  29. Trouillon, T., Welbl, J., Riedel, S., Gaussier, É., Bouchard, G.: Complex embeddings for simple link prediction. In: International Conference on Machine Learning, pp. 2071–2080. PMLR (2016)

    Google Scholar 

  30. Vossen, P.: Eurowordnet: a multilingual database for information retrieval. In: Proceedings of the DELOS workshop on Cross-language Information Retrieval, 5–7 March 1997, Zurich. Vrije Universiteit (1997)

    Google Scholar 

  31. Vrandečić, D., Krötzsch, M.: Wikidata: a free collaborative knowledgebase. Commun. ACM 57(10), 78–85 (2014)

    Article  Google Scholar 

  32. Wang, C., Qiu, M., Huang, J., He, X.: KEML: a knowledge-enriched meta-learning framework for lexical relation classification. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 13924–13932 (2021)

    Google Scholar 

  33. Wolf, T., et al.: Transformers: state-of-the-art natural language processing. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp. 38–45. Association for Computational Linguistics, Online, October 2020. https://www.aclweb.org/anthology/2020.emnlp-demos.6

Download references

Author information

Authors and Affiliations

  1. Data and Web Science Group, University of Mannheim, Mannheim, Germany

    Sven Hertling & Heiko Paulheim

Authors
  1. Sven Hertling
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Heiko Paulheim
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sven Hertling .

Editor information

Editors and Affiliations

  1. Universidade de Lisboa, Lisbon, Portugal

    Catia Pesquita

  2. University of London, London, UK

    Ernesto Jimenez-Ruiz

  3. Rensselaer Polytechnic Institute, Troy, MI, USA

    Jamie McCusker

  4. Universidade de Lisboa, Lisbon, Portugal

    Daniel Faria

  5. Fondazione Bruno Kessler, Povo, Trento, Italy

    Mauro Dragoni

  6. KU Leuven, Sint-Katelijne-Waver, Belgium

    Anastasia Dimou

  7. EURECOM, Biot, France

    Raphael Troncy

  8. University of Mannheim, Mannheim, Germany

    Sven Hertling

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Hertling, S., Paulheim, H. (2023). Transformer Based Semantic Relation Typing for Knowledge Graph Integration. In: Pesquita, C., et al. The Semantic Web. ESWC 2023. Lecture Notes in Computer Science, vol 13870. Springer, Cham. https://doi.org/10.1007/978-3-031-33455-9_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-33455-9_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-33454-2

  • Online ISBN: 978-3-031-33455-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation