Abstract
Numerous prior works have shown how we can use knowledge graph embedding (KGE) models for ranking unseen facts that are likely to be true. Though these KGE models have been shown to make good performance on the ranking task with standard benchmark datasets, in practice only a subset of the top-k ranked list is indeed correct. This is due to the fact that most knowledge graphs are built under the open world assumption, while state-of-the-art KGE exploit the closed world assumption to build negative samples for training. In this paper, we show to address this problem by ensembling calibrated learning, following the principle that multiple calibrated models can make a stronger one. In experiments on the ConceptNet of commonsense knowledge base, we show significant improvement over each individual baseline model. This suggests that ensembling calibrated learning is a promising technique to improve link prediction with KGEs.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
- 2.
https://scikit-learn.org/stable/modules/calibration.html#calibration explains possible concrete implementation of the calibration.
- 3.
- 4.
References
Ali, M., et al.: PyKEEN 1.0: a Python library for training and evaluating knowledge graph embeddings. J. Mach. Learn. Res. 22(82), 1–6 (2021)
Baader, F., Calvanese, D., McGuinness, D., Patel-Schneider, P., Nardi, D.: The Description Logic Handbook: Theory, Implementation and Applications. Cambridge University Press, Cambridge (2003)
Bollacker, K., Evans, C., Paritosh, P., Sturge, T., Taylor, J.: Freebase: a collaboratively created graph database for structuring human knowledge. In: Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data, pp. 1247–1250 (2008)
Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., Yakhnenko, O.: Translating embeddings for modeling multi-relational data. In: Advances in Neural Information Processing Systems, vol. 26 (2013)
Cambria, E., Olsher, D., Rajagopal, D.: SenticNet 3: a common and common-sense knowledge base for cognition-driven sentiment analysis. In: Twenty-Eighth AAAI Conference on Artificial Intelligence (2014)
Dettmers, T., Minervini, P., Stenetorp, P., Riedel, S.: Convolutional 2D knowledge graph embeddings. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32 (2018)
Joshi, U., Urbani, J.: Ensemble-based fact classification with knowledge graph embeddings. In: Groth, P., et al. (eds.) The Semantic Web: 19th International Conference, ESWC 2022, Hersonissos, Crete, Greece, 29 May–2 June 2022, Proceedings, pp. 147–164. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-06981-9_9
Krompaß, D., Tresp, V.: Ensemble solutions for link-prediction in knowledge graphs. In: PKDD ECML 2nd Workshop on Linked Data for Knowledge Discovery (2015)
Kull, M., Silva Filho, T.M., Flach, P.: Beyond sigmoids: how to obtain well-calibrated probabilities from binary classifiers with beta calibration (2017)
Lehmann, J., et al.: DBpedia-a large-scale, multilingual knowledge base extracted from Wikipedia. Semant. Web 6(2), 167–195 (2015)
Lenat, D.B.: CYC: a large-scale investment in knowledge infrastructure. Commun. ACM 38(11), 33–38 (1995)
Li, X., Taheri, A., Tu, L., Gimpel, K.: Commonsense knowledge base completion. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1445–1455 (2016)
Lin, X., Quan, Z., Wang, Z.J., Ma, T., Zeng, X.: KGNN: knowledge graph neural network for drug-drug interaction prediction. In: IJCAI, vol. 380, pp. 2739–2745 (2020)
Miller, G.A.: WordNet: a lexical database for English. Commun. ACM 38(11), 39–41 (1995)
Nickel, M., Murphy, K., Tresp, V., Gabrilovich, E.: A review of relational machine learning for knowledge graphs. Proc. IEEE 104(1), 11–33 (2015)
Nickel, M., Tresp, V., Kriegel, H.P.: A three-way model for collective learning on multi-relational data. In: ICML (2011)
Niculescu-Mizil, A., Caruana, R.: Predicting good probabilities with supervised learning. In: Proceedings of the 22nd International Conference on Machine Learning, pp. 625–632 (2005)
Noy, N., Gao, Y., Jain, A., Narayanan, A., Patterson, A., Taylor, J.: Industry-scale knowledge graphs: lessons and challenges: five diverse technology companies show how it’s done. Queue 17(2), 48–75 (2019)
Reiter, R.: On closed world data bases. In: Readings in Artificial Intelligence, pp. 119–140. Elsevier (1981)
Rivas-Barragan, D., Domingo-Fernández, D., Gadiya, Y., Healey, D.: Ensembles of knowledge graph embedding models improve predictions for drug discovery. Briefings Bioinf. 23(6), bbac481 (2022)
Shi, B., Weninger, T.: Open-world knowledge graph completion. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32 (2018)
Singh, P., Lin, T., Mueller, E.T., Lim, G., Perkins, T., Li Zhu, W.: Open mind common sense: knowledge acquisition from the general public. In: Meersman, R., Tari, Z. (eds.) OTM 2002. LNCS, vol. 2519, pp. 1223–1237. Springer, Heidelberg (2002). https://doi.org/10.1007/3-540-36124-3_77
Speer, R., Chin, J., Havasi, C.: ConceptNet 5.5: an open multilingual graph of general knowledge. In: Thirty-First AAAI Conference on Artificial Intelligence (2017)
Suchanek, F.M., Kasneci, G., Weikum, G.: Yago: a core of semantic knowledge. In: Proceedings of the 16th International Conference on World Wide Web, pp. 697–706 (2007)
Tabacof, P., Costabello, L.: Probability calibration for knowledge graph embedding models. arXiv preprint arXiv:1912.10000 (2019)
Tandon, N., De Melo, G., Suchanek, F., Weikum, G.: WebChild: harvesting and organizing commonsense knowledge from the web. In: Proceedings of the 7th ACM International Conference on Web Search and Data Mining, pp. 523–532 (2014)
Trouillon, T., Welbl, J., Riedel, S., Gaussier, É., Bouchard, G.: Complex embeddings for simple link prediction. In: International Conference on Machine Learning, pp. 2071–2080. PMLR (2016)
Wan, G., Du, B., Pan, S., Wu, J.: Adaptive knowledge subgraph ensemble for robust and trustworthy knowledge graph completion. World Wide Web 23, 471–490 (2020)
Wang, Q., Mao, Z., Wang, B., Guo, L.: Knowledge graph embedding: a survey of approaches and applications. IEEE Trans. Knowl. Data Eng. 29(12), 2724–2743 (2017)
Wang, Z., Zhang, J., Feng, J., Chen, Z.: Knowledge graph embedding by translating on hyperplanes. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 28 (2014)
Wang, Z., Chen, T., Ren, J., Yu, W., Cheng, H., Lin, L.: Deep reasoning with knowledge graph for social relationship understanding. arXiv preprint arXiv:1807.00504 (2018)
Xiong, C., Power, R., Callan, J.: Explicit semantic ranking for academic search via knowledge graph embedding. In: Proceedings of the 26th International Conference on World Wide Web, pp. 1271–1279 (2017)
Yang, B., Yih, W.T., He, X., Gao, J., Deng, L.: Embedding entities and relations for learning and inference in knowledge bases. arXiv preprint arXiv:1412.6575 (2014)
Zang, L.J., Cao, C., Cao, Y.N., Wu, Y.M., Cun-Gen, C.: A survey of commonsense knowledge acquisition. J. Comput. Sci. Technol. 28(4), 689–719 (2013)
Zhou, Z.H.: Ensemble learning. In: Li, S.Z., Jain, A. (eds.) Encyclopedia of Biometrics. Springer, Boston (2021). https://doi.org/10.1007/978-0-387-73003-5_293
Acknowledgement
This work was supported by JSPS KAKENHI Grant Numbers JP22K18004.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
A Appendix: Preliminary Study on ConceptNet-100K
A Appendix: Preliminary Study on ConceptNet-100K
Table 4 shows our preliminary study with ConceptNet-100K. This table together with the results shown on the top part of Table 2 have inspired us to investigate an application of probability calibration on ensembling of KGEs.
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Racharak, T., Jearanaiwongkul, W., Thwe, K.M. (2023). Can Ensemble Calibrated Learning Enhance Link Prediction? A Study on Commonsense Knowledge. In: Nguyen, N.T., et al. Intelligent Information and Database Systems. ACIIDS 2023. Lecture Notes in Computer Science(), vol 13996. Springer, Singapore. https://doi.org/10.1007/978-981-99-5837-5_16
Download citation
DOI: https://doi.org/10.1007/978-981-99-5837-5_16
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-5836-8
Online ISBN: 978-981-99-5837-5
eBook Packages: Computer ScienceComputer Science (R0)