Skip to main content
SpringerLink
Log in

ASLEEP: A Shallow neural modEl for knowlEdge graph comPletion

  • Conference paper
  • First Online:
Neural Information Processing (ICONIP 2022)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1792))

Included in the following conference series:

  • 612 Accesses

Abstract

Knowledge graph completion aims to predict missing relations between entities in a knowledge graph. One of the effective ways for knowledge graph completion is knowledge graph embedding. However, existing embedding methods usually focus on combined models, variant deep neural networks, or additional information, which inevitably increase computational complexity and are unfriendly to real-time applications. In this paper, we take a step back and propose a novel shallow neural network model for knowledge graph completion. Specifically, given an entity pair, our model first extracts features of head and tail entities through linear transformations. Then entity features are integrated into an entity-pair representation via a max operation followed by a non-linear transformation. Finally, according to the entity-pair representation, our model calculates probability of each relation through multi-label modeling to predict relations for the given entity pair. Experimental results over two widely used datasets show that our model outperforms the baseline methods. The source code of this paper can be obtained from https://github.com/Joni-gogogo/KBC-ASLEEP.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Auer, S., Bizer, C., Kobilarov, G., Lehmann, J., Cyganiak, R., Ives, Z.: DBpedia: a nucleus for a web of open data. In: Aberer, K., et al. (eds.) ASWC/ISWC -2007. LNCS, vol. 4825, pp. 722–735. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-76298-0_52

    Chapter  Google Scholar 

  2. Ba, J., Caruana, R.: Do deep nets really need to be deep? In: Advances in Neural Information Processing Systems, vol. 27 (2014)

    Google Scholar 

  3. Bollacker, K., Evans, C., Paritosh, P., Sturge, T., Taylor, J.: Freebase: a collaboratively created graph database for structuring human knowledge. In: Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data, pp. 1247–1250 (2008)

    Google Scholar 

  4. Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., Yakhnenko, O.: Translating embeddings for modeling multi-relational data. In: Advances in Neural Information Processing Systems, vol. 26 (2013)

    Google Scholar 

  5. Bordes, A., Weston, J., Collobert, R., Bengio, Y.: Learning structured embeddings of knowledge bases. In: Burgard, W., Roth, D. (eds.) Proceedings of the Twenty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2011, San Francisco, California, USA, 7–11 August 2011. AAAI Press (2011)

    Google Scholar 

  6. Cai, H., Zheng, V.W., Chang, K.C.C.: A comprehensive survey of graph embedding: problems, techniques, and applications. IEEE Trans. Knowl. Data Eng. 30(9), 1616–1637 (2018)

    Article  Google Scholar 

  7. Carlson, A., Betteridge, J., Kisiel, B., Settles, B., Hruschka, E.R., Mitchell, T.M.: Toward an architecture for never-ending language learning. In: Twenty-Fourth AAAI Conference on Artificial Intelligence (2010)

    Google Scholar 

  8. Dai Quoc Nguyen, T.D.N., Nguyen, D.Q., Phung, D.: A novel embedding model for knowledge base completion based on convolutional neural network. In: Proceedings of NAACL-HLT, pp. 327–333 (2018)

    Google Scholar 

  9. Demir, C., Moussallem, D., Ngomo, A.C.N.: A shallow neural model for relation prediction. In: IEEE 15th International Conference on Semantic Computing, pp. 179–182. IEEE (2021)

    Google Scholar 

  10. Demir, C., Ngomo, A.N.: Out-of-vocabulary entities in link prediction. CoRR abs/2105.12524 (2021). https://arxiv.org/abs/2105.12524

  11. Dettmers, T., Minervini, P., Stenetorp, P., Riedel, S.: Convolutional 2D knowledge graph embeddings. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32 (2018)

    Google Scholar 

  12. Ebisu, T., Ichise, R.: Toruse: knowledge graph embedding on a lie group. In: Thirty-second AAAI Conference on Artificial Intelligence (2018)

    Google Scholar 

  13. Feddoul, L.: Semantics-driven keyword search over knowledge graphs. In: Alani, H., Simperl, E. (eds.) Proceedings of the Doctoral Consortium at ISWC 2020 co-located with 19th International Semantic Web Conference (ISWC 2020), Athens, Greece, 3rd November 2020. CEUR Workshop Proceedings, vol. 2798, pp. 17–24. CEUR-WS.org (2020)

    Google Scholar 

  14. Ji, S., Pan, S., Cambria, E., Marttinen, P., Philip, S.Y.: A survey on knowledge graphs: representation, acquisition, and applications. IEEE Trans. Neural Netw. Learn. Syst. 33(2), 494–514 (2021)

    Article  MathSciNet  Google Scholar 

  15. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: Bengio, Y., LeCun, Y. (eds.) 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, 7–9 May 2015, Conference Track Proceedings (2015)

    Google Scholar 

  16. Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. CoRR abs/1609.02907 (2016)

    Google Scholar 

  17. Krompaß, D., Baier, S., Tresp, V.: Type-constrained representation learning in knowledge graphs. In: Arenas, M., et al. (eds.) ISWC 2015. LNCS, vol. 9366, pp. 640–655. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-25007-6_37

    Chapter  Google Scholar 

  18. Lacroix, T., Usunier, N., Obozinski, G.: Canonical tensor decomposition for knowledge base completion. In: International Conference on Machine Learning, pp. 2863–2872. PMLR (2018)

    Google Scholar 

  19. Le, T., Huynh, N., Le, B.: Link prediction on knowledge graph by rotation embedding on the hyperplane in the complex vector space. In: Farkaš, I., Masulli, P., Otte, S., Wermter, S. (eds.) ICANN 2021. LNCS, vol. 12893, pp. 164–175. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-86365-4_14

    Chapter  Google Scholar 

  20. Li, M., Moens, M.: Dynamic key-value memory enhanced multi-step graph reasoning for knowledge-based visual question answering. In: Thirty-Sixth AAAI Conference on Artificial Intelligence, AAAI 2022, Thirty-Fourth Conference on Innovative Applications of Artificial Intelligence, IAAI 2022, The 12th Symposium on Educational Advances in Artificial Intelligence, EAAI 2022 Virtual Event, 22 February–1 March 2022, pp. 10983–10992. AAAI Press (2022)

    Google Scholar 

  21. Li, Y., Fan, W., Liu, C., Lin, C., Qian, J.: TranSHER: translating knowledge graph embedding with hyper-ellipsoidal restriction. arXiv preprint arXiv:2204.13221 (2022)

  22. Lin, Y., Liu, Z., Luan, H., Sun, M., Rao, S., Liu, S.: Modeling relation paths for representation learning of knowledge bases. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, Lisbon, Portugal, pp. 705–714. Association for Computational Linguistics (2015)

    Google Scholar 

  23. Meilicke, C., Chekol, M.W., Ruffinelli, D., Stuckenschmidt, H.: Anytime bottom-up rule learning for knowledge graph completion. In: Proceedings of the 28th International Joint Conference on Artificial Intelligence, pp. 3137–3143 (2019)

    Google Scholar 

  24. Miller, G.A.: WordNet: a lexical database for English. Commun. ACM 38(11), 39–41 (1995)

    Article  Google Scholar 

  25. Nguyen, D.Q., Tong, V., Phung, D., Nguyen, D.Q.: Node co-occurrence based graph neural networks for knowledge graph link prediction. In: Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining, pp. 1589–1592 (2022)

    Google Scholar 

  26. Nguyen, D.Q.: A survey of embedding models of entities and relationships for knowledge graph completion. In: Proceedings of the Graph-Based Methods for Natural Language Processing (TextGraphs), pp. 1–14 (2020)

    Google Scholar 

  27. Nguyen, D.Q., Sirts, K., Qu, L., Johnson, M.: Stranse: a novel embedding model of entities and relationships in knowledge bases. In: Knight, K., Nenkova, A., Rambow, O. (eds.) The Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego California, USA, 12–17 June 2016, pp. 460–466. The Association for Computational Linguistics (2016)

    Google Scholar 

  28. Nickel, M., Rosasco, L., Poggio, T.: Holographic embeddings of knowledge graphs. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 30 (2016)

    Google Scholar 

  29. Nickel, M., Tresp, V., Kriegel, H.P.: A three-way model for collective learning on multi-relational data. In: ICML (2011)

    Google Scholar 

  30. Ruffinelli, D., Broscheit, S., Gemulla, R.: You CAN teach an old dog new tricks! on training knowledge graph embeddings. In: 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, 26–30 April 2020. OpenReview.net (2020)

    Google Scholar 

  31. Sabour, S., Frosst, N., Hinton, G.E.: Dynamic routing between capsules. In: Guyon, I., et al. (eds.) Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, Long Beach, CA, USA, 4–9 December 2017, pp. 3856–3866 (2017)

    Google Scholar 

  32. Sadeghian, A., Armandpour, M., Ding, P., Wang, D.Z.: Drum: end-to-end differentiable rule mining on knowledge graphs. In: Advances in Neural Information Processing Systems, vol. 32 (2019)

    Google Scholar 

  33. Schlichtkrull, M., Kipf, T.N., Bloem, P., van den Berg, R., Titov, I., Welling, M.: Modeling relational data with graph convolutional networks. In: Gangemi, A., et al. (eds.) ESWC 2018. LNCS, vol. 10843, pp. 593–607. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-93417-4_38

    Chapter  Google Scholar 

  34. Shang, C., Tang, Y., Huang, J., Bi, J., He, X., Zhou, B.: End-to-end structure-aware convolutional networks for knowledge base completion. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 3060–3067 (2019)

    Google Scholar 

  35. Shen, Y., Li, D., Nan, D.: Modeling path information for knowledge graph completion. Neural Comput. Appl. 34(3), 1951–1961 (2022)

    Article  Google Scholar 

  36. Shi, B., Weninger, T.: Proje: embedding projection for knowledge graph completion. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 31 (2017)

    Google Scholar 

  37. Socher, R., Chen, D., Manning, C.D., Ng, A.: Reasoning with neural tensor networks for knowledge base completion. In: Advances in Neural Information Processing Systems, vol. 26 (2013)

    Google Scholar 

  38. Srivastava, N., Hinton, G.E., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014). https://doi.org/10.5555/2627435.2670313

    Article  MathSciNet  MATH  Google Scholar 

  39. Stadelmaier, J., Padó, S.: Modeling paths for explainable knowledge base completion. In: Linzen, T., Chrupala, G., Belinkov, Y., Hupkes, D. (eds.) Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, BlackboxNLP@ACL 2019, Florence, Italy, 1 August 2019, pp. 147–157. Association for Computational Linguistics (2019)

    Google Scholar 

  40. Sun, Z., Deng, Z.H., Nie, J.Y., Tang, J.: Rotate: knowledge graph embedding by relational rotation in complex space. In: International Conference on Learning Representations (2019). https://openreview.net/forum?id=HkgEQnRqYQ

  41. Tong, V., Nguyen, D.Q., Phung, D., Nguyen, D.Q.: Two-view graph neural networks for knowledge graph completion. arXiv preprint arXiv:2112.09231 (2021)

  42. Toutanova, K., Chen, D.: Observed versus latent features for knowledge base and text inference. In: Proceedings of the 3rd Workshop on Continuous Vector Space Models and Their Compositionality, pp. 57–66 (2015)

    Google Scholar 

  43. Trouillon, T., Welbl, J., Riedel, S., Gaussier, É., Bouchard, G.: Complex embeddings for simple link prediction. In: International Conference on Machine Learning, pp. 2071–2080. PMLR (2016)

    Google Scholar 

  44. Vashishth, S., Sanyal, S., Nitin, V., Talukdar, P.: Composition-based multi-relational graph convolutional networks. In: International Conference on Learning Representations (2020)

    Google Scholar 

  45. Vrandečić, D., Krötzsch, M.: Wikidata: a free collaborative knowledgebase. Commun. ACM 57(10), 78–85 (2014)

    Article  Google Scholar 

  46. Vu, T., Nguyen, T.D., Nguyen, D.Q., Phung, D., et al.: A capsule network-based embedding model for knowledge graph completion and search personalization. In: Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 2180–2189 (2019)

    Google Scholar 

  47. Wang, H., Ren, H., Leskovec, J.: Entity context and relational paths for knowledge graph completion. CoRR abs/2002.06757 (2020). https://arxiv.org/abs/2002.06757

  48. Wang, Z., Zhang, J., Feng, J., Chen, Z.: Knowledge graph embedding by translating on hyperplanes. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 28 (2014)

    Google Scholar 

  49. West, R., Gabrilovich, E., Murphy, K., Sun, S., Gupta, R., Lin, D.: Knowledge base completion via search-based question answering. In: Proceedings of the 23rd International Conference on World Wide Web, pp. 515–526 (2014)

    Google Scholar 

  50. Wu, B., Deng, C., Guan, B., Wang, Y., Kangyang, Y.: Enhancing sequential recommendation via decoupled knowledge graphs. In: Groth, P., et al. (eds.) ESWC 2022. LNCS, vol. 13261, pp. 3–20. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-06981-9_1

    Chapter  Google Scholar 

  51. Xie, R., Liu, Z., Jia, J., Luan, H., Sun, M.: Representation learning of knowledge graphs with entity descriptions. In: Schuurmans, D., Wellman, M.P. (eds.) Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, Phoenix, Arizona, USA, 12–17 February 2016, pp. 2659–2665. AAAI Press (2016)

    Google Scholar 

  52. Xu, J., Qiu, X., Chen, K., Huang, X.: Knowledge graph representation with jointly structural and textual encoding. In: Sierra, C. (ed.) Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI 2017, Melbourne, Australia, 19–25 August 2017, pp. 1318–1324. ijcai.org (2017)

    Google Scholar 

  53. Yang, B., Yih, S.W., He, X., Gao, J., Deng, L.: Embedding entities and relations for learning and inference in knowledge bases. In: Proceedings of the International Conference on Learning Representations (2015)

    Google Scholar 

  54. Yao, L., Mao, C., Luo, Y.: KG-BERT: BERT for knowledge graph completion. CoRR abs/1909.03193 (2019)

    Google Scholar 

  55. Zhang, W., et al.: NeuralKG: an open source library for diverse representation learning of knowledge graphs. In: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 3323–3328 (2022)

    Google Scholar 

  56. Zhang, Z., Zhuang, F., Zhu, H., Shi, Z., Xiong, H., He, Q.: Relational graph neural network with hierarchical attention for knowledge graph completion. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 9612–9619 (2020)

    Google Scholar 

Download references

Acknowledgments

We acknowledge anonymous reviewers for their valuable comments. This work was supported by Capital University of Economics and Business (Grant No.01892254413027).

Author information

Authors and Affiliations

  1. Capital University of Economics and Business, Beijing, China

    Ningning Jia

Authors
  1. Ningning Jia
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ningning Jia .

Editor information

Editors and Affiliations

  1. Indian Institute of Technology Indore, Indore, India

    Mohammad Tanveer

  2. Indian Institute of Information Technology - Allahabad, Prayagraj, India

    Sonali Agarwal

  3. Kobe University, Kobe, Japan

    Seiichi Ozawa

  4. Indian Institute of Technology Patna, Patna, India

    Asif Ekbal

  5. University of Innsbruck, Innsbruck, Austria

    Adam Jatowt

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Jia, N. (2023). ASLEEP: A Shallow neural modEl for knowlEdge graph comPletion. In: Tanveer, M., Agarwal, S., Ozawa, S., Ekbal, A., Jatowt, A. (eds) Neural Information Processing. ICONIP 2022. Communications in Computer and Information Science, vol 1792. Springer, Singapore. https://doi.org/10.1007/978-981-99-1642-9_9

Download citation

  • DOI: https://doi.org/10.1007/978-981-99-1642-9_9

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-99-1641-2

  • Online ISBN: 978-981-99-1642-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation