Skip to main content
SpringerLink
Log in

Improving knowledge graph completion via increasing embedding interactions

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Knowledge graphs usually consist of billions of triplet facts describing the real world. Although most of the existing knowledge graphs are huge in scale, they are still far from completion. As a result, varieties of knowledge graph embedding approaches have emerged, which have been proven to be an effective and efficient solution for knowledge graph completion. In this paper, we devise a novel knowledge graph embedding model named InterERP, which aims to improve model performance by increasing Inter actions between the embeddings of E ntities, R elations and relation P aths. Specifically, we first introduce the interaction matrix to obtain the interaction embeddings of entities and relations. Then, we employ the Inception network to learn the query embedding, which can further increase the interactions between entities and relations. Furthermore, we resort to logical rules to construct semantic relation paths and are committed to modeling the interactions between different relations in a relation path. The experimental results on four commonly used datasets, demonstrate that the proposed InterERP matches or outperforms the state-of-the-art approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. TransE is the leading work of knowledge graph embedding methods. Since then, a line of knowledge graph embedding approaches has been proposed. In them, one triplet in a knowledge graph consists of head entity, relation and tail entity, abbreviated as (h, r, t). Head entity is the left entity of one triplet, and tail entity is the right entity of one triple. In our paper, we also follow such a convention.

  2. The incomplete triplet like (X, motherLanguage, ?) also referred as a query in this paper.

  3. Compared with the general embeddings used in traditional models such as TransE, interaction embeddings contains more interrelated information between different embeddings, which are more conducive to realize the tasks of knowledge graph completion.

  4. The query embedding is the compositional representation of the embeddings of the head entity BillGates and the relation founded for a given query (BillGates, founded, ?), which is utilized to predict the tail entity. And it is the same for predicting the head entity.

  5. The term “node” is interchangeable with “entity” in this paper.

  6. Take a non-chain rule Rule2 : r1(z,x) ∧ r2(z,y) ⇒ r(x,y) for instance, we first convert the triplet r1(z,x) into \(r_{1}^{-1}(x,z)\) to obtain the chain rule r1(x,z) ∧ r2(z,y) ⇒ r(x,y).

  7. The code is available at https://github.com/cai-lw/KBGAN.

References

  1. Miller GA (1995) Wordnet: A lexical database for english. Commun. ACM 38(11):39–41. https://doi.org/10.1145/219717.219748

    Article  Google Scholar 

  2. Suchanek FM, Kasneci G, Weikum G (2007) Yago: a core of semantic knowledge. In: Williamson CL, Zurko ME, Patel-Schneider PF, Shenoy PJ (eds) Proceedings of the 16th international conference on world wide web, www 2007, Banff, Alberta, Canada, May 8-12, 2007. ACM, pp 697–706

  3. Bollacker KD, Evans C, Paritosh P, Sturge T, Taylor J (2008) Freebase: a collaboratively created graph database for structuring human knowledge. In: Wang JT-L (ed) Proceedings of the ACM SIGMOD international conference on management of data, SIGMOD 2008, Vancouver, BC, Canada, June 10-12, 2008. ACM, pp 1247–1250

  4. Lehmann J, Isele R, Jakob M, Jentzsch A, Kontokostas D, Mendes PN, Hellmann S, Morsey M, van Kleef P, Auer S, Bizer C (2015) Dbpedia - A large-scale, multilingual knowledge base extracted from wikipedia. Semantic Web 6(2):167–195. https://doi.org/10.3233/SW-140134

    Article  Google Scholar 

  5. Mitchell TM, Cohen WW, Jr. E RH, Talukdar PP, Yang B, Betteridge J, Carlson A, Mishra BD, Gardner M, Kisiel B, Krishnamurthy J, Lao N, Mazaitis K, Mohamed T, Nakashole N, Platanios EA, Ritter A, Samadi M, Settles B, Wang RC, Wijaya D, Gupta A, Chen X, Saparov A, Greaves M, Welling J (2018) Never-ending learning. Commun. ACM 61(5):103–115. https://doi.org/10.1145/3191513

  6. Bast H, Buchhold B, Haussmann E (2016) Semantic search on text and knowledge bases. Found. Trends Inf. Retr. 10(2-3):119–271. https://doi.org/10.1561/1500000032

    Article  Google Scholar 

  7. Xiong C, Power R, Callan J (2017) Explicit semantic ranking for academic search via knowledge graph embedding. In: Barrett R, Cummings R, Agichtein E, Gabrilovich E (eds) Proceedings of the 26th international conference on world wide web, www 2017, Perth, Australia, April 3-7, 2017. ACM, pp 1271–1279

  8. Yih W-, Chang M-W, He X, Gao J (2015) Semantic parsing via staged query graph generation: Question answering with knowledge base. In: Proceedings of the 53rd annual meeting of the association for computational linguistics and the 7th international joint conference on natural language processing of the asian federation of natural language processing, ACL 2015, July 26-31, 2015, Beijing, China, Volume 1: Long Papers. The Association for Computer Linguistics, pp 1321–1331

  9. Zhang Y, Dai H, Kozareva Z, Smola AJ, Song L (2018) Variational reasoning for question answering with knowledge graph. In: McIlraith SA, Weinberger KQ (eds) Proceedings of the thirty-second AAAI conference on artificial intelligence, (AAAI-18), the 30th innovative applications of artificial intelligence (IAAI-18), and the 8th AAAI symposium on educational advances in artificial intelligence (EAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018. AAAI Press, pp 6069–6076

  10. Júnior EAC, deAndradeLopes A, Amancio DR (2018) Word sense disambiguation: a complex network approach. Inf. Sci. 442-443:103–113. https://doi.org/10.1016/j.ins.2018.02.047

    Article  MathSciNet  Google Scholar 

  11. Wang Q, Pan X, Huang L, Zhang B, Jiang Z, Ji H, Knight K (2018) Describing a knowledge base. In: Krahmer E, Gatt A, Goudbeek M (eds) Proceedings of the 11th international conference on natural language generation, Tilburg University, The Netherlands, November 5-8, 2018. Association for Computational Linguistics, pp 10–21

  12. Li W, Peng R, Wang Y, Yan Z (2020) Knowledge graph based natural language generation with adapted pointer-generator networks. Neurocomputing 382:174–187. https://doi.org/10.1016/j.neucom.2019.11.079

    Article  Google Scholar 

  13. Guan S, Jin X, Wang Y, Cheng X (2018) Shared embedding based neural networks for knowledge graph completion. In: Cuzzocrea A, Allan J, Paton NW, Srivastava D, Agrawal R, Broder AZ, Zaki MJ, Candan KS, Labrinidis A, Schuster A, Wang H (eds) Proceedings of the 27th ACM international conference on information and knowledge management, CIKM 2018, Torino, Italy, October 22-26, 2018. ACM, pp 247–256

  14. Nguyen DQ, Nguyen DQ, Nguyen TD, Phung D (2019) A convolutional neural network-based model for knowledge base completion and its application to search personalization. Semantic Web 10(5):947–960. https://doi.org/10.3233/SW-180318

    Article  Google Scholar 

  15. Bordes A, Usunier N, García-Durán A, Weston J, Yakhnenko O (2013) Translating embeddings for modeling multi-relational data. In: Burges C JC, Bottou L, Ghahramani Z, Weinberger KQ (eds) Advances in Neural information processing systems 26: 27th annual conference on neural information processing systems 2013. Proceedings of a meeting held December 5-8, 2013, Lake Tahoe, Nevada, United States, pp 2787–2795

  16. Lin Y, Liu Z, Sun M, Liu Y, Zhu X (2015) Learning entity and relation embeddings for knowledge graph completion. In: Bonet B, Koenig S (eds) Proceedings of the twenty-ninth AAAI conference on artificial intelligence, January 25-30, 2015, Austin, Texas, USA. AAAI Press, pp 2181–2187

  17. Ji G, He S, Xu L, Liu K, Zhao J (2015) Knowledge graph embedding via dynamic mapping matrix. In: Proceedings of the 53rd annual meeting of the association for computational linguistics and the 7th international joint conference on natural language processing of the asian federation of natural language processing, ACL 2015, July 26-31, 2015, Beijing, China, Volume 1: Long Papers. The Association for Computer Linguistics, pp 687–696

  18. Yang B, Yih W-, He X, Gao J, Deng L (2015) Embedding entities and relations for learning and inference in knowledge bases. In: Bengio Y, LeCun Y (eds) 3rd International conference on learning representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings

  19. Trouillon T, Welbl J, Riedel S, Gaussier E, Bouchard G (2016) Complex embeddings for simple link prediction. In: Balcan M-F, Weinberger KQ (eds) Proceedings of the 33nd international conference on machine learning, ICML 2016, New York City, NY, USA, June 19-24, 2016. JMLR Workshop and Conference Proceedings, vol 48. JMLR.org, pp 2071–2080

  20. Dettmers T, Minervini P, Stenetorp P, Riedel S (2018) Convolutional 2d knowledge graph embeddings. In: McIlraith SA, Weinberger KQ (eds) Proceedings of the thirty-second AAAI conference on artificial intelligence, (AAAI-18), the 30th innovative applications of artificial intelligence (IAAI-18), and the 8th AAAI symposium on educational advances in artificial intelligence (EAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018. AAAI Press, pp 1811–1818

  21. Balazevic I, Allen C, Hospedales TM (2019) Hypernetwork knowledge graph embeddings. In: Tetko IV, Kurková V, Karpov P, Theis FJ (eds) Artificial Neural networks and machine learning - ICANN 2019 - 28th international conference on artificial neural networks, Munich, Germany, September 17-19, 2019, Proceedings - workshop and special sessions. Lecture Notes in Computer Science, vol 11731. Springer, pp 553–565

  22. Nguyen DQ, Nguyen TD, Nguyen DQ, Phung DQ (2018) A novel embedding model for knowledge base completion based on convolutional neural network. In: Walker MA, Ji H, Stent A (eds) Proceedings of the 2018 conference of the north american chapter of the association for computational linguistics: human language technologies, NAACL-HLT, New Orleans, Louisiana, USA, June 1-6, 2018, Volume 2 (Short Papers). Association for Computational Linguistics, pp 327–333

  23. García-Durán A, Bordes A, Usunier N (2014) Effective blending of two and three-way interactions for modeling multi-relational data. In: Calders T, Esposito F, Hüllermeier E, Meo R (eds) Machine learning and knowledge discovery in databases - European conference, ECML PKDD 2014, Nancy, France, September 15-19, 2014. Proceedings, part I. Lecture Notes in Computer Science, vol 8724. Springer, pp 434–449

  24. Vashishth S, Sanyal S, Nitin V, Agrawal N, Talukdar PP (2020) Interacte: Improving convolution-based knowledge graph embeddings by increasing feature interactions. In: The thirty-fourth AAAI conference on artificial intelligence, AAAI 2020, the thirty-second innovative applications of artificial intelligence conference, IAAI 2020, The tenth AAAI symposium on educational advances in artificial intelligence, EAAI 2020, New York, NY, USA, February 7-12, 2020. AAAI Press, pp 3009– 3016

  25. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. In: 2016 IEEE conference on computer vision and pattern recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016. IEEE Computer Society, pp 2818–2826

  26. Lin Y, Liu Z, Luan H-B, Sun M, Rao S, Liu S (2015) Modeling relation paths for representation learning of knowledge bases. In: Màrquez L, Callison-Burch C, Su J, Pighin D, Marton Y (eds) Proceedings of the 2015 conference on empirical methods in natural language processing, EMNLP 2015, Lisbon, Portugal, September 17-21, 2015. The Association for Computational Linguistics, pp 705–714

  27. Guu K, Miller J, Liang P (2015) Traversing knowledge graphs in vector space. In: Màrquez L, Callison-Burch C, Su J, Pighin D, Marton Y (eds) Proceedings of the 2015 conference on empirical methods in natural language processing, EMNLP 2015, Lisbon, Portugal, September 17-21, 2015. The Association for Computational Linguistics, pp 318–327

  28. Wang Z, Zhang J, Feng J, Chen Z (2014) Knowledge graph embedding by translating on hyperplanes. In: Brodley CE, Stone P (eds) Proceedings of the twenty-eighth AAAI conference on artificial intelligence, July 27-31, 2014, Québec City, Québec, Canada. AAAI Press, pp 1112–1119

  29. Zhang W, Paudel B, Zhang W, Bernstein A, Chen H (2019) Interaction embeddings for prediction and explanation in knowledge graphs. In: Culpepper JS, Moffat A, Bennett PN, Lerman K (eds) Proceedings of the twelfth ACM international conference on web search and data mining, WSDM 2019, Melbourne, VIC, Australia, February 11-15, 2019. ACM, pp 96–104

  30. Nickel M, Tresp V, Kriegel H-P (2011) A three-way model for collective learning on multi-relational data. In: Getoor L, Scheffer T (eds) Proceedings of the 28th international conference on machine learning, ICML 2011, Bellevue, Washington, USA, June 28 - July 2, 2011. Omnipress, pp 809–816

  31. Kazemi SM, Poole D (2018) Simple embedding for link prediction in knowledge graphs. In: Bengio S, Wallach HM, Larochelle H, Grauman K, Cesa-Bianchi N, Garnett R (eds) Advances in neural information processing systems 31: annual conference on neural information processing systems 2018, NeurIPS 2018, 3-8 December 2018, Montréal, Canada, pp 4289–4300

  32. Balazevic I, Allen C, Hospedales TM (2019) Tucker: Tensor factorization for knowledge graph completion. In: Inui K, Jiang J, Ng V, Wan X (eds) Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3-7, 2019. Association for Computational Linguistics, pp 5184–5193

  33. Das R, Dhuliawala S, Zaheer M, Vilnis L, Durugkar I, Krishnamurthy A, Smola A, McCallum A (2018) Go for a walk and arrive at the answer: reasoning over paths in knowledge bases using reinforcement learning. In: 6th International conference on learning representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings. OpenReview.net

  34. Schlichtkrull MS, Kipf TN, Bloem P, vanden Berg R, Titov I, Welling M (2018) Modeling relational data with graph convolutional networks. In: Gangemi A, Navigli R, Vidal M-E, Hitzler P, Troncy R, Hollink L, Tordai A, Alam M (eds) The semantic web - 15th international conference, ESWC 2018, Heraklion, Crete, Greece, June 3-7, 2018, Proceedings. Lecture Notes in Computer Science, vol 10843. Springer, pp 593–607

  35. Shang C, Tang Y, Huang J, Bi J, He X, Zhou B (2019) End-to-end structure-aware convolutional networks for knowledge base completion. In: The thirty-third AAAI conference on artificial intelligence, AAAI 2019, the thirty-first innovative applications of artificial intelligence conference, IAAI 2019, the ninth AAAI symposium on educational advances in artificial intelligence, EAAI 2019, Honolulu, Hawaii, USA, January 27 - February 1, 2019. AAAI Press, pp 3060–3067

  36. Nathani D, Chauhan J, Sharma C, Kaul M (2019) Learning attention-based embeddings for relation prediction in knowledge graphs. In: Korhonen A, Traum DR, Màrquez L (eds) Proceedings of the 57th conference of the association for computational linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers. Association for Computational Linguistics, pp 4710–4723

  37. Xie Z, Zhou G, Liu J, Huang JX (2020) Reinceptione: Relation-aware inception network with joint local-global structural information for knowledge graph embedding. In: Jurafsky D, Chai J, Schluter N, Tetreault JR (eds) Proceedings of the 58th annual meeting of the association for computational linguistics, ACL 2020, Online, July 5-10, 2020. Association for Computational Linguistics, pp 5929–5939

  38. Guo S, Wang Q, Wang L, Wang B, Guo L (2016) Jointly embedding knowledge graphs and logical rules. In: Su J, Carreras X, Duh K (eds) Proceedings of the 2016 conference on empirical methods in natural language processing, EMNLP 2016, Austin, Texas, USA, November 1-4, 2016. The Association for Computational Linguistics, pp 192–202

  39. Guo S, Wang Q, Wang L, Wang B, Guo L (2018) Knowledge graph embedding with iterative guidance from soft rules. In: McIlraith SA, Weinberger KQ (eds) Proceedings of the thirty-second AAAI conference on artificial intelligence, (AAAI-18), the 30th innovative applications of artificial intelligence (IAAI-18), and the 8th AAAI symposium on educational advances in artificial intelligence (EAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018. AAAI Press, pp 4816–4823

  40. Galárraga L, Teflioudi C, Hose K, Suchanek FM (2015) Fast rule mining in ontological knowledge bases with AMIE+. VLDB J. 24(6):707–730. https://doi.org/10.1007/s00778-015-0394-1

    Article  Google Scholar 

  41. Zhang W, Paudel B, Wang L, Chen J, Zhu H, Zhang W, Bernstein A, Chen H (2019) Iteratively learning embeddings and rules for knowledge graph reasoning. In: Liu L, White RW, Mantrach A, Silvestri F, McAuley JJ, Baeza-Yates R, Zia L (eds) The world wide web conference, WWW 2019, San Francisco, CA, USA, May 13-17, 2019. ACM, pp 2366–2377

  42. Yang F, Yang Z, Cohen WW (2017) Differentiable learning of logical rules for knowledge base reasoning. In: Guyon I, von Luxburg U, Bengio S, Wallach HM, Fergus R, Vishwanathan S VN, Garnett R (eds) Advances in neural information processing systems 30: annual conference on neural information processing systems 2017, 4-9 december 2017, long beach, ca, USA, pp 2319–2328

  43. Li W, Zhang X, Wang Y, Yan Z, Peng R (2019) Graph2seq: Fusion embedding learning for knowledge graph completion. IEEE Access 7:157960–157971. https://doi.org/10.1109/ACCESS.2019.2950230

    Article  Google Scholar 

  44. Glorot X, Bengio Y (2010) Understanding the difficulty of training deep feedforward neural networks. In: Teh YW, Titterington DM (eds) Proceedings of the thirteenth international conference on artificial intelligence and statistics, AISTATS 2010, Chia Laguna Resort, Sardinia, Italy, May 13-15, 2010. JMLR Proceedings, vol 9. JMLR.org, pp 249–256

  45. Omran PG, Wang K, Wang Z (2018) Scalable rule learning via learning representation. In: Lang J (ed) Proceedings of the twenty-seventh international joint conference on artificial intelligence, IJCAI 2018, July 13-19, 2018, Stockholm, Sweden. ijcai.org, pp 2149–2155

  46. Toutanova K, Chen D, Pantel P, Poon H, Choudhury P, Gamon M (2015) Representing text for joint embedding of text and knowledge bases. In: Màrquez L, Callison-Burch C, Su J, Pighin D, Marton Y (eds) Proceedings of the 2015 conference on empirical methods in natural language processing, EMNLP 2015, Lisbon, Portugal, September 17-21, 2015. The Association for Computational Linguistics, pp 1499–1509

  47. Shen Y, Chen J, Huang P-S, Guo Y, Gao J (2018) M-walk: Learning to walk over graphs using monte carlo tree search. In: Bengio S, Wallach HM, Larochelle H, Grauman K, Cesa-Bianchi N, Garnett R (eds) Advances in neural information processing systems 31: annual conference on neural information processing systems 2018, NeurIPS 2018, 3-8 December 2018, Montréal, Canada, pp 6787–6798

  48. Sun Z, Vashishth S, Sanyal S, Talukdar PP, Yang Y (2020) A re-evaluation of knowledge graph completion methods. In: Jurafsky D, Chai J, Schluter N, Tetreault JR (eds) Proceedings of the 58th annual meeting of the association for computational linguistics, ACL 2020, Online, July 5-10, 2020. Association for Computational Linguistics, pp 5516–5522

  49. Hoang N-D, Nguyen Q-L (2019) A novel method for asphalt pavement crack classification based on image processing and machine learning. Eng. Comput. 35(2):487–498. https://doi.org/10.1007/s00366-018-0611-9

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported by the National Key Research and Development Plan of China under Grant No. 2017YFB0503702, 2016YFB0501801, and National Natural Science Foundation of China under Grant No. 61862009, and Guangxi Natural Science Foundation under Grant No. 2018GXNSFAA281314. Rong Peng is the corresponding author of this paper.

Author information

Authors and Affiliations

  1. School of Computer Science, Wuhan University, Wuhan, China

    Weidong Li & Zhi Li

  2. College of Computer Science and Information Technology, Guangxi Normal University, Guangxi, China

    Rong Peng

Authors
  1. Weidong Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rong Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhi Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rong Peng.

Ethics declarations

Conflict of Interests

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, W., Peng, R. & Li, Z. Improving knowledge graph completion via increasing embedding interactions. Appl Intell 52, 9289–9307 (2022). https://doi.org/10.1007/s10489-021-02947-6

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-02947-6

Keywords

Search

Navigation