Skip to main content
SpringerLink
Log in

Open-world knowledge graph completion with multiple interaction attention

  • Published:
World Wide Web Aims and scope Submit manuscript

Abstract

Knowledge Graph Completion (KGC) aims at complementing missing relationships between entities in a Knowledge Graph (KG). While closed-world KGC approaches utilizing the knowledge within KG could only complement very limited number of missing relations, more and more approaches tend to get knowledge from open-world resources such as online encyclopedias and newswire corpus. For instance, a recent proposed open-world KGC model called ConMask learns embeddings of the entity’s name and parts of its text-description to connect unseen entities to the KGs. However, this model does not make full use of the rich feature information in the text descriptions, besides, the proposed relationship-dependent content masking method may easily miss to find the target-words. In this paper, we propose to use a Multiple Interaction Attention (MIA) mechanism to model the interactions between the head entity description, head entity name, the relationship name, and the candidate tail entity descriptions, to form the enriched representations. In addition, we try to use the additional textual features of head entity descriptions to enhance the head entity representation and apply the attention mechanism between candidate tail entities to enhance the representation of them. Besides, we try different scoring functions to increase the convergence of the model. Our empirical study conducted on three real-world data collections shows that our approach achieves significant improvements comparing to state-of-the-art KGC methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. https://github.com/explosion/spaCy

  2. https://github.com/hanxiao/bert-as-service

  3. https://pytorch.org

References

  1. Balažević, I., Allen, C., Hospedales, T.M.: Tucker: Tensor factorization for knowledge graph completion. arXiv preprint arXiv:1901.09590(2019)

  2. Bengio, Y., Simard, P., Frasconi, P., et al.: Learning long-term dependencies with gradient descent is difficult. IEEE Trans. Neural Netw. 5(2), 157–166 (1994)

    Article  Google Scholar 

  3. Bollacker, K., Evans, C., Paritosh, P., Sturge, T., Taylor, J.: Freebase: a collaboratively created graph database for structuring human knowledge. In: Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data, pp 1247–1250. AcM (2008)

  4. Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., Yakhnenko, O.: Translating embeddings for modeling multi-relational data. In: Advances in Neural Information Processing Systems, pp 2787–2795 (2013)

  5. Chen, D., Fisch, A., Weston, J., Bordes, A.: Reading wikipedia to answer open-domain questions. arXiv preprint arXiv:1704.00051 (2017)

  6. Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert:, Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)

  7. Fu, C., Li, Z., Yang, Q., Chen, Z., Fang, J., Zhao, P., Xu, J.: Multiple interaction attention model for open-world knowledge graph completion. In: International Conference on Web Information Systems Engineering, pp 630–644. Springer (2019)

  8. Graves, A., Schmidhuber, J.: Framewise phoneme classification with bidirectional lstm and other neural network architectures. Neural Netw. 18(5-6), 602–610 (2005)

    Article  Google Scholar 

  9. Gu, B., Li, Z., Zhang, X., Liu, A., Liu, G., Zheng, K., Zhao, L., Zhou, X.: The interaction between schema matching and record matching in data integration. IEEE Trans. Knowl. Data Eng. 29(1), 186–199 (2016)

    Article  Google Scholar 

  10. Hachey, B., Radford, W., Nothman, J., Honnibal, M., Curran, J.R.: Evaluating entity linking with wikipedia. In: AI, vol. 194, pp 130–150. Elsevier (2013)

  11. Hochreiter, S., Bengio, Y., Frasconi, P., Schmidhuber, J., et al.: Gradient flow in recurrent nets: the difficulty of learning long-term dependencies (2001)

  12. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)

    Article  Google Scholar 

  13. Huang, L., May, J., Pan, X., Ji, H., Ren, X., Han, J., Zhao, L., Hendler, J. A.: Liberal entity extraction: Rapid construction of fine-grained entity typing systems. Big Data 5(1), 19–31 (2017)

    Article  Google Scholar 

  14. Ji, G., He, S., Xu, L., Liu, K., Zhao, J.: Knowledge graph embedding via dynamic mapping matrix. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), vol. 1, pp 687–696 (2015)

  15. Kadlec, R., Schmid, M., Bajgar, O., Kleindienst, J.: Text understanding with the attention sum reader network. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (2016)

  16. Kazemi, S. M., Poole, D.: Simple embedding for link prediction in knowledge graphs. In: Advances in Neural Information Processing Systems, pp 4284–4295 (2018)

  17. Kingma, D.P., Ba, J.: Adam:, A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

  18. Lee, K., Salant, S., Kwiatkowski, T., Parikh, A., Das, D., Berant, J.: Learning recurrent span representations for extractive question answering. arXiv preprint arXiv:1611.01436 (2016)

  19. Lehmann, J., Isele, R., Jakob, M., Jentzsch, A., Kontokostas, D., Mendes, P.N., Hellmann, S., Morsey, M., Van Kleef, P., Auer, S., et al.: Dbpedia–a large-scale, multilingual knowledge base extracted from wikipedia. Semantic Web 6(2), 167–195 (2015)

    Article  Google Scholar 

  20. Lin, Y., Liu, Z., Luan, H., Sun, M., Rao, S., Liu, S.: Modeling relation paths for representation learning of knowledge bases. In: EMNLP, pp 705–714 (2015)

  21. Lin, Y., Liu, Z., Sun, M., Liu, Y., Zhu, X.: Learning entity and relation embeddings for knowledge graph completion. In: AAAI, vol. 15, pp 2181–2187 (2015)

  22. Liu, G., Wang, Y., Orgun, M.A.: Optimal social trust path selection in complex social networks. In: Twenty-Fourth AAAI Conference on Artificial Intelligence, pp 1391–1398 (2010)

  23. Liu, G., Wang, Y., Orgun, M.A., Lim, E.P.: Finding the optimal social trust path for the selection of trustworthy service providers in complex social networks. IEEE Trans. Serv. Comput. 6(2), 152–167 (2011)

    Article  Google Scholar 

  24. Liu, G., Wang, Y., Orgun, M.A., Lim, E.P.: Finding the optimal social trust path for the selection of trustworthy service providers in complex social networks. IEEE Trans. Serv. Comput. 6(2), 152–167 (2013)

    Article  Google Scholar 

  25. Lukovnikov, D., Fischer, A., Lehmann, J., Auer, S.: Neural network-based question answering over knowledge graphs on word and character level. In: Proceedings of the 26th international conference on World Wide Web, pp 1211–1220. International World Wide Web Conferences Steering Committee (2017)

  26. Mintz, M., Bills, S., Snow, R., Jurafsky, D.: Distant supervision for relation extraction without labeled data. In: Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 2, vol. 2, pp 1003–1011. Association for Computational Linguistics (2009)

  27. Pennington, J., Socher, R., Manning, C.: Glove: Global vectors for word representation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp 1532–1543 (2014)

  28. Qian, C., Zhu, X., Ling, Z.H., Si, W., Inkpen, D.: Enhanced lstm for natural language inference. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (2017)

  29. Schuster, M., Paliwal, K.K.: Bidirectional recurrent neural networks. IEEE Trans. Signal Process. 45(11), 2673–2681 (1997)

    Article  Google Scholar 

  30. Shi, B., Weninger, T.: Open-world knowledge graph completion. In: Thirty-Second AAAI Conference on Artificial Intelligence, pp 1957–1964 (2018)

  31. Socher, R., Chen, D., Manning, C.D., Ng, A.: Reasoning with neural tensor networks for knowledge base completion. In: Advances in Neural Information Processing Systems, pp 926–934 (2013)

  32. Sordoni, A., Bachman, P., Bengio, Y.: Iterative alternating neural attention for machine reading

  33. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)

    MathSciNet  MATH  Google Scholar 

  34. Sun, Z., Deng, Z.H., Nie, J.Y., Tang, J.: Rotate:, Knowledge graph embedding by relational rotation in complex space. arXiv preprint arXiv:1902.10197 (2019)

  35. Trischler, A., Ye, Z., Yuan, X., He, J., Bachman, P., Suleman, K.: A parallel-hierarchical model for machine comprehension on sparse data

  36. Wang, H., Zhang, F., Wang, J., Zhao, M., Li, W., Xie, X., Guo, M.: Ripplenet: Propagating user preferences on the knowledge graph for recommender systems. In: Proceedings of the 27th ACM International Conference on Information and Knowledge Management, pp 417–426. ACM (2018)

  37. Wang, L., Sun, M., Zhao, W., Shen, K., Liu, J.: Yuanfudao at semeval-2018 task 11:, three-way attention and relational knowledge for commonsense machine comprehension. arXiv preprint arXiv:1803.00191 (2018)

  38. Wang, Z., Zhang, J., Feng, J., Chen, Z.: Knowledge graph embedding by translating on hyperplanes. In: Twenty-Eighth AAAI Conference on Artificial Intelligence (2014)

  39. Xie, R., Liu, Z., Jia, J., Luan, H., Sun, M.: Representation learning of knowledge graphs with entity descriptions. In: Thirtieth AAAI Conference on Artificial Intelligence (2016)

  40. Xu, J., Chen, K., Qiu, X., Huang, X.: Knowledge graph representation with jointly structural and textual encoding. Arxiv Preprint Arxiv:1611.08661 (2016)

  41. Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., Hovy, E.: Hierarchical attention networks for document classification. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp 1480–1489 (2016)

  42. Zhang, D., Yuan, B., Wang, D., Liu, R.: Joint semantic relevance learning with text data and graph knowledge. In: Proceedings of the 3rd Workshop on Continuous Vector Space Models and their Compositionality, pp 32–40 (2015)

  43. Zhang, Y., Liu, G., Liu, A., Zhang, Y., Li, Z., Zhang, X., Li, Q.: Personalized geographical influence modeling for poi recommendation. IEEE Intell Sys, (01), 1–1. https://doi.org/10.1109/MIS.2020.2998040 (2020)

  44. Zhu, H., Wei, F., Qin, B., Liu, T.: Hierarchical attention flow for multiple-choice reading comprehension. In: Thirty-Second AAAI Conference on Artificial Intelligence (2018)

Download references

Acknowledgments

This research is partially supported by National Key R&D Program of China (No. 2018AAA0101900), the Priority Academic Program Development of Jiangsu Higher Education Institutions, National Natural Science Foundation of China (Grant No. 62072323, 61632016, 61836007, 61972069, 61832017, 61532018), Natural Science Foundation of Jiangsu Province (No. BK20191420), Natural Science Research Project of Jiangsu Higher Education Institution (No. 17KJA520003), and the Suda-Toycloud Data Intelligence Joint Laboratory.

Author information

Authors and Affiliations

  1. Institute of Artificial Intelligence, School of Computer Science and Technology, Soochow University, Suzhou, China

    Lei Niu, Chenpeng Fu & Zhixu Li

  2. King Abdullah University of Science and Technology, Jeddah, Saudi Arabia

    Qiang Yang

  3. iFLYTEK Research, Suzhou, China

    Zhixu Li

  4. State Key Laboratory of Cognitive Intelligence, iFLYTEK, Suzhou, China

    Zhigang Chen

  5. Anhui Toycloud Technology, Hefei, China

    Qingsheng Liu

  6. School of Computer Science and Engineering, University of Electronic Science and Technology, Chengdu, China

    Kai Zheng

  7. School of Computer Science and Technology, Shandong University of Technology, Shandong, China

    Kai Zheng

Authors
  1. Lei Niu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chenpeng Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qiang Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhixu Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zhigang Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Qingsheng Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Kai Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kai Zheng.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article belongs to the Topical Collection: Special Issue on Web Information Systems Engineering 2019

Guest Editors: Reynold Cheng, Nikos Mamoulis, and Xin Huang

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Niu, L., Fu, C., Yang, Q. et al. Open-world knowledge graph completion with multiple interaction attention. World Wide Web 24, 419–439 (2021). https://doi.org/10.1007/s11280-020-00847-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11280-020-00847-2

Keywords

Search

Navigation