Skip to main content
SpringerLink
Log in

Aligning Internal Regularity and External Influence of Multi-granularity for Temporal Knowledge Graph Embedding

  • Conference paper
  • First Online:
Database Systems for Advanced Applications (DASFAA 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13247))

Included in the following conference series:

Abstract

Representation learning for the Temporal Knowledge Graphs (TKGs) is an emerging topic in the knowledge reasoning community. Existing methods consider the internal and external influence at either element level or fact level. However, the multi-granularity information is essential for TKG modeling and the connection in between is also under-explored. In this paper, we propose the method that Aligning-internal Regularity and external Influence of Multi-granularity for Temporal knowledge graph Embedding (ARIM-TE). In particular, to prepare considerate source information for alignment, ARIM-TE first models element-level information via the addition between internal regularity and the external influence. Based on the element-level information, the merge gate is introduced to model the fact-level information by combining their internal regularity including the local and global influence with external random perturbation. Finally, according to the above obtained multi-granular information of rich features, ARIM-TE conducts alignment for them in both structure and semantics. Experimental results show that ARIM-TE outperforms current state-of-the-art KGE models on several TKG link prediction benchmarks.

T. Zhang and Z. Li—Equal contribution.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Bordes, A., Glorot, X., Weston, J., Bengio, Y.: A semantic matching energy function for learning with multi-relational data. Mach. Learn. 94(2), 233–259 (2013). https://doi.org/10.1007/s10994-013-5363-6

    Article  MathSciNet  MATH  Google Scholar 

  2. Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., Yakhnenko, O.: Translating embeddings for modeling multi-relational data. In: Neural Information Processing Systems (NIPS), pp. 1–9 (2013)

    Google Scholar 

  3. Chung, J., Gulcehre, C., Cho, K., Bengio, Y.: Empirical evaluation of gated recurrent neural networks on sequence modeling. In: NIPS 2014 Workshop on Deep Learning (2014)

    Google Scholar 

  4. Dasgupta, S.S., Ray, S.N., Talukdar, P.: HyTE: hyperplane-based temporally aware knowledge graph embedding. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 2001–2011 (2018)

    Google Scholar 

  5. Dettmers, T., Minervini, P., Stenetorp, P., Riedel, S.: Convolutional 2D knowledge graph embeddings. In: Thirty-Second AAAI Conference on Artificial Intelligence, pp. 1811–1818 (2018)

    Google Scholar 

  6. Dong, L., Wei, F., Zhou, M., Xu, K.: Question answering over Freebase with multi-column convolutional neural networks. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 260–269 (2015)

    Google Scholar 

  7. Garcia-Duran, A., Dumančić, S., Niepert, M.: Learning sequence encoders for temporal knowledge graph completion. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 4816–4821 (2018)

    Google Scholar 

  8. Goel, R., Kazemi, S.M., Brubaker, M., Poupart, P.: Diachronic embedding for temporal knowledge graph completion. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 3988–3995 (2020)

    Google Scholar 

  9. Guo, L., Sun, Z., Hu, W.: Learning to exploit long-term relational dependencies in knowledge graphs. In: International Conference on Machine Learning, pp. 2505–2514 (2019)

    Google Scholar 

  10. Han, Z., Chen, P., Ma, Y., Tresp, V.: DyERNIE: dynamic evolution of Riemannian manifold embeddings for temporal knowledge graph completion. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 7301–7316 (2020)

    Google Scholar 

  11. Ji, G., He, S., Xu, L., Liu, K., Zhao, J.: Knowledge graph embedding via dynamic mapping matrix. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 687–696 (2015)

    Google Scholar 

  12. Ji, G., Liu, K., He, S., Zhao, J.: Knowledge graph completion with adaptive sparse transfer matrix. In: Thirtieth AAAI Conference on Artificial Intelligence, pp. 985–991 (2016)

    Google Scholar 

  13. Ji, S., Pan, S., Cambria, E., Marttinen, P., Yu, P.S.: A survey on knowledge graphs: representation, acquisition, and applications. IEEE Trans. Neural Netw. Learn. Syst. 1–21 (2021). https://doi.org/10.1109/TNNLS.2021.3070843

  14. Jiang, T., et al.: Towards time-aware knowledge graph completion. In: Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pp. 1715–1724 (2016)

    Google Scholar 

  15. Jin, W., Qu, M., Jin, X., Ren, X.: Recurrent event network: autoregressive structure inference over temporal knowledge graphs. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pp. 6669–6683 (2020)

    Google Scholar 

  16. Kazemi, S.M., et al.: Representation learning for dynamic graphs: a survey. J. Mach. Learn. Res. 21, 1–73 (2020)

    MathSciNet  Google Scholar 

  17. Kazemi, S.M., Poole, D.: Simple embedding for link prediction in knowledge graphs. In: Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, pp. 4289–4300 (2018)

    Google Scholar 

  18. Kumar, S., Zhang, X., Leskovec, J.: Predicting dynamic embedding trajectory in temporal interaction networks. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1269–1278 (2019)

    Google Scholar 

  19. Lacroix, T., Obozinski, G., Usunier, N.: Tensor decompositions for temporal knowledge base completion. In: 8th International Conference on Learning Representations (2020)

    Google Scholar 

  20. Leblay, J., Chekol, M.W.: Deriving validity time in knowledge graph. In: Companion Proceedings of the the Web Conference 2018, pp. 1771–1776 (2018)

    Google Scholar 

  21. Li, Z., et al.: Temporal knowledge graph reasoning based on evolutional representation learning. In: The 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2021, pp. 408–417 (2021)

    Google Scholar 

  22. Lin, Y., Liu, Z., Sun, M., Liu, Y., Zhu, X.: Learning entity and relation embeddings for knowledge graph completion. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 2181–2187 (2015)

    Google Scholar 

  23. Liu, Q., et al.: Probabilistic reasoning via deep learning: neural association models. arXiv preprint arXiv:1603.07704 (2016)

  24. Ma, Y., Tresp, V., Daxberger, E.A.: Embedding models for episodic knowledge graphs. J. Web Semant. 100490 (2019)

    Google Scholar 

  25. Montella, S., Rojas-Barahona, L.M., Heinecke, J.: Hyperbolic temporal knowledge graph embeddings with relational and time curvatures. In: Findings of the Association for Computational Linguistics: ACL/IJCNLP 2021, pp. 3296–3308 (2021)

    Google Scholar 

  26. Nathani, D., Chauhan, J., Sharma, C., Kaul, M.: Learning attention-based embeddings for relation prediction in knowledge graphs. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 4710–4723 (2019)

    Google Scholar 

  27. Nickel, M., Rosasco, L., Poggio, T.: Holographic embeddings of knowledge graphs. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 1955–1961 (2016)

    Google Scholar 

  28. Nickel, M., Tresp, V., Kriegel, H.P.: A three-way model for collective learning on multi-relational data. In: Proceedings of the 28th International Conference on International Conference on Machine Learning, pp. 809–816 (2011)

    Google Scholar 

  29. Riedel, S., Yao, L., McCallum, A., Marlin, B.M.: Relation extraction with matrix factorization and universal schemas. In: Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 74–84 (2013)

    Google Scholar 

  30. Sadeghian, A., Armandpour, M., Colas, A., Wang, D.Z.: ChronoR: rotation based temporal knowledge graph embedding. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 6471–6479 (2021)

    Google Scholar 

  31. Schlichtkrull, M., Kipf, T.N., Bloem, P., van den Berg, R., Titov, I., Welling, M.: Modeling relational data with graph convolutional networks. In: Gangemi, A., et al. (eds.) ESWC 2018. LNCS, vol. 10843, pp. 593–607. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-93417-4_38

    Chapter  Google Scholar 

  32. Trivedi, R., Dai, H., Wang, Y., Song, L.: Know-evolve: deep temporal reasoning for dynamic knowledge graphs. In: Proceedings of the 34th International Conference on Machine Learning, pp. 3462–3471 (2017)

    Google Scholar 

  33. Trouillon, T., Welbl, J., Riedel, S., Gaussier, E., Bouchard, G.: Complex embeddings for simple link prediction. In: Proceedings of The 33rd International Conference on Machine Learning, pp. 2071–2080 (2016)

    Google Scholar 

  34. Wang, Q., Mao, Z., Wang, B., Guo, L.: Knowledge graph embedding: a survey of approaches and applications. IEEE Trans. Knowl. Data Eng. 29(12), 2724–2743 (2017)

    Article  Google Scholar 

  35. Wang, Z., Zhang, J., Feng, J., Chen, Z.: Knowledge graph embedding by translating on hyperplanes. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 1112–1119 (2014)

    Google Scholar 

  36. West, R., Gabrilovich, E., Murphy, K., Sun, S., Gupta, R., Lin, D.: Knowledge base completion via search-based question answering. In: Proceedings of the 23rd International Conference on World Wide Web, pp. 515–526 (2014)

    Google Scholar 

  37. Weston, J., Bordes, A., Yakhnenko, O., Usunier, N.: Connecting language and knowledge bases with embedding models for relation extraction. In: Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp. 1366–1371 (2013)

    Google Scholar 

  38. Wu, J., Cao, M., Cheung, J.C.K., Hamilton, W.L.: TeMP: temporal message passing for temporal knowledge graph completion. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 5730–5746 (2020)

    Google Scholar 

  39. Xu, C., Chen, Y.Y., Nayyeri, M., Lehmann, J.: Temporal knowledge graph completion using a linear temporal regularizer and multivector embeddings. In: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 2569–2578 (2021)

    Google Scholar 

  40. Xu, C., Nayyeri, M., Alkhoury, F., Yazdi, H.S., Lehmann, J.: TeRo: a time-aware knowledge graph embedding via temporal rotation. In: Proceedings of the 28th International Conference on Computational Linguistics, pp. 1583–1593 (2020)

    Google Scholar 

  41. Xu, C., Nayyeri, M., Alkhoury, F., Yazdi, H., Lehmann, J.: Temporal knowledge graph completion based on time series gaussian embedding. In: International Semantic Web Conference, pp. 654–671 (2020)

    Google Scholar 

  42. Xu, K., Reddy, S., Feng, Y., Huang, S., Zhao, D.: Question answering on Freebase via relation extraction and textual evidence. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 2326–2336 (2016)

    Google Scholar 

  43. Yang, B., Yih, S.W.T., He, X., Gao, J., Deng, L.: Embedding entities and relations for learning and inference in knowledge bases. In: Proceedings of the International Conference on Learning Representations (ICLR) 2015 (2015)

    Google Scholar 

  44. You, J., Wang, Y., Pal, A., Eksombatchai, P., Rosenberg, C., Leskovec, J.: Hierarchical temporal convolutional networks for dynamic recommender systems. In: The World Wide Web Conference, pp. 2236–2246 (2019)

    Google Scholar 

  45. Zhu, C., Chen, M., Fan, C., Cheng, G., Zhang, Y.: Learning from history: modeling temporal knowledge graphs with sequential copy-generation networks. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 4732–4740 (2021)

    Google Scholar 

Download references

Acknowledgement

This research is supported by the National Key R&D Program of China (No. 2018AAA-0101900), the National Natural Science Foundation of China (Grant No. 62072323, 62102276), the Natural Science Foundation of Jiangsu Province (No. BK20191420, BK20210705, BK20211307), the Major Program of Natural Science Foundation of Educational Commission of Jiangsu Province, China (Grant No. 19KJA610002, 21KJD520-005), the Priority Academic Program Development of Jiangsu Higher Education Institutions, and the Collaborative Innovation Center of Novel Software Technology and Industrialization.

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Soochow University, Suzhou, China

    Tingyi Zhang, Jiaan Wang, Jianfeng Qu, Lin Yuan, An Liu & Lei Zhao

  2. Shanghai Key Laboratory of Data Science, School of Computer Science, Fudan University, Shanghai, China

    Zhixu Li

  3. iFLYTEK Research, Suzhou, China

    Zhigang Chen

Authors
  1. Tingyi Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhixu Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jiaan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jianfeng Qu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lin Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. An Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Lei Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Zhigang Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jianfeng Qu .

Editor information

Editors and Affiliations

  1. Indian Institute of Technology Kanpur, Kanpur, India

    Arnab Bhattacharya

  2. National University of Singapore, Singapore, Singapore

    Janice Lee Mong Li

  3. University of California, Santa Barbara, Santa Barbara, CA, USA

    Divyakant Agrawal

  4. IIIT Hyderabad, Hyderabad, India

    P. Krishna Reddy

  5. Indraprastha Institute of Information Technology Delhi, New Delhi, India

    Mukesh Mohania

  6. Ashoka University, Sonepat, Haryana, India

    Anirban Mondal

  7. Indraprastha Institute of Information Technology Delhi, New Delhi, India

    Vikram Goyal

  8. University of Aizu, Aizu, Japan

    Rage Uday Kiran

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhang, T. et al. (2022). Aligning Internal Regularity and External Influence of Multi-granularity for Temporal Knowledge Graph Embedding. In: Bhattacharya, A., et al. Database Systems for Advanced Applications. DASFAA 2022. Lecture Notes in Computer Science, vol 13247. Springer, Cham. https://doi.org/10.1007/978-3-031-00129-1_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-00129-1_10

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-00128-4

  • Online ISBN: 978-3-031-00129-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation