Abstract
Link prediction aims to infer the behavior of the network evolution process by predicting missed or future relationships based on currently observed connections. It has become an attractive area of research since it allows us to understand how networks will evolve. Early studies cast the link prediction task as an entity identifying problem on graphs and adopt vertex representation strategies to perform predictive analysis. Although these methods are effective to some extent, they overlook the special properties of network evolution.
In this paper, we propose a new method named TKGE, short for Temporal Knowledge Graph Embedding, to learn the evolutional representations of temporal knowledge graph for link prediction task. Specifically, we employ the self-attention mechanism to incorporate the static structural information and dynamic temporal information by aggregating the context from related entities. By introducing the position embedding characterizing the dynamic information of temporal knowledge graph, TKGE can generate the evolutional embedding of entities and relations for downstream applications, such as link prediction, recommender system, and so on. We conduct experiments on several real datasets. Both quantitative results and qualitative analysis verify the effectiveness and rationality of our TKGE method.
Y. Zhang, Z. Deng and D. Meng—These authors contribute equally to this work.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Cheng, K., Yang, Z., Zhang, M., Sun, Y.: Uniker: a unified framework for combining embedding and definite horn rule reasoning for knowledge graph inference. In: EMNLP, pp. 9753–9771 (2021)
Che, F., Zhang, D., Tao, J., Niu, M., Zhao, B.: Parame: regarding neural network parameters as relation embeddings for knowledge graph completion. In: AAAI, pp. 2774–2781 (2020)
Goel, R., Kazemi, S.M., Brubaker, M., Poupart, P.: Diachronic embedding for temporal knowledge graph completion. In: AAAI, pp. 3988–3995 (2020)
Mei, H., Eisner, J.: The neural hawkes process: a neurally self-modulating multivariate point process. In: NIPS, pp. 6754–6764 (2017)
Trivedi, R., Farajtabar, M., Biswal, P., Zha, H.: Dyrep: learning representations over dynamic graphs. In: International Conference on Learning Representations (2019)
Li, Z., et al.: Temporal knowledge graph reasoning based on evolutional representation learning. In: SIGIR, pp. 408–417 (2021)
Jin, W., Qu, M., Jin, X., Ren, X.: Recurrent event network: autoregressive structure inferenceover temporal knowledge graphs. In: EMNLP, pp. 6669–6683 (2020)
Kong, C., Chen, B., Li, S., Chen, Y., Chen, J., Zhang, L.: GNE: generic heterogeneous information network embedding. In: WISA, pp. 120–127 (2020)
Vaswani, A., et al.: Attention is all you need. In: NIPS, pp. 5998–6008 (2017)
Cheng, S., Xie, M., Ma, Z., Li, S., Gu, S., Yang, F.: Spatio-temporal self-attention weighted VLAD neural network for action recognition. IEICE 104-D, pp. 220–224 (2021)
Liu, J., Chen, S., Wang, B., Zhang, J., Li, N., Xu, T.: Attention as relation: learning supervised multi-head self-attention for relation extraction. In: IJCAI, pp. 3787–3793 (2020)
Xu, Y., Huang, H., Feng, C., Hu, Y.: A supervised multi-head self-attention network for nested named entity recognition. In: AAAI, pp. 14185–14193 (2021)
Shang, C., Tang, Y., Huang, J., Bi, J., He, X., Zhou, B.: End-to-end structure-aware convolutional networks for knowledge base completion. In: AAAI, pp. 3060–3067 (2019)
Dasgupta, S.S., Ray, S.N., Talukdar, P.P.: Hyte: hyperplane-based temporally aware knowledge graph embedding. In: EMNLP, pp. 2001–2011 (2018)
Leblay, J., Chekol, M.W.: Deriving validity time in knowledge graph. In: WWW, pp. 1771–1776. ACM (2018)
García-Durán, A., Dumancic, S., Niepert, M.: Learning sequence encoders for temporal knowledge graph completion. In: EMNLP, pp. 4816–4821 (2018)
Schlichtkrull, M.S., Kipf, T.N., an Rianne van den Berg, P.B., Titov, I., Welling, M.: Modeling relational data with graph convolutional networks. In: ESWC, vol. 10843, pp. 593–607 (2018)
Zhu, C., Chen, M., Fan, C., Cheng, G., Zhang, Y.: Learning from history: modeling temporal knowledge graphs with sequential copy-generation networks. In: AAAI, pp. 4732–4740 (2021)
Garg, K., Panagou, D.: Fixed-time stable gradient flows: applications to continuous-time optimization. IEEE Trans. Autom. Control. 66(5), 2002–2015 (2021)
Chien, J., Chen, Y.: Continuous-time attention for sequential learning. In: AAAI, pp. 7116–7124 (2021)
Zhang, L., Zhao, L., Qin, S., Pfoser, D., Ling, C.: TG-GAN: continuous-time temporal graph deep generative models with time-validity constraints. In: WWW, pp. 2104–2116 (2021)
Acknowledgment
This work was supported in part by the National Natural Science Foundation of China Youth Fund (No. 61902001), the Open Project of Shanghai Big Data Management System Engineering Research Center (No. 40500-21203-542500/021), the Industry Collaborative Innovation Fund of Anhui Polytechnic University-Jiujiang District (No. 2021cyxtb4), and the Science Research Project of Anhui Polytechnic University (No. Xjky072019C02, No. Xjky2020120). We would also thank the anonymous reviewers for their detailed comments, which have helped us to improve the quality of this work. All opinions, findings, conclusions and recommendations in this paper are those of the authors and do not necessarily reflect the views of the funding agencies.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Zhang, Y. et al. (2022). Temporal Knowledge Graph Embedding for Link Prediction. In: Zhao, X., Yang, S., Wang, X., Li, J. (eds) Web Information Systems and Applications. WISA 2022. Lecture Notes in Computer Science, vol 13579. Springer, Cham. https://doi.org/10.1007/978-3-031-20309-1_1
Download citation
DOI: https://doi.org/10.1007/978-3-031-20309-1_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-20308-4
Online ISBN: 978-3-031-20309-1
eBook Packages: Computer ScienceComputer Science (R0)