skip to main content
research-article

Embedding Heterogeneous Information Network in Hyperbolic Spaces

Published: 03 September 2021 Publication History

Abstract

Heterogeneous information network (HIN) embedding, aiming to project HIN into a low-dimensional space, has attracted considerable research attention. Most of the existing HIN embedding methods focus on preserving the inherent network structure and semantic correlations in Euclidean spaces. However, one fundamental problem is whether the Euclidean spaces are the intrinsic spaces of HIN? Recent researches find the complex network with hyperbolic geometry can naturally reflect some properties, e.g., hierarchical and power-law structure. In this article, we make an effort toward embedding HIN in hyperbolic spaces. We analyze the structures of three HINs and discover some properties, e.g., the power-law distribution, also exist in HINs. Therefore, we propose a novel HIN embedding model HHNE. Specifically, to capture the structure and semantic relations between nodes, HHNE employs the meta-path guided random walk to sample the sequences for each node. Then HHNE exploits the hyperbolic distance as the proximity measurement. We also derive an effective optimization strategy to update the hyperbolic embeddings iteratively. Since HHNE optimizes different relations in a single space, we further propose the extended model HHNE++. HHNE++ models different relations in different spaces, which enables it to learn complex interactions in HINs. The optimization strategy of HHNE++ is also derived to update the parameters of HHNE++ in a principle manner. The experimental results demonstrate the effectiveness of our proposed models.

References

[1]
Mukund Balasubramanian and Eric L. Schwartz. 2002. The isomap algorithm and topological stability. Science 295, 5552 (2002), 7–7.
[2]
Ivana Balazevic, Carl Allen, and Timothy Hospedales. 2019. Multi-relational poincaré graph embeddings. In Proceedings of the 33rd Conference on Neural Information Processing Systems.3672–3681.
[3]
Mikhail Belkin and Partha Niyogi. 2002. Laplacian eigenmaps and spectral techniques for embedding and clustering. In Proceedings of the 14th International Conference on Neural Information Processing Systems: Natural and Synthetic. 585–591.
[4]
Silvere Bonnabel. 2013. Stochastic gradient descent on riemannian manifolds.IEEE Transactions on Automatic Control 58, 9 (2013), 2217–2229.
[5]
Michael M Bronstein, Joan Bruna, Yann LeCun, Arthur Szlam, and Pierre Vandergheynst. 2017. Geometric deep learning: Going beyond euclidean data. IEEE Signal Processing Magazine 34, 4 (2017), 18–42.
[6]
James W. Cannon, William J. Floyd, Richard Kenyon, and Walter R. Parry. 1997. Hyperbolic geometry. Flavors of Geometry 31, 9 (1997), 59–115.
[7]
Iván Cantador, Peter Brusilovsky, and Tsvi Kuflik. 2011. 2nd workshop on information heterogeneity and fusion in recommender systems (HetRec 2011). In Proceedings of the 5th ACM Conference on Recommender Systems.
[8]
Shiyu Chang, Wei Han, Jiliang Tang, Guo-Jun Qi, Charu C. Aggarwal, and Thomas S Huang. 2015. Heterogeneous network embedding via deep architectures. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 119–128.
[9]
Hongxu Chen, Hongzhi Yin, Weiqing Wang, Hao Wang, Quoc Viet Hung Nguyen, and Xue Li. 2018. PME: Projected metric embedding on heterogeneous networks for link prediction. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 1177–1186.
[10]
Peng Cui, Xiao Wang, Jian Pei, and Wenwu Zhu. 2018. A survey on network embedding. IEEE Transactions on Knowledge and Data Engineering 31, 5 (2018) 833–852.
[11]
Bhuwan Dhingra, Christopher Shallue, Mohammad Norouzi, Andrew Dai, and George Dahl. 2018. Embedding text in hyperbolic spaces. In Proceedings of the 12th Workshop on Graph-Based Methods for Natural Language Processing. 59–69.
[12]
Yuxiao Dong, Nitesh V. Chawla, and Ananthram Swami. 2017. metapath2vec: Scalable representation learning for heterogeneous networks. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 135–144.
[13]
Tom Fawcett. 2006. An introduction to ROC analysis. Pattern Recognition Letters 27, 8 (2006), 861–874.
[14]
Tao-yang Fu, Wang-Chien Lee, and Zhen Lei. 2017. Hin2vec: Explore meta-paths in heterogeneous information networks for representation learning. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management. 1797–1806.
[15]
Octavian Ganea, Gary Becigneul, and Thomas Hofmann. 2018. Hyperbolic entailment cones for learning hierarchical embeddings. In Proceedings of the 35th International Conference on Machine Learning. 1646–1655.
[16]
Aditya Grover and Jure Leskovec. 2016. node2vec: Scalable feature learning for networks. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 855–864.
[17]
Xiaotian Han, Chuan Shi, Senzhang Wang, S. Yu Philip, and Li Song. 2018. Aspect-level deep collaborative filtering via heterogeneous information networks. In Proceedings of the 27th International Joint Conference on Artificial Intelligence. 3393–3399.
[18]
Zhicheng He, Jie Liu, Na Li, and Yalou Huang. 2019. Learning network-to-network model for content-rich network embedding. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 1037–1045.
[19]
Sigurdur Helgason. 1979. Differential geometry, Lie groups, and symmetric spaces. Vol. 80. Academic press.
[20]
Binbin Hu, Chuan Shi, Wayne Xin Zhao, and Philip S. Yu. 2018. Leveraging meta-path based context for top-n recommendation with a neural co-attention model. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 1531–1540.
[21]
Zhipeng Huang and Nikos Mamoulis. 2017. Heterogeneous information network embedding for meta path based proximity. arXiv:1701.05291. Retrieved from https://arxiv.org/abs/1701.05291.
[22]
Lawrence Hubert and Phipps Arabie. 1985. Comparing partitions. Journal of Classification 2, 1 (1985), 193–218.
[23]
Dmitri Krioukov, Fragkiskos Papadopoulos, Maksim Kitsak, Amin Vahdat, and Marián Boguná. 2010. Hyperbolic geometry of complex networks. Physical Review E 82, 3 (2010), 036106.
[24]
Marc Law, Renjie Liao, Jake Snell, and Richard Zemel. 2019. Lorentzian distance learning for hyperbolic representations. In Proceedings of the 36th International Conference on Machine Learning. 3672–3681.
[25]
Jeffrey Lee and Jeffrey Marc Lee. 2009. Manifolds and Differential Geometry, Vol. 107. American Mathematical Soc.
[26]
David McDonald and Shan He. 2020. HEAT: Hyperbolic embedding of attributed networks. In Proceedings of the 21st International Conference on Intelligent Data Engineering and Automated Learning. 28–40.
[27]
Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S. Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Proceedings of the 26th International Conference on Neural Information Processing Systems. 3111–3119.
[28]
Maximillian Nickel and Douwe Kiela. 2017. Poincaré embeddings for learning hierarchical representations. In Proceedings of the 31st International Conference on Neural Information Processing Systems. 6338–6347.
[29]
Maximillian Nickel and Douwe Kiela. 2018. Learning continuous hierarchies in the lorentz model of hyperbolic geometry. In Proceedings of the 35th International Conference on Machine Learning. 3776–3785.
[30]
Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. 2014. Deepwalk: Online learning of social representations. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 701–710.
[31]
Jiezhong Qiu, Yuxiao Dong, Hao Ma, Jian Li, Kuansan Wang, and Jie Tang. 2018. Network embedding as matrix factorization: Unifying deepwalk, line, pte, and node2vec. In Proceedings of the 11th ACM International Conference on Web Search and Data Mining. ACM, 459–467.
[32]
Frederic Sala, Chris De Sa, Albert Gu, and Christopher Re. 2018. Representation tradeoffs for hyperbolic embeddings. In Proceedings of the 35th International Conference on Machine Learning. 4457–4466.
[33]
Chuan Shi, Binbin Hu, Xin Zhao, and Philip Yu. 2018. Heterogeneous information network embedding for recommendation. IEEE Transactions on Knowledge and Data Engineering 31, 2 (2018) 357–370.
[34]
Chuan Shi, Yitong Li, Jiawei Zhang, Yizhou Sun, and S. Yu Philip. 2017. A survey of heterogeneous information network analysis. IEEE Transactions on Knowledge and Data Engineering 29, 1 (2017), 17–37.
[35]
Yu Shi, Qi Zhu, Fang Guo, Chao Zhang, and Jiawei Han. 2018. Easing embedding learning by comprehensive transcription of heterogeneous information networks. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 2190–2199.
[36]
Yizhou Sun, Jiawei Han, Xifeng Yan, Philip S. Yu, and Tianyi Wu. 2011. Pathsim: Meta path-based top-k similarity search in heterogeneous information networks. In Proceedings of the VLDB Endowment. 992–1003.
[37]
Ryota Suzuki, Ryusuke Takahama, and Shun Onoda. 2019. Hyperbolic disk embeddings for directed acyclic graphs. In Proceedings of the 36th International Conference on Machine Learning. 6066–6075.
[38]
Jian Tang, Meng Qu, and Qiaozhu Mei. 2015. Pte: Predictive text embedding through large-scale heterogeneous text networks. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 1165–1174.
[39]
Jian Tang, Meng Qu, Mingzhe Wang, Ming Zhang, Jun Yan, and Qiaozhu Mei. 2015. Line: Large-scale information network embedding. In Proceedings of the 24th International Conference on World Wide Web. 1067–1077.
[40]
Ke Tu, Jianxin Ma, Peng Cui, Jian Pei, and Wenwu Zhu. 2019. AutoNE: Hyperparameter optimization for massive network embedding. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.
[41]
Abraham A. Ungar. 2001. Hyperbolic trigonometry and its application in the Poincaré ball model of hyperbolic geometry. Computers and Mathematics with Applications 41, 1–2 (2001), 135–147.
[42]
Abraham A. Ungar. 2008. Analytic Hyperbolic Geometry and Albert Einstein’s Special Theory of Relativity. World Scientific.
[43]
Abraham Albert Ungar. 2008. A gyrovector space approach to hyperbolic geometry. Synthesis Lectures on Mathematics and Statistics 1, 1 (2008), 1–194.
[44]
Daixin Wang, Peng Cui, and Wenwu Zhu. 2016. Structural deep network embedding. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 1225–1234.
[45]
Hongwei Wang, Fuzheng Zhang, Min Hou, Xing Xie, Minyi Guo, and Qi Liu. 2018. SHINE: Signed heterogeneous information network embedding for sentiment link prediction. In Proceedings of the 11th ACM International Conference on Web Search and Data Mining. 592–600.
[46]
Junshan Wang, Zhicong Lu, Guojie Song, Yue Fan, Lun Du, and Wei Lin. 2019. Tag2Vec: Learning tag representations in tag networks. International World Wide Web Conference, 3314–3320.
[47]
Richard C Wilson, Edwin R Hancock, Elżbieta Pekalska, and Robert PW Duin. 2014. Spherical and hyperbolic embeddings of data. IEEE Transactions on Pattern Analysis and Machine Intelligence 36, 11 (2014), 2255–2269.
[48]
Linchuan Xu, Xiaokai Wei, Jiannong Cao, and Philip S Yu. 2017. Embedding of embedding (eoe): Joint embedding for coupled heterogeneous networks. In Proceedings of the 10th ACM International Conference on Web Search and Data Mining. ACM, 741–749.

Cited By

View all
  • (2025)Latent representation learning for classification of the Doppler ultrasound imagesComputers in Biology and Medicine10.1016/j.compbiomed.2024.109575185(109575)Online publication date: Feb-2025
  • (2024)BSIN: A Behavior Schema of Information Networks Based on Approximate BisimulationTsinghua Science and Technology10.26599/TST.2023.901008129:4(1092-1104)Online publication date: Aug-2024
  • (2024)Question Embedding on Weighted Heterogeneous Information Network for Knowledge TracingACM Transactions on Knowledge Discovery from Data10.1145/370315819:1(1-28)Online publication date: 10-Dec-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Knowledge Discovery from Data
ACM Transactions on Knowledge Discovery from Data  Volume 16, Issue 2
April 2022
514 pages
ISSN:1556-4681
EISSN:1556-472X
DOI:10.1145/3476120
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 September 2021
Accepted: 01 May 2021
Revised: 01 March 2021
Received: 01 August 2020
Published in TKDD Volume 16, Issue 2

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Heterogeneous information network
  2. network embedding
  3. social network analysis

Qualifiers

  • Research-article
  • Refereed

Funding Sources

  • National Natural Science Foundation of China
  • Fundamental Research Funds for the Central Universities
  • BUPT Excellent Ph.D. Students Foundation

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)48
  • Downloads (Last 6 weeks)1
Reflects downloads up to 17 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2025)Latent representation learning for classification of the Doppler ultrasound imagesComputers in Biology and Medicine10.1016/j.compbiomed.2024.109575185(109575)Online publication date: Feb-2025
  • (2024)BSIN: A Behavior Schema of Information Networks Based on Approximate BisimulationTsinghua Science and Technology10.26599/TST.2023.901008129:4(1092-1104)Online publication date: Aug-2024
  • (2024)Question Embedding on Weighted Heterogeneous Information Network for Knowledge TracingACM Transactions on Knowledge Discovery from Data10.1145/370315819:1(1-28)Online publication date: 10-Dec-2024
  • (2024)Rcoco: contrastive collective link prediction across multiplex network in Riemannian spaceInternational Journal of Machine Learning and Cybernetics10.1007/s13042-024-02118-215:9(3745-3767)Online publication date: 5-Apr-2024
  • (2023)Heterogeneous Network Embedding: A SurveyComputer Modeling in Engineering & Sciences10.32604/cmes.2023.024781137:1(83-130)Online publication date: 2023
  • (2023)Efficient and Effective Academic Expert Finding on Heterogeneous Graphs through (k, 𝒫)-Core based EmbeddingACM Transactions on Knowledge Discovery from Data10.1145/357836517:6(1-35)Online publication date: 22-Mar-2023
  • (2023)Partial Multilabel Learning Using Fuzzy Neighborhood-Based Ball Clustering and Kernel Extreme Learning MachineIEEE Transactions on Fuzzy Systems10.1109/TFUZZ.2022.322294131:7(2277-2291)Online publication date: 1-Jul-2023
  • (2022)Checking for non-Euclidean latent geometry of biological networks2022 IEEE International Conference on Bioinformatics and Biomedicine (BIBM)10.1109/BIBM55620.2022.9995274(2526-2535)Online publication date: 6-Dec-2022

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media