Abstract
Network embedding is an emerging research field which aims at projecting network elements into lower dimensional spaces. However, most network embedding algorithms focus on homogeneous networks, thus cannot be directly applied to the Heterogeneous Information Networks (HINs) which are prevalent in real world systems. Therefore, how to effectively preserve both the structural and semantic information, as well as higher-level proximity of large-scale HINs still remains an open problem. In this paper, we propose a novel heterogeneous network embedding model called Meta-path based Graph ATtention nEtwork (MetaGATE). To tackle the problem of preserving both structural and semantic information for HINs, we adopt a multiple meta-paths based random walk scheme with Skip-Gram for sampling and pre-training. In addition, the model involves graph attention networks to collect and aggregate heterogeneous information, which may reveal higher-level implicit semantics. Besides, to ensure stability, multi-head attention mechanism is employed in the graph attention model by concatenating the embedding of multiple independent self-attention processes. Experiments on two real-world bibliography networks are conducted. Compared to state-of-the-art baselines, our method achieves the best performance on the node classification task, which shows the effectiveness of the proposed model.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
- 3.
- 4.
The first method is to randomly-initialize feature vectors with node type encoded as binaries (e.g., for node type “P”, we encode [1 0 0] as the first three bits of the feature vector). The second method is to assign random values only to part of the feature vector, the rest are set to zeros (e.g., for node type “P”, we only assign non-zero values to the first 32 bits of feature vector).
- 5.
References
Bengio, Y., Courville, A., Vincent, P.: Representation learning: a review and new perspectives, vol. 35, pp. 1798–1828. IEEE (2013). https://doi.org/10.1109/TPAMI.2013.50
Chen, J., Ma, T., Xiao, C.: FastGCN: fast learning with graph convolutional networks via importance sampling. arXiv preprint arXiv:1801.10247 (2018)
Dong, Y., Chawla, N.V., Swami, A.: metapath2vec: scalable representation learning for heterogeneous networks. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 135–144. ACM (2017). https://doi.org/10.1145/3097983.3098036
Fu, T.Y., Lee, W.C., Lei, Z.: Hin2Vec: explore meta-paths in heterogeneous information networks for representation learning. In: Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, pp. 1797–1806. ACM (2017). https://doi.org/10.1145/3132847.3132953
Grover, A., Leskovec, J.: node2vec: scalable feature learning for networks. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 855–864. ACM (2016). https://doi.org/10.1145/2939672.2939754
Hamilton, W., Ying, Z., Leskovec, J.: Inductive representation learning on large graphs. In: Advances in Neural Information Processing Systems, pp. 1024–1034 (2017)
Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016)
Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Advances in Neural Information Processing Systems, pp. 3111–3119 (2013)
Perozzi, B., Al-Rfou, R., Skiena, S.: DeepWalk: online learning of social representations. In: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 701–710. ACM (2014). https://doi.org/10.1145/2623330.2623732
Tang, J., Qu, M., Wang, M., Zhang, M., Yan, J., Mei, Q.: Line: large-scale information network embedding. In: Proceedings of the 24th International Conference on World Wide Web, pp. 1067–1077. International World Wide Web Conferences Steering Committee (2015). https://doi.org/10.1145/2736277.2741093
Veličković, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., Bengio, Y.: Graph attention networks. In: ICLR 2018 (2018)
Acknowledgments
This paper is supported by the National Key Research and Development Program of China (Grant No. 2018YFB1403400), the National Natural Science Foundation of China (Grant No. 61876080), the Collaborative Innovation Center of Novel Software Technology and Industrialization at Nanjing University.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Cao, M., Ma, X., Xu, M., Wang, C. (2019). Heterogeneous Information Network Embedding with Meta-path Based Graph Attention Networks. In: Tetko, I., Kůrková, V., Karpov, P., Theis, F. (eds) Artificial Neural Networks and Machine Learning – ICANN 2019: Workshop and Special Sessions. ICANN 2019. Lecture Notes in Computer Science(), vol 11731. Springer, Cham. https://doi.org/10.1007/978-3-030-30493-5_57
Download citation
DOI: https://doi.org/10.1007/978-3-030-30493-5_57
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-30492-8
Online ISBN: 978-3-030-30493-5
eBook Packages: Computer ScienceComputer Science (R0)