ABSTRACT
Network embedding, which aims at learning low-dimensional representations of nodes in a network, has drawn much attention for various network mining tasks, ranging from link prediction to node classification. In addition to network topological information, there also exist rich attributes associated with network structure, which exerts large effects on the network formation. Hence, many efforts have been devoted to tackling attributed network embedding tasks. However, they are also limited in their assumption of static network data as they do not account for evolving network structure as well as changes in the associated attributes. Furthermore, scalability is a key factor when performing representation learning on large-scale networks with huge number of nodes and edges. In this work, we address these challenges by developing the DRLAN-Dynamic Representation Learning framework for large-scale Attributed Networks. The DRLAN model generalizes the dynamic attributed network embedding from two perspectives: First, we develop an integrative learning framework with an offline batch embedding module to preserve both the node and attribute proximities, and online network embedding model that recursively updates learned representation vectors. Second, we design a recursive pre-projection mechanism to efficiently model the attribute correlations based on the associative property of matrices. Finally, we perform extensive experiments on three real-world network datasets to show the superiority of DRLAN against state-of-the-art network embedding techniques in terms of both effectiveness and efficiency. The source code is available at: https://github.com/ZhijunLiu95/DRLAN.
Supplemental Material
- Diogo M Camacho, Katherine M Collins, Rani K Powers, James C Costello, and James J Collins. 2018. Next-generation machine learning for biological networks. Cell, Vol. 173, 7 (2018), 1581--1592.Google ScholarCross Ref
- Hongxu Chen, Hongzhi Yin, Weiqing Wang, Hao Wang, Quoc Viet Hung Nguyen, and Xue Li. 2018. PME: projected metric embedding on heterogeneous networks for link prediction. In KDD. ACM, 1177--1186.Google Scholar
- Hongchang Gao and Heng Huang. 2018. Deep Attributed Network Embedding. In IJCAI. 3364--3370.Google Scholar
- Hongyang Gao, Zhengyang Wang, and Shuiwang Ji. 2018. Large-scale learnable graph convolutional networks. In KDD. 1416--1424.Google Scholar
- Aditya Grover and Jure Leskovec. 2016. node2vec: Scalable feature learning for networks. In KDD. ACM, 855--864.Google Scholar
- Chao Huang, Baoxu Shi, Xuchao Zhang, Xian Wu, et al. 2019 a. Similarity-aware network embedding with self-paced learning. In CIKM. 2113--2116.Google Scholar
- Chao Huang, Xian Wu, Xuchao Zhang, Chuxu Zhang, Jiashu Zhao, Dawei Yin, and Nitesh V Chawla. 2019 c. Online purchase prediction via multi-scale modeling of behavior dynamics. In KDD. 2613--2622.Google Scholar
- Xiao Huang, Jundong Li, and Xia Hu. 2017. Label informed attributed network embedding. In WSDM. 731--739.Google Scholar
- Xiao Huang, Qingquan Song, Fan Yang, and Xia Hu. 2019 b. Large-Scale Heterogeneous Feature Embedding. In AAAI. 3878--3885.Google Scholar
- William B Johnson and Joram Lindenstrauss. 1984. Extensions of Lipschitz mappings into a Hilbert space. Contemporary mathematics, Vol. 26, 189--206 (1984), 1.Google Scholar
- Jundong Li, Harsh Dani, Xia Hu, Jiliang Tang, et al. 2017. Attributed network embedding for learning in a dynamic environment. In CIKM. 387--396.Google Scholar
- Ping Li, Trevor J Hastie, and Kenneth W Church. 2006. Very sparse random projections. In KDD. ACM, 287--296.Google Scholar
- Ruirui Li, Xian Wu, Xian Wu, and Wei Wang. 2020. Few-Shot Learning for New User Recommendation in Location-based Social Networks. In WWW. 2472--2478.Google Scholar
- Lizi Liao, Xiangnan He, Hanwang Zhang, and Tat-Seng Chua. 2018. Attributed social network embedding. TKDE, Vol. 30, 12 (2018), 2257--2270.Google ScholarDigital Library
- Linyuan Lü and Tao Zhou. 2011. Link prediction in complex networks: A survey. Physica A: statistical mechanics and its applications, Vol. 390, 6 (2011), 1150--1170.Google Scholar
- Yuanfu Lu, Xiao Wang, Chuan Shi, Philip S Yu, and Yanfang Ye. 2019. Temporal Network Embedding with Micro-and Macro-dynamics. In CIKM. 469--478.Google Scholar
- Zaiqiao Meng, Shangsong Liang, Hongyan Bao, and Xiangliang Zhang. 2019. Co-embedding Attributed Networks. In WSDM. ACM, 393--401.Google Scholar
- Giang Hoang Nguyen, John Boaz Lee, Ryan A Rossi, Nesreen K Ahmed, Eunyee Koh, and Sungchul Kim. 2018. Continuous-time dynamic network embeddings. In Companion Proceedings of the The Web Conference. 969--976.Google ScholarDigital Library
- Mingdong Ou, Peng Cui, Jian Pei, Ziwei Zhang, and Wenwu Zhu. 2016. Asymmetric transitivity preserving graph embedding. In KDD. ACM, 1105--1114.Google Scholar
- Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. 2014. Deepwalk: Online learning of social representations. In KDD. 701--710.Google ScholarDigital Library
- Jiezhong Qiu, Yuxiao Dong, Hao Ma, Jian Li, Chi Wang, Kuansan Wang, and Jie Tang. 2019. NetSMF: Large-Scale Network Embedding as Sparse Matrix Factorization. In WWW. 1509--1520.Google Scholar
- Aravind Sankar, Yanhong Wu, Liang Gou, Wei Zhang, and Hao Yang. 2020. DySAT: Deep Neural Representation Learning on Dynamic Graphs via Self-Attention Networks. In WSDM. 519--527.Google Scholar
- Yiwei Sun, Suhang Wang, Tsung-Yu Hsieh, Xianfeng Tang, and Vasant Honavar. 2019. Megan: A generative adversarial network for multi-view network embedding. In IJCAI. 3527--3533.Google Scholar
- Jian Tang, Meng Qu, Mingzhe Wang, Ming Zhang, Jun Yan, and Qiaozhu Mei. 2015. Line: Large-scale information network embedding. In WWW. 1067--1077.Google ScholarDigital Library
- Xianfeng Tang, Boqing Gong, Yanwei Yu, Huaxiu Yao, Yandong Li, Haiyong Xie, and Xiaoyu Wang. 2019. Joint modeling of dense and incomplete trajectories for citywide traffic volume inference. In WWW. 1806--1817.Google Scholar
- Rakshit Trivedi, Mehrdad Farajtabar, Prasenjeet Biswal, and Hongyuan Zha. 2019. DyRep: Learning Representations over Dynamic Graphs. In ICLR.Google Scholar
- Cunchao Tu, Han Liu, Zhiyuan Liu, and Maosong Sun. 2017. Cane: Context-aware network embedding for relation modeling. In ACL. 1722--1731.Google Scholar
- Wei Wu, Bin Li, Ling Chen, and Chengqi Zhang. 2018. Efficient Attributed Network Embedding via Recursive Randomized Hashing. In IJCAI. 2861--2867.Google Scholar
- Cheng Yang, Zhiyuan Liu, Deli Zhao, Maosong Sun, and Edward Chang. 2015. Network representation learning with rich text information. In IJCAI. 2111--2117.Google Scholar
- Cheng Yang, Maosong Sun, et al. 2017. Fast Network Embedding Enhancement via High Order Proximity Approximation. In IJCAI. 3894--3900.Google Scholar
- Yanwei Yu, Xianfeng Tang, Huaxiu Yao, Xiuwen Yi, and Zhenhui Li. 2019. Citywide Traffic Volume Inference with Surveillance Camera Records. IEEE Transactions on Big Data (2019).Google Scholar
- Yanwei Yu, Hongjian Wang, and Zhenhui Li. 2018a. Inferring mobility relationship via graph embedding. IMWUT, Vol. 2, 3 (2018), 1--21.Google Scholar
- Yanwei Yu, Huaxiu Yao, Hongjian Wang, Xianfeng Tang, and Zhenhui Li. 2018b. Representation learning for large-scale dynamic networks. In DASFAA. Springer, 526--541.Google Scholar
- Chuxu Zhang, Dongjin Song, Chao Huang, Ananthram Swami, and Nitesh V Chawla. 2019. Heterogeneous graph neural network. In KDD. 793--803.Google Scholar
- Ziwei Zhang, Peng Cui, Haoyang Li, Xiao Wang, and Wenwu Zhu. 2018a. Billion-scale Network Embedding with Iterative Random Projection. In ICDM. IEEE, 787--796.Google Scholar
- Ziwei Zhang, Peng Cui, Jian Pei, Xiao Wang, and Wenwu Zhu. 2018b. Timers: Error-bounded svd restart on dynamic networks. In AAAI.Google Scholar
- Lekui Zhou, Yang Yang, Xiang Ren, Fei Wu, and Yueting Zhuang. 2018. Dynamic network embedding by modeling triadic closure process. In AAAI.Google Scholar
- Dingyuan Zhu, Peng Cui, Ziwei Zhang, Jian Pei, and Wenwu Zhu. 2018. High-order proximity preserved embedding for dynamic networks. TKDE, Vol. 30, 11 (2018), 2134--2144.Google ScholarDigital Library
Index Terms
Dynamic Representation Learning for Large-Scale Attributed Networks
Recommendations
Fast Attributed Multiplex Heterogeneous Network Embedding
CIKM '20: Proceedings of the 29th ACM International Conference on Information & Knowledge ManagementIn recent years, heterogeneous network representation learning has attracted considerable attentions with the consideration of multiple node types. However, most of them ignore the rich set of network attributes (attributed network) and different types ...
Motif-Preserving Dynamic Attributed Network Embedding
WWW '21: Proceedings of the Web Conference 2021Network embedding has emerged as a new learning paradigm to embed complex network into a low-dimensional vector space while preserving node proximities in both network structures and properties. It advances various network mining tasks, ranging from ...
Attributed Network Embedding for Learning in a Dynamic Environment
CIKM '17: Proceedings of the 2017 ACM on Conference on Information and Knowledge ManagementNetwork embedding leverages the node proximity manifested to learn a low-dimensional node vector representation for each node in the network. The learned embeddings could advance various learning tasks such as node classification, network clustering, ...
Comments