skip to main content
research-article

Attributed Network Embedding with Micro-Meso Structure

Published: 18 April 2021 Publication History

Abstract

Recently, network embedding has received a large amount of attention in network analysis. Although some network embedding methods have been developed from different perspectives, on one hand, most of the existing methods only focus on leveraging the plain network structure, ignoring the abundant attribute information of nodes. On the other hand, for some methods integrating the attribute information, only the lower-order proximities (e.g., microscopic proximity structure) are taken into account, which may suffer if there exists the sparsity issue and the attribute information is noisy. To overcome this problem, the attribute information and mesoscopic community structure are utilized. In this article, we propose a novel network embedding method termed Attributed Network Embedding with Micro-Meso structure, which is capable of preserving both the attribute information and the structural information including the microscopic proximity structure and mesoscopic community structure. In particular, both the microscopic proximity structure and node attributes are factorized by Nonnegative Matrix Factorization (NMF), from which the low-dimensional node representations can be obtained. For the mesoscopic community structure, a community membership strength matrix is inferred by a generative model (i.e., BigCLAM) or modularity from the linkage structure, which is then factorized by NMF to obtain the low-dimensional node representations. The three components are jointly correlated by the low-dimensional node representations, from which two objective functions (i.e., ANEM_B and ANEM_M) can be defined. Two efficient alternating optimization schemes are proposed to solve the optimization problems. Extensive experiments have been conducted to confirm the superior performance of the proposed models over the state-of-the-art network embedding methods.

References

[1]
Zeynep Akata, Christian Thurau, and Christian Bauckhage. 2011. Non-negative matrix factorization in multimodality data for segmentation and label prediction. In Proceedings of the 16th Computer Vision Winter Workshop.
[2]
John D. Burger, John Henderson, George Kim, and Guido Zarrella. 2011. Discriminating gender on Twitter. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. 1301--1309.
[3]
Deng Cai, Xiaofei He, Jiawei Han, and Thomas S. Huang. 2011. Graph regularized nonnegative matrix factorization for data representation. IEEE Transactions on Pattern Analysis and Machine Intelligence 33, 8 (2011), 1548--1560.
[4]
Shaosheng Cao, Wei Lu, and Qiongkai Xu. 2015. GraRep: Learning graph representations with global structural information. In Proceedings of the 24th ACM International on Conference on Information and Knowledge Management. 891--900.
[5]
Dorwin Cartwright and Frank Harary. 1956. Structural balance: A generalization of Heider’s theory. Psychological Review 63, 5 (1956), 277.
[6]
Sandro Cavallari, Vincent W. Zheng, Hongyun Cai, Kevin Chen-Chuan Chang, and Erik Cambria. 2017. Learning community embedding with community detection and node embedding on graphs. In Proceedings of the 2017 ACM International on Conference on Information and Knowledge Management.
[7]
Yukuo Cen, Xu Zou, Jianwei Zhang, Hongxia Yang, Jingren Zhou, and Jie Tang. 2019. Representation learning for attributed multiplex heterogeneous network. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM, 1358--1368.
[8]
Jonathan Chang and David Blei. 2009. Relational topic models for document networks. In Proceedings of the 12th International Conference on Artificial Intelligence and Statistics. 81--88.
[9]
Hongxu Chen, Hongzhi Yin, Weiqing Wang, Hao Wang, Quoc Viet Hung Nguyen, and Xue Li. 2018. PME: Projected metric embedding on heterogeneous networks for link prediction. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. Yike Guo and Faisal Farooq (Eds.). ACM, 1177--1186.
[10]
Siheng Chen, Sufeng Niu, Leman Akoglu, Jelena Kovačević, and Christos Faloutsos. 2017. Fast, warped graph embedding: Unifying framework and one-click algorithm. arXiv preprint arXiv:1702.05764 (2017).
[11]
Fan R. K. Chung and Fan Chung Graham. 1997. Spectral Graph Theory. Number 92. American Mathematical Society.
[12]
Marek Cygan, Marcin Pilipczuk, Michał Pilipczuk, and Jakub Onufry Wojtaszczyk. 2012. Sitting closer to friends than enemies, revisited. In Proceedings of the International Symposium on Mathematical Foundations of Computer Science. Springer, 296--307.
[13]
Hongchang Gao and Heng Huang. 2018. Deep attributed network embedding. In Proceedings of the 27th International Joint Conference on Artificial Intelligence. Vol. 18. New York, NY, 3364--3370.
[14]
Aditya Grover and Jure Leskovec. 2016. node2vec: Scalable feature learning for networks. In Proceedings of the 2016 ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 855--864.
[15]
Xiao Huang, Jundong Li, and Xia Hu. 2017a. Accelerated attributed network embedding. In Proceedings of the 2017 SIAM International Conference on Data Mining. 633--641.
[16]
Xiao Huang, Jundong Li, and Xia Hu. 2017b. Label informed attributed network embedding. In Proceedings of the 10th ACM International Conference on Web Search and Data Mining. 731--739.
[17]
Xiao Huang, Jundong Li, Na Zou, and Xia Hu. 2018a. A general embedding framework for heterogeneous information learning in large-scale networks. ACM Transactions on Knowledge Discovery from Data 12, 6 (2018), 1--24.
[18]
Xiao Huang, Qingquan Song, Jundong Li, and Xia Ben Hu. 2018b. Exploring expert cognition for attributed network embedding. In Proceedings of the 2018 ACM International Conference on Web Search and Data Mining.
[19]
Yi-An Lai, Chin-Chi Hsu, Wen Hao Chen, Mi-Yen Yeh, and Shou-De Lin. 2017. Preserving proximity and global ranking for node embedding. In Proceedings of the 31st International Conference on Neural Information Processing Systems. 5261--5270.
[20]
Tuan M. V. Le and Hady W. Lauw. 2014. Probabilistic latent document network embedding. In Proceedings of the 2014 IEEE International Conference on Data Mining (ICDM’14). IEEE, 270--279.
[21]
Daniel D. Lee and H. Sebastian Seung. 1999. Learning the parts of objects by non-negative matrix factorization. Nature 401, 6755 (1999), 788.
[22]
Daniel D. Lee and H. Sebastian Seung. 2001. Algorithms for non-negative matrix factorization. In Proceedings of the 2001 International Conference on Neural Information Processing Systems. 556--562.
[23]
Chaozhuo Li, Zhoujun Li, Senzhang Wang, Yang Yang, Xiaoming Zhang, and Jianshe Zhou. 2017b. Semi-supervised network embedding. In Proceedings of the International Conference on Database Systems for Advanced Applications. Springer, 131--147.
[24]
Chaozhuo Li, Senzhang Wang, Dejian Yang, Zhoujun Li, Yang Yang, Xiaoming Zhang, and Jianshe Zhou. 2017d. PPNE: Property preserving network embedding. In Proceedings of the International Conference on Database Systems for Advanced Applications. 163--179.
[25]
Hang Li, Haozheng Wang, Zhenglu Yang, and Masato Odagaki. 2017c. Variation autoencoder based network representation learning for classification. In Proceedings of ACL 2017, Student Research Workshop. 56--61.
[26]
Jundong Li, Harsh Dani, Xia Hu, Jiliang Tang, Yi Chang, and Huan Liu. 2017a. Attributed network embedding for learning in a dynamic environment. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management. 387--396.
[27]
Juan-Hui Li, Chang-Dong Wang, Ling Huang, Dong Huang, Jian-Huang Lai, and Pei Chen. 2018. Attributed network embedding with micro-meso structure. In Proceedings of the 23rd International Conference on Database Systems for Advanced Applications. 20--36.
[28]
Lizi Liao, Xiangnan He, Hanwang Zhang, and Tat-Seng Chua. 2018. Attributed social network embedding. IEEE Transactions on Knowledge and Data Engineering 30, 12 (2018), 2257--2270.
[29]
Liyuan Liu, Linli Xu, Zhen Wangy, and Enhong Chen. 2015. Community detection based on structure and content: A content propagation perspective. In Proceedings of the 2015 IEEE International Conference on Data Mining. 271--280.
[30]
Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of Machine Learning Research 9, 86 (2008), 2579--2605.
[31]
Zaiqiao Meng, Shangsong Liang, Hongyan Bao, and Xiangliang Zhang. 2019. Co-embedding attributed networks. In Proceedings of the 2019 ACM International Conference on Web Search and Data Mining. ACM, 393--401.
[32]
Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013).
[33]
Mark E. J. Newman. 2006a. Finding community structure in networks using the eigenvectors of matrices. Physical Review E 74, 3 (2006), 036104.
[34]
Mark E. J. Newman. 2006b. Modularity and community structure in networks. Proceedings of the National Academy of Sciences 103, 23 (2006), 8577--8582.
[35]
Mingdong Ou, Peng Cui, Jian Pei, Ziwei Zhang, and Wenwu Zhu. 2016. Asymmetric transitivity preserving graph embedding. In Proceedings of the 2016 ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1105--1114.
[36]
Shirui Pan, Jia Wu, Xingquan Zhu, Chengqi Zhang, and Yang Wang. 2016. Tri-party deep network representation. In Proceedings of the 2016 International Joint Conference on Artificial Intelligence. 1895--1901.
[37]
Marco Pennacchiotti and Ana-Maria Popescu. 2011. Democrats, republicans and starbucks afficionados: user classification in Twitter. In Proceedings of the 2011 ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 430--438.
[38]
Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. 2014. Deepwalk: Online learning of social representations. In Proceedings of the 2014 ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 701--710.
[39]
Jiezhong Qiu, Yuxiao Dong, Hao Ma, Jian Li, Kuansan Wang, and Jie Tang. 2018. Network embedding as matrix factorization: Unifying deepwalk, line, pte, and node2vec. In Proceedings of the 11th ACM International Conference on Web Search and Data Mining. ACM, 459--467.
[40]
Leonardo F. R. Ribeiro, Pedro H. P. Saverese, and Daniel R. Figueiredo. 2017. struc2vec: Learning node representations from structural identity. In Proceedings of the 2017 ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 385--394.
[41]
Ludovic Dos Santos, Benjamin Piwowarski, Ludovic Denoyer, and Patrick Gallinari. 2018. Representation learning for classification in heterogeneous graphs with application to social networks. ACM Transactions on Knowledge Discovery from Data 12, 5 (2018), 62.
[42]
Xiaofei Sun, Jiang Guo, Xiao Ding, and Ting Liu. 2016. A general framework for content-enhanced network representation learning. arXiv preprint arXiv:1610.02906 (2016).
[43]
Jian Tang, Meng Qu, Mingzhe Wang, Ming Zhang, Jun Yan, and Qiaozhu Mei. 2015. Line: Large-scale information network embedding. In Proceedings of the 24th International Conference on World Wide Web. 1067--1077.
[44]
Joshua B. Tenenbaum, Vin De Silva, and John C. Langford. 2000. A global geometric framework for nonlinear dimensionality reduction. Science 290, 5500 (2000), 2319--2323.
[45]
Cunchao Tu, Weicheng Zhang, Zhiyuan Liu, and Maosong Sun. 2016. Max-margin deepwalk: Discriminative learning of network representation. In Proceedings of the 2016 International Joint Conference on Artificial Intelligence. 3889--3895.
[46]
Ulrike Von Luxburg. 2007. A tutorial on spectral clustering. Statistics and Computing 17, 4 (2007), 395--416.
[47]
Chang-Dong Wang, Wei Shi, Ling Huang, Kun-Yu Lin, Dong Huang, and Philip S. Yu. 2020. Node-pair information preserving network embedding based on adversarial networks. IEEE Transactions on Cybernetics (2020), 1--15.
[48]
Daixin Wang, Peng Cui, and Wenwu Zhu. 2016. Structural deep network embedding. In Proceedings of the 2016 ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1225--1234.
[49]
Hongwei Wang, Fuzheng Zhang, Min Hou, Xing Xie, Minyi Guo, and Qi Liu. 2018. SHINE: Signed heterogeneous information network embedding for sentiment link prediction. In Proceedings of the 2018 ACM International Conference on Web Search and Data Mining.
[50]
Suhang Wang, Jiliang Tang, Charu Aggarwal, Yi Chang, and Huan Liu. 2017b. Signed network embedding in social media. In Proceedings of the 2017 SIAM International Conference on Data Mining. SIAM, 327--335.
[51]
Xiao Wang, Peng Cui, Jing Wang, Jian Pei, Wenwu Zhu, and Shiqiang Yang. 2017a. Community preserving network embedding. In Proceedings of the 31st AAAI Conference on Artificial Intelligence. 203--209.
[52]
Cheng Yang, Zhiyuan Liu, Deli Zhao, Maosong Sun, and Edward Y. Chang. 2015. Network representation learning with rich text information. In Proceedings of the 2015 International Joint Conference on Artificial Intelligence. 2111--2117.
[53]
Dejian Yang, Senzhang Wang, Chaozhuo Li, Xiaoming Zhang, and Zhoujun Li. 2017. From properties to links: Deep network embedding on incomplete graphs. In Proceedings of the 2017 ACM International on Conference on Information and Knowledge Management. 367--376.
[54]
Jaewon Yang and Jure Leskovec. 2013. Overlapping community detection at scale: A nonnegative matrix factorization approach. In Proceedings of the 2013 ACM International Conference on Web Search and Data Mining. 587--596.
[55]
Zhen Zhang, Hongxia Yang, Jiajun Bu, Sheng Zhou, Pinggang Yu, Jianwei Zhang, Martin Ester, and Can Wang. 2018. ANRL: Attributed network representation learning via deep neural networks. In Proceedings of the 2018 International Joint Conference on Artificial Intelligence. ijcai.org, 3155--3161.
[56]
Bin Zhao, Prithviraj Sen, and Lise Getoor. 2006. Event classification and relationship labeling in affiliation networks. In Proceedings of the Workshop on Statistical Network Analysis (SNA) at the 23rd International Conference on Machine Learning.
[57]
Wayne Xin Zhao, Feifan Fan, Ji-Rong Wen, and Edward Y. Chang. 2018. Joint representation learning for location-based social networks with multi-grained sequential contexts. ACM Transactions on Knowledge Discovery from Data 12, 2 (2018), 22.
[58]
Le-kui Zhou, Yang Yang, Xiang Ren, Fei Wu, and Yueting Zhuang. 2018. Dynamic network embedding by modeling triadic closure process. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence, (AAAI’18), the 30th Innovative Applications of Artificial Intelligence (IAAI’18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI’18). Sheila A. McIlraith and Kilian Q. Weinberger (Eds.). AAAI Press, 571--578.

Cited By

View all
  • (2024)Multicomponent Similarity Graphs for Cross-Network Node ClassificationIEEE Transactions on Artificial Intelligence10.1109/TAI.2023.33071055:3(1411-1424)Online publication date: Mar-2024
  • (2024)Detecting communities with multiple topics in attributed networks via self-supervised adaptive graph convolutional networkInformation Fusion10.1016/j.inffus.2024.102254105:COnline publication date: 16-May-2024
  • (2024)Decoupling Anomaly Discrimination and Representation Learning: Self-supervised Learning for Anomaly Detection on Attributed GraphData Science and Engineering10.1007/s41019-024-00249-89:3(264-277)Online publication date: 4-May-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Knowledge Discovery from Data
ACM Transactions on Knowledge Discovery from Data  Volume 15, Issue 4
August 2021
486 pages
ISSN:1556-4681
EISSN:1556-472X
DOI:10.1145/3458847
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 April 2021
Accepted: 01 December 2020
Revised: 01 October 2020
Received: 01 July 2018
Published in TKDD Volume 15, Issue 4

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Network embedding
  2. mesoscopic community structure
  3. microscopic proximity structure
  4. node attribute

Qualifiers

  • Research-article
  • Research
  • Refereed

Funding Sources

  • NSFC
  • Guangdong Natural Science Funds for Distinguished Young Scholar

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)31
  • Downloads (Last 6 weeks)2
Reflects downloads up to 27 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Multicomponent Similarity Graphs for Cross-Network Node ClassificationIEEE Transactions on Artificial Intelligence10.1109/TAI.2023.33071055:3(1411-1424)Online publication date: Mar-2024
  • (2024)Detecting communities with multiple topics in attributed networks via self-supervised adaptive graph convolutional networkInformation Fusion10.1016/j.inffus.2024.102254105:COnline publication date: 16-May-2024
  • (2024)Decoupling Anomaly Discrimination and Representation Learning: Self-supervised Learning for Anomaly Detection on Attributed GraphData Science and Engineering10.1007/s41019-024-00249-89:3(264-277)Online publication date: 4-May-2024
  • (2023)Multi-View Graph Convolutional Networks with Differentiable Node SelectionACM Transactions on Knowledge Discovery from Data10.1145/360895418:1(1-21)Online publication date: 10-Aug-2023
  • (2023)Dual-channel embedding learning model for partially labeled attributed networksPattern Recognition10.1016/j.patcog.2023.109644142(109644)Online publication date: Oct-2023
  • (2023)Community preserving adaptive graph convolutional networks for link prediction in attributed networksKnowledge-Based Systems10.1016/j.knosys.2023.110589272:COnline publication date: 19-Jul-2023
  • (2023)Fuzzy rough dimensionality reduction: A feature set partition-based approachInformation Sciences10.1016/j.ins.2023.119266(119266)Online publication date: Jun-2023
  • (2023)An optimized feature extraction algorithm for abnormal network traffic detectionFuture Generation Computer Systems10.1016/j.future.2023.07.039149(330-342)Online publication date: Dec-2023
  • (2023)Community detection based on unsupervised attributed network embeddingExpert Systems with Applications: An International Journal10.1016/j.eswa.2022.118937213:PAOnline publication date: 1-Mar-2023
  • (2023)Polynomial linear discriminant analysisThe Journal of Supercomputing10.1007/s11227-023-05485-980:1(413-434)Online publication date: 24-Jun-2023
  • Show More Cited By

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media