skip to main content
10.1145/3488560.3498533acmconferencesArticle/Chapter ViewAbstractPublication PageswsdmConference Proceedingsconference-collections
research-article

Efficient Graph Convolution for Joint Node Representation Learning and Clustering

Published: 15 February 2022 Publication History

Abstract

Attributed graphs are used to model a wide variety of real-world networks. Recent graph convolutional network-based representation learning methods have set state-of-the-art results on the clustering of attributed graphs. However, these approaches deal with clustering as a downstream task while better performances can be attained by incorporating the clustering objective into the representation learning process. In this paper, we propose, in a unified framework, an objective function taking into account both tasks simultaneously. Based on a variant of the simple graph convolutional network, our model does clustering by minimizing the difference between the convolved node representations and their reconstructed cluster representatives. We showcase the efficiency of the derived algorithm against state-of-the-art methods both in terms of clustering performance and computational cost on thede facto benchmark graph clustering datasets. We further demonstrate the usefulness of the proposed approach for graph visualization through generating embeddings that exhibit a clustering structure.

Supplementary Material

MP4 File (WSDM22-fp493.mp4)
This is a video presentation for the paper "Efficient Graph Convolution for Joint Node Representation Learning and Clustering", which has been accepted in WSDM'22.

References

[1]
Amr Ahmed, Nino Shervashidze, Shravan Narayanamurthy, Vanja Josifovski, and Alexander J Smola. 2013. Distributed large-scale natural graph factorization. In Proceedings of the 22nd international conference on World Wide Web. 37--48.
[2]
Kais Allab, Lazhar Labiod, and Mohamed Nadif. 2016. A semi-NMF-PCA unified framework for data clustering. IEEE Transactions on Knowledge and Data Engineering, Vol. 29, 1 (2016), 2--16.
[3]
Kais Allab, Lazhar Labiod, and Mohamed Nadif. 2018. Simultaneous spectral data embedding and clustering. IEEE transactions on neural networks and learning systems, Vol. 29, 12 (2018), 6396--6401.
[4]
Jinwon An and Sungzoon Cho. 2015. Variational autoencoder based anomaly detection using reconstruction probability. Special Lecture on IE, Vol. 2, 1 (2015), 1--18.
[5]
Bryan Perozzi Anton Tsitsulin, John Palowitch and Emmanuel Müller. 2020. Graph Clustering with Graph Neural Networks. In Proceedings of the 16th International Workshop on Mining and Learning with Graphs (MLG) .
[6]
Yuki M. Asano, Christian Rupprecht, and Andrea Vedaldi. 2020. Self-labelling via simultaneous clustering and representation learning. In International Conference on Learning Representations (ICLR) .
[7]
Mikhail Belkin and Partha Niyogi. 2003. Laplacian Eigenmaps for Dimensionality Reduction and Data Representation. Neural Computation, Vol. 15, 6 (2003), 1373--1396.
[8]
Ganqu Cui, Jie Zhou, Cheng Yang, and Zhiyuan Liu. 2020. Adaptive graph encoder for attributed graph embedding. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining . 976--985.
[9]
Geert De Soete and J Douglas Carroll. 1994. K-means clustering in a low-dimensional Euclidean space. In New approaches in classification and data analysis. Springer, 212--219.
[10]
Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. 2016. Convolutional neural networks on graphs with fast localized spectral filtering. Advances in neural information processing systems, Vol. 29 (2016), 3844--3852.
[11]
Inderjit S. Dhillon, Yuqiang Guan, and Brian Kulis. 2007. Weighted Graph Cuts without Eigenvectors A Multilevel Approach. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 29, 11 (2007), 1944--1957.
[12]
Wenqi Fan, Yao Ma, Qing Li, Yuan He, Eric Zhao, Jiliang Tang, and Dawei Yin. 2019. Graph Neural Networks for Social Recommendation. In The World Wide Web Conference (San Francisco, CA, USA) (WWW '19). Association for Computing Machinery, New York, NY, USA, 417--426. https://doi.org/10.1145/3308558.3313488
[13]
Maziar Moradi Fard, Thibaut Thonet, and Eric Gaussier. 2020. Deep k-means: Jointly clustering with k-means and learning representations. Pattern Recognition Letters, Vol. 138 (2020), 185--192.
[14]
Kamran Ghasedi Dizaji, Amirhossein Herandi, Cheng Deng, Weidong Cai, and Heng Huang. 2017. Deep clustering via joint convolutional autoencoder embedding and relative entropy minimization. In Proceedings of the IEEE international conference on computer vision. 5736--5745.
[15]
Aditya Grover and Jure Leskovec. 2016. node2vec: Scalable feature learning for networks. In Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining . 855--864.
[16]
Nathan Halko, Per-Gunnar Martinsson, and Joel A Tropp. 2011. Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions. SIAM review, Vol. 53, 2 (2011), 217--288.
[17]
William L. Hamilton, Rex Ying, and Jure Leskovec. 2017. Representation Learning on Graphs: Methods and Applications. IEEE Data Eng. Bull., Vol. 40, 3 (2017), 52--74.
[18]
Kurt Hornik, Ingo Feinerer, Martin Kober, and Christian Buchta. 2012. Spherical k-Means Clustering. Journal of Statistical Software, Articles, Vol. 50, 10 (2012), 1--22.
[19]
Yedid Hoshen. 2017. VAIN: Attentional Multi-agent Predictive Modeling. In Neural Information Processing Systems (NIPS). 2701--2711.
[20]
Md Rezaul Karim, Oya Beyan, Achille Zappa, Ivan G Costa, Dietrich Rebholz-Schuhmann, Michael Cochez, and Stefan Decker. 2020 a. Deep learning-based clustering approaches for bioinformatics. Briefings in Bioinformatics (2020), 1--23.
[21]
Md Rezaul Karim, Michael Cochez, Achille Zappa, Ratnesh Sahay, Dietrich Rebholz-Schuhmann, Oya Beyan, and Stefan Decker. 2020 b. Convolutional Embedded Networks for Population Scale Clustering and Bio-ancestry Inferencing. IEEE/ACM Transactions on Computational Biology and Bioinformatics (2020).
[22]
Thomas N Kipf and Max Welling. 2016. Variational Graph Auto-Encoders. NIPS Workshop on Bayesian Deep Learning (2016).
[23]
Thomas N. Kipf and Max Welling. 2017. Semi-Supervised Classification with Graph Convolutional Networks. In International Conference on Learning Representations (ICLR) .
[24]
Lazhar Labiod and Mohamed Nadif. 2021. Efficient regularized spectral data embedding. Advances in Data Analysis and Classification, Vol. 15, 1 (2021), 99--119.
[25]
S. P. Lloyd. 1982. Least squares quantization in PCM. IEEE Trans. Inf. Theory, Vol. 28 (1982), 129--136.
[26]
Diego Marcheggiani and Ivan Titov. 2017. Encoding Sentences with Graph Convolutional Networks for Semantic Role Labeling. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Copenhagen, Denmark, 1506--1515.
[27]
Costas Mavromatis and George Karypis. 2021. Graph InfoClust: Maximizing Coarse-Grain Mutual Information in Graphs. In PAKDD (1) . 541--553.
[28]
Andrew Y. Ng, Michael I. Jordan, and Yair Weiss. 2001. On Spectral Clustering: Analysis and an Algorithm. In International Conference on Neural Information Processing Systems: Natural and Synthetic. 849--856.
[29]
Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. 2014. DeepWalk: online learning of social representations. In The 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining . 701--710.
[30]
Xiaojuan Qi, Renjie Liao, Jiaya Jia, Sanja Fidler, and Raquel Urtasun. 2017. 3D Graph Neural Networks for RGBD Semantic Segmentation. In 2017 IEEE International Conference on Computer Vision (ICCV). 5209--5218.
[31]
Benedek Rozemberczki, Ryan Davies, Rik Sarkar, and Charles Sutton. 2019. Gemsec: Graph embedding with self clustering. In Proceedings of the 2019 IEEE/ACM international conference on advances in social networks analysis and mining . 65--72.
[32]
Aghiles Salah and Mohamed Nadif. 2017. Social regularized von Mises--Fisher mixture model for item recommendation. Data Mining and Knowledge Discovery, Vol. 31, 5 (2017), 1218--1241.
[33]
Guillaume Salha, Romain Hennequin, and Michalis Vazirgiannis. 2020. Simple and Effective Graph Autoencoders with One-Hop Linear Models. In European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML-PKDD). 319--334.
[34]
Alvaro Sanchez-Gonzalez, Nicolas Heess, Jost Tobias Springenberg, Josh Merel, Martin Riedmiller, Raia Hadsell, and Peter Battaglia. 2018. Graph networks as learnable physics engines for inference and control. In International Conference on Machine Learning. 4470--4479.
[35]
Victor Garcia Satorras and Joan Bruna Estrach. 2018. Few-Shot Learning with Graph Neural Networks. In International Conference on Learning Representations . https://openreview.net/forum?id=BJj6qGbRW
[36]
Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi-Rad. 2008. Collective classification in network data. AI magazine, Vol. 29, 3 (2008), 93--93.
[37]
Linfeng Song, Yue Zhang, Zhiguo Wang, and Daniel Gildea. 2018. A Graph-to-Sequence Model for AMR-to-Text Generation. In the Association for Computational Linguistics, ACL 2018. 1616--1626.
[38]
Fan-Yun Sun, Jordan Hoffman, Vikas Verma, and Jian Tang. 2020. InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximization. In International Conference on Learning Representations . https://openreview.net/forum?id=r1lfF2NYvH
[39]
Laurens van der Maaten and Geoffrey E. Hinton. 2008. Visualizing High-Dimensional Data Using t-SNE. Journal of Machine Learning Research, Vol. 9 (2008), 2579--2605.
[40]
Petar Velickovic, William Fedus, William L Hamilton, Pietro Liò, Yoshua Bengio, and R Devon Hjelm. 2019. Deep Graph Infomax. ICLR (Poster), Vol. 2, 3 (2019), 4.
[41]
Daixin Wang, Peng Cui, and Wenwu Zhu. 2016. Structural deep network embedding. In Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining . 1225--1234.
[42]
Felix Wu, Amauri Souza, Tianyi Zhang, Christopher Fifty, Tao Yu, and Kilian Weinberger. 2019. Simplifying graph convolutional networks. In International conference on machine learning . 6861--6871.
[43]
Michio Yamamoto and Heungsun Hwang. 2014. A general formulation of cluster analysis with dimension reduction and subspace separation. Behaviormetrika, Vol. 41, 1 (2014), 115--129.
[44]
Bo Yang, Xiao Fu, Nicholas D. Sidiropoulos, and Mingyi Hong. 2017. Towards K-means-friendly Spaces: Simultaneous Deep Learning and Clustering. In Proceedings of the 34th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 70), Doina Precup and Yee Whye Teh (Eds.). 3861--3870.
[45]
Cheng Yang, Zhiyuan Liu, Deli Zhao, Maosong Sun, and Edward Y. Chang. 2015. Network Representation Learning with Rich Text Information. In IJCAI .
[46]
Jianwei Yang, Jiasen Lu, Stefan Lee, Dhruv Batra, and Devi Parikh. 2018. Graph R-CNN for Scene Graph Generation., bibinfonumpages690--706 pages.
[47]
Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L Hamilton, and Jure Leskovec. 2018. Graph convolutional neural networks for web-scale recommender systems. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining . 974--983.
[48]
Xiaotong Zhang, Han Liu, Qimai Li, and Xiao-Ming Wu. 2019. Attributed Graph Clustering via Adaptive Graph Convolution. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19. International Joint Conferences on Artificial Intelligence Organization, 4327--4333.
[49]
Hao Zhu and Piotr Koniusz. 2021. Simple Spectral Graph Convolution. In 9th International Conference on Learning Representations, ICLR, Virtual Event, Austria, May 3--7, 2021 . OpenReview.net.
[50]
Wenwu Zhu, Xin Wang, and Peng Cui. 2020. Deep learning for learning graph representations. In Deep Learning: Concepts and Architectures. Springer, 169--210.

Cited By

View all
  • (2025)Multi-view attributed graph clustering based on graph diffusion convolution with adaptive fusionExpert Systems with Applications10.1016/j.eswa.2024.125286260(125286)Online publication date: Jan-2025
  • (2024)Generative and Contrastive Paradigms Are Complementary for Graph Self-Supervised Learning2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00234(3364-3378)Online publication date: 13-May-2024
  • (2024)Unveiling community structures in static networks through graph variational Bayes with evolution informationNeurocomputing10.1016/j.neucom.2024.127349576:COnline publication date: 25-Jun-2024
  • Show More Cited By

Index Terms

  1. Efficient Graph Convolution for Joint Node Representation Learning and Clustering

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      WSDM '22: Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining
      February 2022
      1690 pages
      ISBN:9781450391320
      DOI:10.1145/3488560
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 15 February 2022

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. graph convolutional networks
      2. node clustering
      3. node embedding

      Qualifiers

      • Research-article

      Conference

      WSDM '22

      Acceptance Rates

      Overall Acceptance Rate 498 of 2,863 submissions, 17%

      Upcoming Conference

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)173
      • Downloads (Last 6 weeks)12
      Reflects downloads up to 13 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2025)Multi-view attributed graph clustering based on graph diffusion convolution with adaptive fusionExpert Systems with Applications10.1016/j.eswa.2024.125286260(125286)Online publication date: Jan-2025
      • (2024)Generative and Contrastive Paradigms Are Complementary for Graph Self-Supervised Learning2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00234(3364-3378)Online publication date: 13-May-2024
      • (2024)Unveiling community structures in static networks through graph variational Bayes with evolution informationNeurocomputing10.1016/j.neucom.2024.127349576:COnline publication date: 25-Jun-2024
      • (2024)A unified framework of semi-supervised community detection integrating network topology and node contentInformation Sciences10.1016/j.ins.2024.121349(121349)Online publication date: Aug-2024
      • (2024)ELSNC: A Semi-supervised Community Detection Method with Integration of Embedding-Enhanced Links and Node Content in Attributed NetworksApplied Soft Computing10.1016/j.asoc.2024.112250(112250)Online publication date: Sep-2024
      • (2024)A Multi-Embedding Fusion Network for attributed graph clusteringApplied Soft Computing10.1016/j.asoc.2024.112073165:COnline publication date: 1-Nov-2024
      • (2024)Graph analysis using a GPU-based parallel algorithm: quantum clusteringApplied Intelligence10.1007/s10489-024-05587-854:17-18(7765-7776)Online publication date: 14-Jun-2024
      • (2024)Hybrid Graph Representation Learning: Integrating Euclidean and Hyperbolic SpacePattern Recognition10.1007/978-3-031-78189-6_18(276-290)Online publication date: 11-Dec-2024
      • (2023)CONGREGATEProceedings of the Thirty-Second International Joint Conference on Artificial Intelligence10.24963/ijcai.2023/255(2296-2305)Online publication date: 19-Aug-2023
      • (2023)Homophily-enhanced Structure Learning for Graph ClusteringProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3614915(577-586)Online publication date: 21-Oct-2023
      • Show More Cited By

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media