research-article

Dropping Pathways Towards Deep Multi-View Graph Subspace Clustering Networks

Authors:

Wei FengAuthors Info & Claims

MM '23: Proceedings of the 31st ACM International Conference on Multimedia

Pages 3259 - 3267

https://doi.org/10.1145/3581783.3612332

Published: 27 October 2023 Publication History

Abstract

Multi-view graph clustering aims to leverage different views to obtain consistent information and improve clustering performance by sharing the graph structure. Existing multi-view graph clustering algorithms generally adopt a single-pathway network reconstruction and consistent feature extraction, building on top of auto-encoders and graph convolutional networks (GCN). Despite their promising results, these single-pathway methods may ignore the significant complementary information between different layers and the rich multi-level context inside. On the other hand, GCN usually employs a shallow network structure (2-3 layers) due to the over-smoothing with the increase of network depth, while few multi-view graph clustering methods explore the performance of deep networks. In this work, we propose a novel Dropping Pathways strategy toward building a deep Multi-view Graph Subspace Clustering network, namely DPMGSC, to fully exploit the deep and multi-level graph network representations. The proposed method implements a multi-pathway self-expressive network to capture pairwise affinities of graph nodes among multiple views. Moreover, we empirically study the impact of a series of dropping methods on deep multi-pathway networks. Extensive experiments demonstrate the effectiveness of the proposed DPMGSC compared with its deep counterpart and state-of-the-art methods.

References

[1]

Galen Andrew, Raman Arora, Jeff Bilmes, and Karen Livescu. 2013. Deep canonical correlation analysis. In ICML. PMLR, 1247--1255.

[2]

Vladimir Batagelj. 2003. Efficient Algorithms for Citation Network Analysis. CoRR, Vol. cs.DL/0309023 (2003).

[3]

Mikhail Belkin and Partha Niyogi. 2001. Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering. In NIPS. MIT Press, 585--591.

[4]

Deyu Bo, Xiao Wang, Chuan Shi, Meiqi Zhu, Emiao Lu, and Peng Cui. 2020. Structural deep clustering network. In Web Conference. 1400--1410.

Digital Library

[5]

Shaosheng Cao, Wei Lu, and Qiongkai Xu. 2015. Grarep: Learning graph representations with global structural information. In ACM CIKM. 891--900.

Digital Library

[6]

Ming Chen, Zhewei Wei, Zengfeng Huang, Bolin Ding, and Yaliang Li. 2020. Simple and deep graph convolutional networks. In ICML. PMLR, 1725--1735.

[7]

Jiafeng Cheng, Qianqian Wang, Zhiqiang Tao, Deyan Xie, and Quanxue Gao. 2021. Multi-view attribute graph convolution networks for clustering. In IJCAI. 2973--2979.

[8]

Peter Csermely, Tamás Korcsmáros, Huba JM Kiss, Gábor London, and Ruth Nussinov. 2013. Structure and dynamics of molecular networks: a novel paradigm of drug discovery: a comprehensive review. Pharmacology & therapeutics, Vol. 138, 3 (2013), 333--408.

[9]

Robert Duin. [n.,d.]. Multiple Features. UCI Machine Learning Repository.

[10]

Claudio Gallicchio and Alessio Micheli. 2020. Fast and Deep Graph Neural Networks. In AAAI. AAAI Press, 3898--3905.

[11]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In IEEE CVPR. 770--778.

[12]

Ruiqi Hu, Shirui Pan, Guodong Long, Xingquan Zhu, Jing Jiang, and Chengqi Zhang. 2016. Co-clustering enterprise social networks. In IJCNN. IEEE, 107--114.

[13]

Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In ICLR.

[14]

Thomas N Kipf and Max Welling. 2016. Variational Graph Auto-Encoders. NIPS Workshop on Bayesian Deep Learning (2016).

[15]

Thomas N. Kipf and Max Welling. 2017. Semi-Supervised Classification with Graph Convolutional Networks. In ICLR. OpenReview.net.

[16]

Guohao Li, Matthias Muller, Ali Thabet, and Bernard Ghanem. 2019. Deepgcns: Can gcns go as deep as cnns?. In IEEE ICCV. 9267--9276.

[17]

Qimai Li, Zhichao Han, and Xiao-Ming Wu. 2018. Deeper Insights Into Graph Convolutional Networks for Semi-Supervised Learning. In AAAI. AAAI Press, 3538--3545.

[18]

Shu Li, Wen-Tao Li, and Wei Wang. 2020. Co-gcn for multi-view semi-supervised learning. In AAAI, Vol. 34. 4691--4698.

[19]

James MacQueen. 1967. Some methods for classification and analysis of multivariate observations. In Mathematical Statistics and Probability. 281--297.

[20]

Shirui Pan, Ruiqi Hu, Guodong Long, Jing Jiang, Lina Yao, and Chengqi Zhang. 2018. Adversarially Regularized Graph Autoencoder for Graph Embedding. In IJCAI. ijcai.org, 2609--2615.

[21]

Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. 2014. Deepwalk: Online learning of social representations. In ACM SIGKDD. 701--710.

Digital Library

[22]

Yu Rong, Wenbing Huang, Tingyang Xu, and Junzhou Huang. 2020. DropEdge: Towards Deep Graph Convolutional Networks on Node Classification. In ICLR.

[23]

Amin Salehi and Hasan Davulcu. 2020. Graph Attention Auto-Encoders. In IEEE ICTAI. IEEE, 989--996.

[24]

Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. 2008. The graph neural network model. IEEE TNN, Vol. 20, 1 (2008), 61--80.

[25]

Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi-Rad. 2008. Collective classification in network data. AI magazine, Vol. 29, 3 (2008), 93--93.

[26]

Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research, Vol. 15, 1 (2014), 1929--1958.

[27]

Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018. Graph Attention Networks. In ICLR. OpenReview.net.

[28]

Chun Wang, Shirui Pan, Ruiqi Hu, Guodong Long, Jing Jiang, and Chengqi Zhang. 2019b. Attributed Graph Clustering: A Deep Attentional Embedding Approach. In IJCAI. ijcai.org, 3670--3676.

[29]

Qianqian Wang, Zhengming Ding, Zhiqiang Tao, Quanxue Gao, and Yun Fu. 2018. Partial multi-view clustering via consistent GAN. In IEEE ICDM. 1290--1295.

[30]

Qianqian Wang, Zhengming Ding, Zhiqiang Tao, Quanxue Gao, and Yun Fu. 2021. Generative partial multi-view clustering with adaptive fusion and cycle consistency. IEEE TIP, Vol. 30 (2021), 1771--1783.

[31]

Weiran Wang, Raman Arora, Karen Livescu, and Jeff Bilmes. 2015. On deep multi-view representation learning. In ICML. PMLR, 1083--1092.

[32]

Xiao Wang, Houye Ji, Chuan Shi, Bai Wang, Yanfang Ye, Peng Cui, and Philip S. Yu. 2019a. Heterogeneous Graph Attention Network. In WWW. ACM, 2022--2032.

[33]

Yiming Wang, Dongxia Chang, Zhiqiang Fu, and Yao Zhao. 2023. Consistent Multiple Graph Embedding for Multi-View Clustering. IEEE TMM, Vol. 25 (2023), 1008--1018.

Digital Library

[34]

Bo Wu, Yang Liu, Bo Lang, and Lei Huang. 2018. DGCNN: Disordered graph convolutional neural network based on the Gaussian mixture model. Neurocomputing, Vol. 321 (2018), 346--356.

[35]

Wei Xia, Qianqian Wang, Quanxue Gao, Xiangdong Zhang, and Xinbo Gao. 2021. Self-supervised graph convolutional network for multi-view clustering. IEEE TMM, Vol. 24 (2021), 3182--3192.

[36]

Cheng Yang, Zhiyuan Liu, Deli Zhao, Maosong Sun, and Edward Y Chang. 2015. Network representation learning with rich text information. In IJCAI, Vol. 2015. 2111--2117.

[37]

Changqing Zhang, Qinghua Hu, Huazhu Fu, Pengfei Zhu, and Xiaochun Cao. 2017. Latent multi-view subspace clustering. In IEEE CVPR. 4279--4287.

[38]

Hongyuan Zhang, Pei Li, Rui Zhang, and Xuelong Li. 2022. Embedding graph auto-encoder for graph clustering. IEEE TNNLS (2022), 1--11.

[39]

Ronghang Zhu, Zhiqiang Tao, Yaliang Li, and Sheng Li. 2021. Automated graph learning via population based self-tuning GCN. In ACM SIGIR. 2096--2100.

Cited By

Lin YLiu HYu XZhang C(2025)Leveraging Transformer-based autoencoders for low-rank multi-view subspace clusteringPattern Recognition10.1016/j.patcog.2024.111331(111331)Online publication date: Jan-2025
https://doi.org/10.1016/j.patcog.2024.111331
Li JGao QWang QDeng CXie DBaeza-Yates RBonchi F(2024)Label Learning Method Based on Tensor ProjectionProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671671(1599-1609)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3671671
Qin YQin CZhang XFeng G(2024)Dual Consensus Anchor Learning for Fast Multi-View ClusteringIEEE Transactions on Image Processing10.1109/TIP.2024.345965133(5298-5311)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TIP.2024.3459651
Show More Cited By

Index Terms

Dropping Pathways Towards Deep Multi-View Graph Subspace Clustering Networks
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Unsupervised learning
        Cluster analysis
    2. Machine learning approaches
      1. Learning latent representations

Recommendations

Deep Multi-view Sparse Subspace Clustering
ICNCC '18: Proceedings of the 2018 VII International Conference on Network, Communication and Computing

Most multi-view subspace clustering algorithms construct the affinity matrix with shallow features extracted from each view separately. The integration of multi-view features are left for extended spectral clustering algorithm. The lack of deep feature ...
Multi-view Clustering with Graph Embedding for Connectome Analysis
CIKM '17: Proceedings of the 2017 ACM on Conference on Information and Knowledge Management

Multi-view clustering has become a widely studied problem in the area of unsupervised learning. It aims to integrate multiple views by taking advantages of the consensus and complimentary information from multiple views. Most of the existing works in ...
Deep Multi-view Clustering Based on Graph Embedding
Advanced Intelligent Computing Technology and Applications
Abstract
In recent decades, multi-view clustering has garnered more attention in the fields of machine learning and pattern recognition. Traditional multi-view clustering algorithms typically aim to identify a common latent space among the multi-view data ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '23: Proceedings of the 31st ACM International Conference on Multimedia

October 2023

9913 pages

ISBN:9798400701085

DOI:10.1145/3581783

General Chairs:
Abdulmotaleb El Saddik
University of Ottawa, Canada & MBZUAI, UAE
,
Tao Mei
HiDream.ai, China
,
Rita Cucchiara
University of Modena and Reggio Emilia, Italy
,
Program Chairs:
Marco Bertini
University of Florence, Italy
,
Diana Patricia Tobon Vallejo
Unversidad de Medellin, Colombia
,
Pradeep K. Atrey
University at Albany, State University of New York, USA
,
M. Shamim Hossain
M. Shamim Hossain (King Saud University, KSA

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 October 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

MM '23

Sponsor:

SIGMM

MM '23: The 31st ACM International Conference on Multimedia

October 29 - November 3, 2023

Ottawa ON, Canada

Acceptance Rates

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
249
Total Downloads

Downloads (Last 12 months)141
Downloads (Last 6 weeks)7

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Lin YLiu HYu XZhang C(2025)Leveraging Transformer-based autoencoders for low-rank multi-view subspace clusteringPattern Recognition10.1016/j.patcog.2024.111331(111331)Online publication date: Jan-2025
https://doi.org/10.1016/j.patcog.2024.111331
Li JGao QWang QDeng CXie DBaeza-Yates RBonchi F(2024)Label Learning Method Based on Tensor ProjectionProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671671(1599-1609)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3671671
Qin YQin CZhang XFeng G(2024)Dual Consensus Anchor Learning for Fast Multi-View ClusteringIEEE Transactions on Image Processing10.1109/TIP.2024.345965133(5298-5311)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TIP.2024.3459651
Zhang ZWang QGao QPei CFeng W(2024)Unsupervised Cross-View Subspace Clustering via Adaptive Contrastive LearningIEEE Transactions on Big Data10.1109/TBDATA.2024.336608410:5(609-619)Online publication date: Oct-2024
https://doi.org/10.1109/TBDATA.2024.3366084
Zhang ZWang QPei CGao Q(2024)Deep cross-modal subspace clustering with Contrastive Neighbour EmbeddingNeurocomputing10.1016/j.neucom.2024.127318576:COnline publication date: 25-Jun-2024
https://dl.acm.org/doi/10.1016/j.neucom.2024.127318

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten