Yiqin Lv
ABSTRACT
The recent progress in graph representation learning boosts the development of many graph classification tasks, such as protein classification and social network classification. One of the mainstream approaches for graph representation learning is the hierarchical pooling method. It learns the graph representation by gradually reducing the scale of the graph, so it can be easily adapted to large-scale graphs. However, existing graph pooling methods discard the original graph structure during downsizing the graph, resulting in a lack of graph topological structure. In this paper, we propose a multi-scale graph neural network (MSGNN) model that not only retains the topological information of the graph but also maintains the key-subgraph for better interpretability. MSGNN gradually discards the unimportant nodes and retains the important subgraph structure during the iteration. The key subgraphs are first chosen by experience and then adaptively evolved to tailor specific graph structures for downstream tasks. The extensive experiments on seven datasets show that MSGNN improves the SOTA performance on graph classification and better retains key subgraphs.
- James Atwood and Don Towsley. 2016. Diffusion-Convolutional Neural Networks. In NeurIPS. 2001--2009.Google Scholar
- Jinheon Baek, Minki Kang, and Sung Ju Hwang. 2021. Accurate Learning of Graph Representations with Graph Multiset Pooling. In ICLR. https://openreview.net/forum?id=JHcqXGaqiGnGoogle Scholar
- Yunsheng Bai, Hao Ding, Song Bian, Ting Chen, Yizhou Sun, and Wei Wang. 2019. SimGNN - A Neural Network Approach to Fast Graph Similarity Computation. WSDM (2019), 384--392.Google Scholar
- Filippo Maria Bianchi, Daniele Grattarola, Lorenzo Livi, and Cesare Alippi. 2022. Hierarchical Representation Learning in Graph Neural Networks With Node Decimation Pooling. IEEE Transactions on Neural Networks and Learning Systems 33, 5 (2022), 2195--2207. https://doi.org/10.1109/TNNLS.2020.3044146Google ScholarCross Ref
- M. Karsten Borgwardt and Hans-Peter Kriegel. 2005. Shortest-Path Kernels on Graphs. ICDM (2005), 74--81.Google Scholar
- M. Karsten Borgwardt, Soon Cheng Ong, Stefan Schönauer, V. N. S. Vishwanathan, J. Alexander Smola, and Hans-Peter Kriegel. 2005. Protein function prediction via graph kernels. ISMB (Supplement of Bioinformatics) (2005), 47--56.Google Scholar
- Hanjun Dai, Bo Dai, and Le Song. 2016. Discriminative Embeddings of Latent Variable Models for Structured Data. ICML (2016).Google Scholar
- Asim Kumar Debnath, Rosa L. Lopez de Compadre, Gargi Debnath, Alan J. Shusterman, and Corwin Hansch. 1991. Structure-activity relationship of mutagenic aromatic and heteroaromatic nitro compounds. Correlation with molecular orbital energies and hydrophobicity. Journal of Medicinal Chemistry 34, 2 (1991), 786--797.Google ScholarCross Ref
- Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. 2016. Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering. NeurIPS (2016), 3844--3852.Google Scholar
- Paul D. Dobson and Andrew J. Doig. 2003. Distinguishing enzyme structures from non-enzymes without alignments. Journal of molecular biology 330, 4 (2003), 771--783.Google ScholarCross Ref
- Jinlong Du, Senzhang Wang, Hao Miao, and Jiaqiang Zhang. 2021. Multi-Channel Pooling Graph Neural Networks. IJCAI (2021), 1442--1448.Google Scholar
- David K Duvenaud, Dougal Maclaurin, Jorge Iparraguirre, Rafael Bombarell, Timothy Hirzel, Alán Aspuru-Guzik, and Ryan P Adams. 2015. Convolutional networks on graphs for learning molecular fingerprints. NeurIPS (2015).Google Scholar
- Hongyang Gao and Shuiwang Ji. 2019. Graph U-Nets. In ICML. 2083--2092.Google Scholar
- Justin Gilmer, Samuel S. Schoenholz, Patrick F. Riley, Oriol Vinyals, and George E. Dahl. 2017. Neural Message Passing for Quantum Chemistry. In ICML. 1263--1272.Google ScholarDigital Library
- L. William Hamilton, Rex Ying, and Jure Leskovec. 2017. Inductive Representation Learning on Large Graphs. NeurIPS (2017), 1024--1034.Google Scholar
- Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long Short-Term Memory. Neural Computation 9, 8 (11 1997), 1735--1780. https://doi.org/10.1162/neco.1997.9.8.1735 arXiv:https://direct.mit.edu/neco/article-pdf/9/8/1735/813796/neco.1997.9.8.1735.pdfGoogle ScholarDigital Library
- Jiatao Jiang, Chunyan Xu, Zhen Cui, Tong Zhang, Wenming Zheng, and Jian Yang. 2020. Walk-Steered Convolution for Graph Classification. IEEE Transactions on Neural Networks and Learning Systems 31, 11 (2020), 4553--4566. https://doi.org/10.1109/TNNLS.2019.2956095Google ScholarCross Ref
- Jeroen Kazius, Ross McGuire, and Roberta Bursi. 2005. Derivation and Validation of Toxicophores for Mutagenicity Prediction. Journal of medicinal chemistry 48, 1 (02 2005), 312--20. https://doi.org/10.1021/jm040835aGoogle ScholarCross Ref
- Diederik Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. ICLR (12 2015).Google Scholar
- N. Thomas Kipf and Max Welling. 2017. Semi-Supervised Classification with Graph Convolutional Networks. ICLR (2017).Google Scholar
- Kwei-Herng Lai, Daochen Zha, Kaixiong Zhou, and Xia Hu. 2020. Policy-GNN: Aggregation Optimization for Graph Neural Networks. In KDD. 461--471.Google Scholar
- Junhyun Lee, Inyeop Lee, and Jaewoo Kang. 2019. Self-attention graph pooling. In ICML. 3734--3743.Google Scholar
- Maosen Li, Siheng Chen, Ya Zhang, and Ivor Tsang. 2020. Graph cross networks with vertex infomax pooling. In NeurIPS, Vol. 33. 14093--14105.Google Scholar
- Yujia Li, Chenjie Gu, Thomas Dullien, Oriol Vinyals, and Pushmeet Kohli. 2019. Graph Matching Networks for Learning the Similarity of Graph Structured Objects. ICML (2019), 3835--3845.Google Scholar
- Hanxiao Liu, Karen Simonyan, and Yiming Yang. 2019. DARTS: Differentiable Architecture Search. ICLR (2019).Google Scholar
- Yixin Liu, Yu Zheng, Daokun Zhang, Hongxu Chen, Hao Peng, and Shirui Pan. 2022. Towards Unsupervised Deep Graph Structure Learning. In WWW. 1392--1403.Google Scholar
- Pierre Mahé, Nobuhisa Ueda, Tatsuya Akutsu, Jean-Luc Perret, and Jean-Philippe Vert. 2004. Extensions of marginalized graph kernels. ICML (2004), 70--70.Google Scholar
- Yujie Mo, Liang Peng, Jie Xu, Xiaoshuang Shi, and Xiaofeng Zhu. 2022. Simple Unsupervised Graph Representation Learning. In AAAI. 7797--7805.Google Scholar
- Nicolò Navarin, Van Dinh Tran, and Alessandro Sperduti. 2019. Universal Readout for Graph Convolutional Neural Networks. IJCNN (2019), 1--7.Google Scholar
- Wei Peng, Xiaopeng Hong, Haoyu Chen, and Guoying Zhao. 2020. Learning Graph Convolutional Network for Skeleton-based Human Action Recognition by Neural Searching. In AAAI. 2669--2676.Google Scholar
- Ekagra Ranjan, Soumya Sanyal, and Partha Talukdar. 2020. ASAP: Adaptive structure aware pooling for learning hierarchical graph representations. In AAAI, Vol. 34. 5470--5477.Google ScholarCross Ref
- Esteban Real, Alok Aggarwal, Yanping Huang, and Quoc V. Le. 2019. Regularized Evolution for Image Classifier Architecture Search. In AAAI. 4780--4789.Google Scholar
- Michael Schlichtkrull, Thomas N. Kipf, Peter Bloem, Rianne van den Berg, Ivan Titov, and Max Welling. 2018. Modeling Relational Data with Graph Convolutional Networks. In The Semantic Web. 593--607.Google Scholar
- Nino Shervashidze, Pascal Schweitzer, Jan van Erik Leeuwen, Kurt Mehlhorn, and M. Karsten Borgwardt. 2011. Weisfeiler-Lehman Graph Kernels. JMLR 12 (2011), 2539--2561.Google ScholarDigital Library
- Nino Shervashidze, SVN Vishwanathan, Tobias Petri, Kurt Mehlhorn, and Karsten Borgwardt. 2009. Efficient graphlet kernels for large graph comparison. In Artificial intelligence and statistics. 488--495.Google Scholar
- Martin Simonovsky and Nikos Komodakis. 2017. Dynamic Edge-Conditioned Filters In Convolutional Neural Networks On Graphs. CVPR (2017), 29--38.Google Scholar
- Qingyun Sun, Jianxin Li, Hao Peng, Jia Wu, Yuanxing Ning, Phillip S Yu, and Lifang He. 2021. SUGAR: Subgraph Neural Network with Reinforcement Pooling and Self-Supervised Mutual Information Mechanism. In WWW. 2081--2091.Google Scholar
- Naftali Tishby, Fernando Pereira, and William Bialek. 1999. The Information Bottleneck Method. Proceedings of the 37th Allerton Conference on Communication, Control and Computation (1999), 368--377.Google Scholar
- Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2018. Graph Attention Networks. ICLR (2018).Google Scholar
- Petar Velickovic, William Fedus, William L. Hamilton, Pietro Liò, Yoshua Bengio, and R. Devon Hjelm. 2019. Deep Graph Infomax. ICLR (2019).Google Scholar
- Oriol Vinyals, Samy Bengio, and Manjunath Kudlur. 2016. Order Matters: Sequence to sequence for sets. ICLR (2016).Google Scholar
- Nikil Wale and George Karypis. 2008. Comparison of Descriptor Spaces for Chemical Compound Retrieval and Classification. Knowledge and Information Systems (2008), 678--689.Google Scholar
- Chengcheng Wei, Wengang Zhou, Junfu Pu, and Houqiang Li. 2019. Msgcnn: Multi-Scale Graph Convolutional Neural Network For Point Cloud Segmentation. BIGMM (2019), 118--127.Google Scholar
- Lanning Wei, Huan Zhao, Quanming Yao, and Zhiqiang He. 2021. Pooling Architecture Search for Graph Classification. In CIKM.Google Scholar
- Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and Philip S. Yu. 2021. A Comprehensive Survey on Graph Neural Networks. IEEE Transactions on Neural Networks and Learning Systems 32, 1 (2021), 4--24. https://doi.org/10.1109/TNNLS.2020.2978386Google ScholarCross Ref
- Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. 2019. How Powerful are Graph Neural Networks? ICLR (2019).Google Scholar
- Keyulu Xu, Chengtao Li, Yonglong Tian, Tomohiro Sonobe, Ken-ichi Kawarabayashi, and Stefanie Jegelka. 2018. Representation learning on graphs with jumping knowledge networks. In ICML. 5453--5462.Google Scholar
- Xiaowei Xu, Nurcan Yuruk, Zhidan Feng, and Thomas A. J. Schweiger. 2007. SCAN: A Structural Clustering Algorithm for Networks. In KDD. 824--833.Google Scholar
- Pinar Yanardag and SVN Vishwanathan. 2015. A structural smoothing framework for robust graph comparison. NeurIPS 28 (2015).Google Scholar
- Zhitao Ying, Jiaxuan You, Christopher Morris, Xiang Ren, Will Hamilton, and Jure Leskovec. 2018. Hierarchical graph representation learning with differentiable pooling. In NeurIPS, Vol. 31. 4800--4810.Google Scholar
- Junchi Yu, Tingyang Xu, Yu Rong, Yatao Bian, Junzhou Huang, and Ran He. 2021. Graph Information Bottleneck for Subgraph Recognition. In ICLR.Google Scholar
- Hao Yuan and Shuiwang Ji. 2020. Structpool: Structured graph pooling via conditional random fields. In ICLR.Google Scholar
- Muhan Zhang and Yixin Chen. 2018. Link Prediction Based on Graph Neural Networks. In NeurIPS (NIPS'18). 5171--5181.Google Scholar
- Muhan Zhang, Zhicheng Cui, Marion Neumann, and Yixin Chen. 2018. An end-to-end deep learning architecture for graph classification. In AAAI.Google Scholar
- Zhen Zhang, Jiajun Bu, Martin Ester, Jianfeng Zhang, Zhao Li, Chengwei Yao, Dai Huifen, Zhi Yu, and Can Wang. 2021. Hierarchical multi-view graph pooling with structure learning. IEEE Trans. Knowl. Data Eng. (2021).Google ScholarCross Ref
- Ying Zhao, Haojin Jiang, Qi'an Chen, Yaqi Qin, Huixuan Xie, Yitao Wu, Shixia Liu, Zhiguang Zhou, Jiazhi Xia, and Fangfang Zhou. 2021. Preserving Minority Structures in Graph Sampling. IEEE Transactions on Visualization and Computer Graphics 27, 2 (2021), 1698--1708. https://doi.org/10.1109/TVCG.2020.3030428Google ScholarCross Ref
- Barret Zoph and Quoc V Le. 2017. Neural architecture search with reinforcement learning. In ICLR.Google Scholar
Index Terms
- Multi-scale Graph Pooling Approach with Adaptive Key Subgraph for Graph Representations
Recommendations
Node Information Awareness Pooling for Graph Representation Learning
Advances in Knowledge Discovery and Data MiningAbstractGraph neural networks (GNNs), which generalize deep neural network models to graph structure data, have attracted increasing attention and achieved state-of-the-art performance in graph-related tasks such as graph classification, link prediction, ...
Structure-Feature based Graph Self-adaptive Pooling
WWW '20: Proceedings of The Web Conference 2020Various methods to deal with graph data have been proposed in recent years. However, most of these methods focus on graph feature aggregation rather than graph pooling. Besides, the existing top-k selection graph pooling methods have a few problems. ...
Edge but not Least: Cross-View Graph Pooling
Machine Learning and Knowledge Discovery in DatabasesAbstractGraph neural networks have emerged as a powerful representation learning model for undertaking various graph prediction tasks. Various graph pooling methods have been developed to coarsen an input graph into a succinct graph-level representation ...
Comments