skip to main content
10.1145/3503161.3547949acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
research-article

Simple Self-supervised Multiplex Graph Representation Learning

Published: 10 October 2022 Publication History

Abstract

Self-supervised multiplex graph representation learning (SMGRL) aims to capture the information from the multiplex graph, and generates discriminative embedding without labels. However, previous SMGRL methods still suffer from the issues of efficiency and effectiveness due to the processes, e.g., data augmentation, negative sample encoding, complex pretext tasks, etc. In this paper, we propose a simple method to achieve efficient and effective SMGRL. Specifically, the proposed method removes the processes (i.e., data augmentation and negative sample encoding) for the SMGRL and designs a simple pretext task, for achieving the efficiency. Moreover, the proposed method also designs an intra-graph decorrelation loss and an inter-graph decorrelation loss, respectively, to capture the common information within individual graphs and the common information across graphs, for achieving the effectiveness. Extensive experimental results verify the efficiency and effectiveness of our method, compared to 11 comparison methods on 4 public benchmark datasets, on the node classification task.

Supplementary Material

MP4 File (MM22-fp0817.mp4)
Presentation video

References

[1]
Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. Enriching word vectors with subword information. Transactions of the association for computational linguistics, Vol. 5 (2017), 135--146.
[2]
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020. A simple framework for contrastive learning of visual representations. In ICML. 1597--1607.
[3]
Xinlei Chen and Kaiming He. 2021. Exploring simple siamese representation learning. In CVPR. 15750--15758.
[4]
Xiaokai Chu, Xinxin Fan, Di Yao, Zhihua Zhu, Jianhui Huang, and Jingping Bi. 2019. Cross-Network Embedding for Multi-Network Alignment. In WWW. 273--284.
[5]
Xavier Glorot and Yoshua Bengio. 2010. Understanding the difficulty of training deep feedforward neural networks. In AISTATS. 249--256.
[6]
Caroline Haythornthwaite. 2001. Exploring multiplexity: Social network structures in a computer-supported distance learning class. The information society, Vol. 17, 3 (2001), 211--226.
[7]
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. 2020. Momentum contrast for unsupervised visual representation learning. In CVPR. 9729--9738.
[8]
R. Devon Hjelm, Alex Fedorov, Samuel Lavoie-Marchildon, Karan Grewal, Philip Bachman, Adam Trischler, and Yoshua Bengio. 2019. Learning deep representations by mutual information estimation and maximization. In ICLR.
[9]
Rongyao Hu, Zhenyun Deng, and Xiaofeng Zhu. 2021a. Multi-scale graph fusion for co-saliency detection. In AAAI, Vol. 35. 7789--7796.
[10]
Rongyao Hu, Jiangzhang Gan, Xiaofeng Zhu, Tong Liu, and Xiaoshuang Shi. 2022. Multi-task multi-modality SVM for early COVID-19 Diagnosis using chest CT data. Information Processing & Management, Vol. 59, 1 (2022), 102782.
[11]
Rongyao Hu, Ziwen Peng, Xiaofeng Zhu, Jiangzhang Gan, Yonghua Zhu, Junbo Ma, and Guorong Wu. 2021b. Multi-band brain network analysis for functional neuroimaging biomarker identification. IEEE Transactions on Medical Imaging, Vol. 40, 12 (2021), 3843--3855.
[12]
Baoyu Jing, Chanyoung Park, and Hanghang Tong. 2021a. Hdmi: High-order deep multiplex infomax. In WWW. 2414--2424.
[13]
Baoyu Jing, Yuejia Xiang, Xi Chen, Yu Chen, and Hanghang Tong. 2021b. Graph-MVP: Multi-View Prototypical Contrastive Learning for Multiplex Graphs. CoRR, Vol. abs/2109.03560 (2021).
[14]
P. Diederik Kingma and Lei Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In ICLR.
[15]
N. Thomas Kipf and Max Welling. 2017. Semi-Supervised Classification with Graph Convolutional Networks. In ICLR.
[16]
Ryong Lee, Kyoung-Sook Kim, Komei Sugiura, Koji Zettsu, and Yutaka Kidawara. 2013. Complementary Integration of Heterogeneous Crowd-Sourced Datasets for Enhanced Social Analytics. In MDM. 234--243.
[17]
Junnan Li, Pan Zhou, Caiming Xiong, and Steven C. H. Hoi. 2021. Prototypical Contrastive Learning of Unsupervised Representations. In ICLR.
[18]
Changshu Liu, Liangjian Wen, Zhao Kang, Guangchun Luo, and Ling Tian. 2021. Self-supervised Consensus Representation Learning for Attributed Graph. In ACM MM. 2654--2662.
[19]
Weiyi Liu, Pin-Yu Chen, Sailung Yeung, Toyotaro Suzumura, and Lingli Chen. 2017. Principled multilayer network embedding. In ICDMW. 134--141.
[20]
Yao Ma, Suhang Wang, Charu C. Aggarwal, Dawei Yin, and Jiliang Tang. 2019. Multi-dimensional Graph Convolutional Networks. In ICDM. 657--665.
[21]
Ryuta Matsuno and Tsuyoshi Murata. 2018. MELL: effective embedding method for multiplex networks. In WWW. 1261--1268.
[22]
Tomá s Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013a. Efficient Estimation of Word Representations in Vector Space. In ICLR, Workshop.
[23]
Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013b. Distributed representations of words and phrases and their compositionality. In NeurIPS, Vol. 26.
[24]
Yujie Mo, Liang Peng, Jie Xu, Xiaoshuang Shi, and Xiaofeng Zhu. 2022. Simple Unsupervised Graph Representation Learning. In AAAI. 7797--7805.
[25]
Vinod Nair and Geoffrey E. Hinton. 2010. Rectified Linear Units Improve Restricted Boltzmann Machines. In ICML. 807--814.
[26]
Erlin Pan and Zhao Kang. 2021. Multi-view Contrastive Graph Clustering. In NeurIPS, Vol. 34.
[27]
Chanyoung Park, Donghyun Kim, Jiawei Han, and Hwanjo Yu. 2020. Unsupervised Attributed Multiplex Network Embedding. In AAAI. 5371--5378.
[28]
Liang Peng, Rongyao Hu, Fei Kong, Jiangzhang Gan, Yujie Mo, Xiaoshuang Shi, and Xiaofeng Zhu. 2022a. Reverse Graph Learning for Graph Neural Network. IEEE Transactions on Neural Networks and Learning Systems (2022), 1--12.
[29]
Liang Peng, Nan Wang, Nicha Dvornek, Xiaofeng Zhu, and Xiaoxiao Li. 2022b. FedNI: Federated Graph Learning with Network Inpainting for Population-Based Disease Prediction. IEEE Transactions on Medical Imaging (2022), 1--1.
[30]
Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. 2014. Deepwalk: Online learning of social representations. In KDD. 701--710.
[31]
Willard V Quine. 1946. Concatenation as a basis for arithmetic. The Journal of Symbolic Logic, Vol. 11, 4 (1946), 105--114.
[32]
Yuxiang Ren, Bo Liu, Chao Huang, Peng Dai, Liefeng Bo, and Jiawei Zhang. 2019. Heterogeneous Deep Graph Infomax. CoRR, Vol. abs/1911.08538 (2019).
[33]
Joshua Robinson, Ching-Yao Chuang, Suvrit Sra, and Stefanie Jegelka. 2020. Contrastive Learning with Hard Negative Samples. In ICLR.
[34]
Jingkuan Song, Hanwang Zhang, Xiangpeng Li, Lianli Gao, Meng Wang, and Richang Hong. 2018. Self-Supervised Video Hashing With Hierarchical Binary Auto-Encoder. IEEE Transactions on Image Processing, Vol. 27, 7 (2018), 3210--3221.
[35]
Fan-Yun Sun, Jordan Hoffmann, Vikas Verma, and Jian Tang. 2020. InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximization. In ICLR.
[36]
Yonglong Tian, Dilip Krishnan, and Phillip Isola. 2020. Contrastive Multiview Coding. In ECCV, Vol. 12356. 776--794.
[37]
Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of machine learning research, Vol. 9, 11 (2008).
[38]
Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018. Graph Attention Networks. In ICLR.
[39]
Petar Velickovic, William Fedus, William L. Hamilton, Pietro Liò, Yoshua Bengio, and R. Devon Hjelm. 2019. Deep Graph Infomax. In ICLR.
[40]
Xiao Wang, Houye Ji, Chuan Shi, Bai Wang, Yanfang Ye, Peng Cui, and Philip S. Yu. 2019. Heterogeneous Graph Attention Network. In WWW. 2022--2032.
[41]
Xiao Wang, Nian Liu, Hui Han, and Chuan Shi. 2021. Self-supervised Heterogeneous Graph Neural Network with Co-contrastive Learning. In KDD. 1726--1736.
[42]
Lirong Wu, Haitao Lin, Cheng Tan, Zhangyang Gao, and Stan Z Li. 2021. Self-supervised learning on graphs: Contrastive, generative, or predictive. IEEE Transactions on Knowledge and Data Engineering (2021).
[43]
Jie Xu, Chao Li, Yazhou Ren, Liang Peng, Yujie Mo, Xiaoshuang Shi, and Xiaofeng Zhu. 2022a. Deep Incomplete Multi-View Clustering via Mining Cluster Complementarity. In AAAI.
[44]
Jie Xu, Yazhou Ren, Huayi Tang, Xiaorong Pu, Xiaofeng Zhu, Ming Zeng, and Lifang He. 2021a. Multi-VAE: Learning Disentangled View-Common and View-Peculiar Visual Representations for Multi-View Clustering. In ICCV. 9234--9243.
[45]
Jie Xu, Yazhou Ren, Huayi Tang, Zhimeng Yang, Lili Pan, Yang Yang, Xiaorong Pu, Philip S Yu, and Lifang He. 2022b. Self-Supervised Discriminative Feature Learning for Deep Multi-View Clustering. IEEE Transactions on Knowledge and Data Engineering (2022).
[46]
Jie Xu, Huayi Tang, Yazhou Ren, Liang Peng, Xiaofeng Zhu, and Lifang He. 2022c. Multi-Level Feature Learning for Contrastive Multi-View Clustering. In CVPR. 16051--16060.
[47]
Xing Xu, Huimin Lu, Jingkuan Song, Yang Yang, Heng Tao Shen, and Xuelong Li. 2020. Ternary Adversarial Networks With Self-Supervision for Zero-Shot Cross-Modal Retrieval. IEEE Transactions on Cybernetics, Vol. 50, 6 (2020), 2400--2413.
[48]
Xing Xu, Tan Wang, Yang Yang, Alan Hanjalic, and Heng Tao Shen. 2021b. Radial Graph Convolutional Network for Visual Question Generation. IEEE Transactions on Neural Networks and Learning Systems, Vol. 32, 4 (2021), 1654--1667.
[49]
Quanzeng You, Hailin Jin, Zhaowen Wang, Chen Fang, and Jiebo Luo. 2016. Image Captioning with Semantic Attention. In CVPR. 4651--4659.
[50]
Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, and Yang Shen. 2020. Graph contrastive learning with augmentations. In NeurIPS.
[51]
Junliang Yu, Hongzhi Yin, Min Gao, Xin Xia, Xiangliang Zhang, and Nguyen Quoc Viet Hung. 2021a. Socially-Aware Self-Supervised Tri-Training for Recommendation. In KDD. 2084--2092.
[52]
Junliang Yu, Hongzhi Yin, Jundong Li, Qinyong Wang, Nguyen Quoc Viet Hung, and Xiangliang Zhang. 2021b. Self-Supervised Multi-Channel Hypergraph Convolutional Network for Social Recommendation. In WWW. 413--424.
[53]
Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and Sté phane Deny. 2021. Barlow Twins: Self-Supervised Learning via Redundancy Reduction. In ICML, Vol. 139. 12310--12320.
[54]
Hongming Zhang, Liwei Qiu, Lingling Yi, and Yangqiu Song. 2018. Scalable Multiplex Network Embedding. In IJCAI. 3082--3088.
[55]
Ji Zhang, Jingkuan Song, Lianli Gao, Ye Liu, and Heng Tao Shen. 2022. Progressive Meta-learning with Curriculum. IEEE Transactions on Circuits and Systems for Video Technology (2022).
[56]
Jingran Zhang, Xing Xu, Fumin Shen, Yazhou Yao, Jie Shao, and Xiaofeng Zhu. 2021. Video Representation Learning with Graph Contrastive Augmentation. In ACM MM. 3043--3051.
[57]
Han Zhao, Xu Yang, Zhenru Wang, Erkun Yang, and Cheng Deng. 2021. Graph Debiased Contrastive Learning with Joint Representation Clustering. In IJCAI. 3434--3440.
[58]
Xiang Zhou, Fumin Shen, Li Liu, Wei Liu, Liqiang Nie, Yang Yang, and Heng Tao Shen. 2020. Graph Convolutional Network Hashing. IEEE Transactions on Cybernetics, Vol. 50, 4 (2020), 1460--1472.
[59]
Yanqiao Zhu, Yichen Xu, Feng Yu, Qiang Liu, Shu Wu, and Liang Wang. 2020. Deep graph contrastive representation learning. arXiv preprint arXiv:2006.04131 (2020).
[60]
Yanqiao Zhu, Yichen Xu, Feng Yu, Qiang Liu, Shu Wu, and Liang Wang. 2021. Graph contrastive learning with adaptive augmentation. In WWW. 2069--2080.

Cited By

View all
  • (2025)AmGNN: A Framework for Adaptive Processing of Inter-layer Information in Multi-layer GraphSocial Networks Analysis and Mining10.1007/978-3-031-78541-2_29(472-488)Online publication date: 24-Jan-2025
  • (2024)Forecasting resource usage pattern changes in clouds via contrast graph-evolution learningFuture Generation Computer Systems10.1016/j.future.2024.01.013154:C(373-383)Online publication date: 25-Jun-2024
  • (2024)COVID-19 diagnosis based on swin transformer model with demographic information fusion and enhanced multi-head attention mechanismExpert Systems with Applications10.1016/j.eswa.2023.122805243(122805)Online publication date: Jun-2024
  • Show More Cited By

Index Terms

  1. Simple Self-supervised Multiplex Graph Representation Learning

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      MM '22: Proceedings of the 30th ACM International Conference on Multimedia
      October 2022
      7537 pages
      ISBN:9781450392037
      DOI:10.1145/3503161
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 10 October 2022

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. multiplex graph
      2. representation learning
      3. self-supervised learning

      Qualifiers

      • Research-article

      Funding Sources

      Conference

      MM '22
      Sponsor:

      Acceptance Rates

      Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)113
      • Downloads (Last 6 weeks)7
      Reflects downloads up to 17 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2025)AmGNN: A Framework for Adaptive Processing of Inter-layer Information in Multi-layer GraphSocial Networks Analysis and Mining10.1007/978-3-031-78541-2_29(472-488)Online publication date: 24-Jan-2025
      • (2024)Forecasting resource usage pattern changes in clouds via contrast graph-evolution learningFuture Generation Computer Systems10.1016/j.future.2024.01.013154:C(373-383)Online publication date: 25-Jun-2024
      • (2024)COVID-19 diagnosis based on swin transformer model with demographic information fusion and enhanced multi-head attention mechanismExpert Systems with Applications10.1016/j.eswa.2023.122805243(122805)Online publication date: Jun-2024
      • (2024)Representation Learning in Multiplex Graphs: Where and How to Fuse Information?Computational Science – ICCS 202410.1007/978-3-031-63778-0_1(3-18)Online publication date: 2-Jul-2024
      • (2023)Global and Nodal Mutual Information Maximization in Heterogeneous GraphsICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP49357.2023.10095282(1-5)Online publication date: 4-Jun-2023
      • (2023)Neighborhood-enhanced contrast for pre-training graph neural networksNeural Computing and Applications10.1007/s00521-023-09274-636:8(4195-4205)Online publication date: 10-Dec-2023
      • (2022)A Self-supervised Graph Autoencoder with Barlow TwinsPRICAI 2022: Trends in Artificial Intelligence10.1007/978-3-031-20865-2_37(501-512)Online publication date: 10-Nov-2022

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media