skip to main content
10.1145/3446999.3447013acmotherconferencesArticle/Chapter ViewAbstractPublication PagesicitConference Proceedingsconference-collections
research-article

Graph Contrastive Learning with Local and Global Mutual Information Maximization

Published: 09 April 2021 Publication History

Abstract

Graph is a common data structure in social networks, citation networks, bio-protein molecules and so on. Recent years, Graph Neural Networks(GNNs) have attracted more and more research attention because of its superior performance on some graph learning tasks. Training of GNNs needs large amount of labeled data, which casts shadows on the usability and expansibility of GNNs. Inspired by recent unsupervised research on computer vision and natural language processing, we propose a novel unsupervised graph representation learning model together with several graph data augmentation methods (drop edge, blur, mask features) and a local and global graph mutual information maximization strategy. By maximize two types of mutual information between original graph and augmented graph, the model is forced to learn some useful prior domain knowledge. We conduct experiments on both node classification and graph classification tasks and show the superior performance of the proposed model over state-of-the-art baselines.

References

[1]
Kipf, Thomas N., and Max Welling. "Semi-supervised classification with graph convolutional networks." arXiv preprint arXiv:1609.02907 (2016).
[2]
Hamilton, Will, Zhitao Ying, and Jure Leskovec. "Inductive representation learning on large graphs." Advances in neural information processing systems. 2017.
[3]
Veličković, Petar, "Graph attention networks." arXiv preprint arXiv:1710.10903 (2017)
[4]
Gao, Hongyang, and Shuiwang Ji. "Graph u-nets." arXiv preprint arXiv:1905.05178 (2019).
[5]
Lee, Junhyun, Inyeop Lee, and Jaewoo Kang. "Self-attention graph pooling." arXiv preprint arXiv:1904.08082 (2019)
[6]
Goyal, Palash, and Emilio Ferrara. "Graph embedding techniques, applications, and performance: A survey." Knowledge-Based Systems 151 (2018): 78-94.
[7]
Kipf, Thomas N., and Max Welling. "Variational graph auto-encoders." arXiv preprint arXiv:1611.07308 (2016).
[8]
Velickovic, Petar, "Deep Graph Infomax." ICLR (Poster). 2019.
[9]
Hassani, Kaveh, and Amir Hosein Khasahmadi. "Contrastive Multi-View Representation Learning on Graphs." arXiv preprint arXiv:2006.05582 (2020).
[10]
Hjelm, R. Devon, "Learning deep representations by mutual information estimation and maximization." arXiv preprint arXiv:1808.06670 (2018).
[11]
Chen, Ting, "A simple framework for contrastive learning of visual representations."  arXiv preprint arXiv:2002.05709 (2020).
[12]
Bachman, Philip, R. Devon Hjelm, and William Buchwalter. "Learning representations by maximizing mutual information across views." Advances in Neural Information Processing Systems. 2019.
[13]
Tian, Yonglong, Dilip Krishnan, and Phillip Isola. "Contrastive multiview coding." arXiv preprint arXiv:1906.05849 (2019).
[14]
Jaiswal, Ashish, "A Survey on Contrastive Self-supervised Learning." arXiv preprint arXiv:2011.00362 (2020).
[15]
Tian, Yonglong, "What makes for good views for contrastive learning."  arXiv preprint arXiv:2005.10243 (2020).
[16]
Rong, Yu, "Dropedge: Towards deep graph convolutional networks on node classification." International Conference on Learning Representations. 2019.
[17]
Waltz, Frederick M., and John WV Miller. "Efficient algorithm for gaussian blur using finite-state machines." Machine Vision Systems for Inspection and Metrology VII. Vol. 3521. International Society for Optics and Photonics, 1998.
[18]
Klicpera, Johannes, Stefan Weißenberger, and Stephan Günnemann. "Diffusion improves graph learning." Advances in Neural Information Processing Systems. 2019.
[19]
Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi-Rad. Collective classification in network data. AI magazine, 29(3):93, 2008.
[20]
Cox, David R. "The regression analysis of binary sequences." Journal of the Royal Statistical Society: Series B (Methodological) 20.2 (1958): 215-232.
[21]
Sun, Fan-Yun, "Infograph: Unsupervised and semi-supervised graph-level representation learning via mutual information maximization." arXiv preprint arXiv:1908.01000 (2019).
[22]
Narayanan, Annamalai, "graph2vec: Learning distributed representations of graphs." arXiv preprint arXiv:1707.05005 (2017).
[23]
Shervashidze, Nino, "Weisfeiler-lehman graph kernels." Journal of Machine Learning Research 12.9 (2011).
[24]
Yanardag, Pinar, and S. V. N. Vishwanathan. "Deep graph kernels." Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2015.

Cited By

View all

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
ICIT '20: Proceedings of the 2020 8th International Conference on Information Technology: IoT and Smart City
December 2020
266 pages
ISBN:9781450388559
DOI:10.1145/3446999
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 April 2021

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Contrastive learning
  2. GCNs
  3. Unsupervised learning

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Conference

ICIT 2020
ICIT 2020: IoT and Smart City
December 25 - 27, 2020
Xi'an, China

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 208
    Total Downloads
  • Downloads (Last 12 months)20
  • Downloads (Last 6 weeks)1
Reflects downloads up to 17 Feb 2025

Other Metrics

Citations

Cited By

View all

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media