research-article

Public Access

Graph Attention Networks over Edge Content-Based Channels

Authors:

Hongning WangAuthors Info & Claims

KDD '20: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining

Pages 1819 - 1827

https://doi.org/10.1145/3394486.3403233

Published: 20 August 2020 Publication History

Abstract

Edges play a crucial role in passing information on a graph, especially when they carry textual content reflecting semantics behind how nodes are linked and interacting with each other. In this paper, we propose a channel-aware attention mechanism enabled by edge text content when aggregating information from neighboring nodes; and we realize this mechanism in a graph autoencoder framework. Edge text content is encoded as low-dimensional mixtures of latent topics, which serve as semantic channels for topic-level information passing on edges. We embed nodes and topics in the same latent space to capture their mutual dependency when decoding the structural and textual information on graph. We evaluated the proposed model on Yelp user-item bipartite graph and StackOverflow user-user interaction graph. The proposed model outperformed a set of baselines on link prediction and content prediction tasks. Qualitative evaluations also demonstrated the descriptive power of the learnt node embeddings, showing its potential as an interpretable representation of graphs.

References

[1]

Sami Abu-El-Haija, Bryan Perozzi, Rami Al-Rfou, and Alexander A Alemi. 2018. Watch your step: Learning node embeddings via graph attention. In NeurIPS. 9180--9190.

[2]

Charu C Aggarwal, Haixun Wang, et al. 2010. Managing and mining graph data. Vol. 40. Springer.

[3]

Rianne van den Berg, Thomas N Kipf, and Max Welling. 2017. Graph convolutional matrix completion. arXiv preprint arXiv:1706.02263 (2017).

[4]

Smriti Bhagat, Graham Cormode, and S Muthukrishnan. 2011. Node classification in social networks. In Social network data analytics. Springer, 115--148.

[5]

DavidMBlei, AndrewY Ng, and Michael I Jordan. 2003. Latent dirichlet allocation. Journal of machine Learning research 3, Jan (2003), 993--1022.

Digital Library

[6]

Joan Bruna,Wojciech Zaremba, Arthur Szlam, and Yann LeCun. 2013. Spectral networks and locally connected networks on graphs. arXiv preprint arXiv:1312.6203 (2013).

[7]

Hongyun Cai, Vincent W Zheng, and Kevin Chen-Chuan Chang. 2018. A comprehensive survey of graph embedding: Problems, techniques, and applications. TKDE 30, 9 (2018), 1616--1637.

Digital Library

[8]

Yukuo Cen, Xu Zou, Jianwei Zhang, Hongxia Yang, Jingren Zhou, and Jie Tang. 2019. Representation learning for attributed multiplex heterogeneous network. In SIGKDD. 1358--1368.

[9]

Pengfei Chen, Weiwen Liu, Chang-Yu Hsieh, Guangyong Chen, and Shengyu Zhang. 2019. Utilizing edge features in graph neural networks via variational information maximization. arXiv preprint arXiv:1906.05488 (2019).

[10]

Robert B Cialdini and Noah J Goldstein. 2004. Social influence: Compliance and conformity. Annu. Rev. Psychol. 55 (2004), 591--621.

[11]

Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. 2016. Convolutional neural networks on graphs with fast localized spectral filtering. In NeurIPS. 3844--3852.

[12]

Frederik Diehl. 2019. Edge Contraction Pooling for Graph Neural Networks. arXiv preprint arXiv:1905.10990 (2019).

[13]

Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. 2017. Neural message passing for quantum chemistry. In ICML. JMLR. org, 1263--1272.

[14]

Liyu Gong and Qiang Cheng. 2019. Exploiting Edge Features for Graph Neural Networks. In CVPR. 9211--9219.

[15]

Lin Gong, Lu Lin, Weihao Song, and Hongning Wang. 2020. JNET: Learning User Representations via Joint Network Embedding and Topic Embedding. In WSDM. 205--213.

[16]

Aditya Grover and Jure Leskovec. 2016. node2vec: Scalable feature learning for networks. In SIGKDD. 855--864.

[17]

Will Hamilton, Zhitao Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. In NeurIPS. 1024--1034.

[18]

Diederik P Kingma and Max Welling. 2013. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 (2013).

[19]

Diederik P Kingma, Max Welling, et al. 2019. An introduction to variational autoencoders. Foundations and Trends® in Machine Learning 12, 4 (2019), 307--392.

[20]

Thomas N Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016).

[21]

Lu Lin, Lin Gong, and Hongning Wang. 2019. Learning Personalized Topical Compositions with Item Response Theory. In WSDM. 609--617.

[22]

Miller McPherson, Lynn Smith-Lovin, and James M Cook. 2001. Birds of a feather: Homophily in social networks. Annual review of sociology 27, 1 (2001), 415--444.

[23]

Ashwin Paranjape, Austin R Benson, and Jure Leskovec. 2017. Motifs in temporal networks. In WSDM. 601--610.

[24]

Heiko Paulheim. 2017. Knowledge graph refinement: A survey of approaches and evaluation methods. Semantic web 8, 3 (2017), 489--508.

[25]

Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. 2014. Deepwalk: Online learning of social representations. In SIGKDD. 701--710.

Digital Library

[26]

Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne Van Den Berg, Ivan Titov, and Max Welling. 2018. Modeling relational data with graph convolutional networks. In ESWC. Springer, 593--607.

[27]

Akash Srivastava and Charles Sutton. 2017. Autoencoding variational inference for topic models. arXiv preprint arXiv:1703.01488 (2017).

[28]

Ben Taskar, Ming-Fai Wong, Pieter Abbeel, and Daphne Koller. 2004. Link prediction in relational data. In NeurIPS. 659--666.

[29]

Cunchao Tu, Han Liu, Zhiyuan Liu, and Maosong Sun. 2017. Cane: Context-aware network embedding for relation modeling. In ACL. 1722--1731.

[30]

Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2017. Graph attention networks. arXiv preprint arXiv:1710.10903 (2017).

[31]

Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. 2014. Knowledge graph embedding by translating on hyperplanes. In AAAI.

[32]

Carl Yang, Jieyu Zhang, Haonan Wang, Sha Li, Myungwan Kim, Matt Walker, Yiou Xiao, and Jiawei Han. 2020. Relation Learning on Social Networks with Multi-Modal Graph Edge Variational Autoencoders. In WSDM. 699--707.

[33]

Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L Hamilton, and Jure Leskovec. 2018. Graph convolutional neural networks for web-scale recommender systems. In SIGKDD. 974--983.

[34]

Hongming Zhang, Liwei Qiu, Lingling Yi, and Yangqiu Song. 2018. Scalable Multiplex Network Embedding. In IJCAI, Vol. 18. 3082--3088.

Cited By

Wan QWan CXiao KHu RLiu DLiao GLiu XShuai Y(2024)A Multifocal Graph-Based Neural Network Scheme for Topic Event ExtractionACM Transactions on Information Systems10.1145/369635343:1(1-36)Online publication date: 19-Sep-2024
https://dl.acm.org/doi/10.1145/3696353
Liu XHao YZhao LLiu GSheng VZhao P(2024)LMACL: Improving Graph Collaborative Filtering with Learnable Model Augmentation Contrastive LearningACM Transactions on Knowledge Discovery from Data10.1145/365730218:7(1-24)Online publication date: 19-Jun-2024
https://dl.acm.org/doi/10.1145/3657302
Wo ZShao MWang WGuo XLin LChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Graph Contrastive Learning via Interventional View GenerationProceedings of the ACM Web Conference 202410.1145/3589334.3645687(1024-1034)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645687
Show More Cited By

Index Terms

Graph Attention Networks over Edge Content-Based Channels
1. Computing methodologies
  1. Machine learning
    1. Machine learning approaches

Recommendations

Variational Gaussian topic model with invertible neural projections
Abstract
Neural topic models have triggered a surge of interest in extracting topics from text automatically since they avoid the sophisticated derivations in conventional topic models. However, most of these models only utilize word count information, ...
Multi-granularity heterogeneous graph attention networks for extractive document summarization
Abstract
Extractive document summarization is a fundamental task in natural language processing (NLP). Recently, several Graph Neural Networks (GNNs) are proposed for this task. However, most existing GNN-based models can neither effectively encode ...
Cycling topic graph learning for neural topic modeling
Abstract
Topic models aim to discover a set of latent topics in a textual corpus. Graph Neural Networks (GNNs) have been recently utilized in Neural Topic Models (NTMs) due to their strong capacity to model document representations with the text graph. ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

KDD '20: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining

August 2020

3664 pages

ISBN:9781450379984

DOI:10.1145/3394486

General Chairs:
Rajesh Gupta
UC San Diego, USA
,
Yan Liu
USC, USA
,
Program Chairs:
Mohak Shah
LG Electronics, USA
,
Suju Rajan
Linkedin, USA
,
Publications Chairs:
Jiliang Tang
Michigan State, USA
,
B. Aditya Prakash
Georgia Tech, USA

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 August 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Science Foundation

Conference

KDD '20

Sponsor:

KDD '20: The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

July 6 - 10, 2020

CA, Virtual Event, USA

Acceptance Rates

Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

Upcoming Conference

KDD '25

Sponsor:
sigkdd
sigkdd

The 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining

August 3 - 7, 2025

Toronto , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

22
Total Citations
View Citations
2,171
Total Downloads

Downloads (Last 12 months)236
Downloads (Last 6 weeks)29

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wan QWan CXiao KHu RLiu DLiao GLiu XShuai Y(2024)A Multifocal Graph-Based Neural Network Scheme for Topic Event ExtractionACM Transactions on Information Systems10.1145/369635343:1(1-36)Online publication date: 19-Sep-2024
https://dl.acm.org/doi/10.1145/3696353
Liu XHao YZhao LLiu GSheng VZhao P(2024)LMACL: Improving Graph Collaborative Filtering with Learnable Model Augmentation Contrastive LearningACM Transactions on Knowledge Discovery from Data10.1145/365730218:7(1-24)Online publication date: 19-Jun-2024
https://dl.acm.org/doi/10.1145/3657302
Wo ZShao MWang WGuo XLin LChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Graph Contrastive Learning via Interventional View GenerationProceedings of the ACM Web Conference 202410.1145/3589334.3645687(1024-1034)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645687
Nian YChang YJin WLin LChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Globally Interpretable Graph Learning via Distribution MatchingProceedings of the ACM Web Conference 202410.1145/3589334.3645674(992-1002)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645674
Xu YWang JGuang MJiang C(2024)Graph Multi-Convolution and Attention Pooling for Graph ClassificationIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.344325346:12(10546-10557)Online publication date: Dec-2024
https://doi.org/10.1109/TPAMI.2024.3443253
Wu LLin HHu BTan CGao ZLiu ZLi S(2024)Beyond Homophily and Homogeneity Assumption: Relation-Based Frequency Adaptive Graph Neural NetworksIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2022.323041735:6(8497-8509)Online publication date: Jun-2024
https://doi.org/10.1109/TNNLS.2022.3230417
Zhang LLiu GLiu XWu J(2024)Denoising Item Graph With Disentangled Learning for RecommendationIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.336148236:7(2942-2955)Online publication date: Jul-2024
https://doi.org/10.1109/TKDE.2024.3361482
Yuan LWang JYu LZhang X(2024)Syntactic Graph Attention Network for Aspect-Level Sentiment AnalysisIEEE Transactions on Artificial Intelligence10.1109/TAI.2022.32275355:1(140-153)Online publication date: Jan-2024
https://doi.org/10.1109/TAI.2022.3227535
Ni PWang XLv BWu L(2024)GTR: An explainable Graph Topic-aware Recommender for scholarly documentElectronic Commerce Research and Applications10.1016/j.elerap.2024.101439(101439)Online publication date: Jul-2024
https://doi.org/10.1016/j.elerap.2024.101439
Wu LZhao HLi ZHuang ZLiu QChen E(2023)Learning the Explainable Semantic Relations via Unified Graph Topic-Disentangled Neural NetworksACM Transactions on Knowledge Discovery from Data10.1145/358996417:8(1-23)Online publication date: 12-May-2023
https://dl.acm.org/doi/10.1145/3589964
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten