skip to main content
10.1145/3394486.3403233acmconferencesArticle/Chapter ViewAbstractPublication PageskddConference Proceedingsconference-collections
research-article
Public Access

Graph Attention Networks over Edge Content-Based Channels

Published: 20 August 2020 Publication History

Abstract

Edges play a crucial role in passing information on a graph, especially when they carry textual content reflecting semantics behind how nodes are linked and interacting with each other. In this paper, we propose a channel-aware attention mechanism enabled by edge text content when aggregating information from neighboring nodes; and we realize this mechanism in a graph autoencoder framework. Edge text content is encoded as low-dimensional mixtures of latent topics, which serve as semantic channels for topic-level information passing on edges. We embed nodes and topics in the same latent space to capture their mutual dependency when decoding the structural and textual information on graph. We evaluated the proposed model on Yelp user-item bipartite graph and StackOverflow user-user interaction graph. The proposed model outperformed a set of baselines on link prediction and content prediction tasks. Qualitative evaluations also demonstrated the descriptive power of the learnt node embeddings, showing its potential as an interpretable representation of graphs.

References

[1]
Sami Abu-El-Haija, Bryan Perozzi, Rami Al-Rfou, and Alexander A Alemi. 2018. Watch your step: Learning node embeddings via graph attention. In NeurIPS. 9180--9190.
[2]
Charu C Aggarwal, Haixun Wang, et al. 2010. Managing and mining graph data. Vol. 40. Springer.
[3]
Rianne van den Berg, Thomas N Kipf, and Max Welling. 2017. Graph convolutional matrix completion. arXiv preprint arXiv:1706.02263 (2017).
[4]
Smriti Bhagat, Graham Cormode, and S Muthukrishnan. 2011. Node classification in social networks. In Social network data analytics. Springer, 115--148.
[5]
DavidMBlei, AndrewY Ng, and Michael I Jordan. 2003. Latent dirichlet allocation. Journal of machine Learning research 3, Jan (2003), 993--1022.
[6]
Joan Bruna,Wojciech Zaremba, Arthur Szlam, and Yann LeCun. 2013. Spectral networks and locally connected networks on graphs. arXiv preprint arXiv:1312.6203 (2013).
[7]
Hongyun Cai, Vincent W Zheng, and Kevin Chen-Chuan Chang. 2018. A comprehensive survey of graph embedding: Problems, techniques, and applications. TKDE 30, 9 (2018), 1616--1637.
[8]
Yukuo Cen, Xu Zou, Jianwei Zhang, Hongxia Yang, Jingren Zhou, and Jie Tang. 2019. Representation learning for attributed multiplex heterogeneous network. In SIGKDD. 1358--1368.
[9]
Pengfei Chen, Weiwen Liu, Chang-Yu Hsieh, Guangyong Chen, and Shengyu Zhang. 2019. Utilizing edge features in graph neural networks via variational information maximization. arXiv preprint arXiv:1906.05488 (2019).
[10]
Robert B Cialdini and Noah J Goldstein. 2004. Social influence: Compliance and conformity. Annu. Rev. Psychol. 55 (2004), 591--621.
[11]
Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. 2016. Convolutional neural networks on graphs with fast localized spectral filtering. In NeurIPS. 3844--3852.
[12]
Frederik Diehl. 2019. Edge Contraction Pooling for Graph Neural Networks. arXiv preprint arXiv:1905.10990 (2019).
[13]
Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. 2017. Neural message passing for quantum chemistry. In ICML. JMLR. org, 1263--1272.
[14]
Liyu Gong and Qiang Cheng. 2019. Exploiting Edge Features for Graph Neural Networks. In CVPR. 9211--9219.
[15]
Lin Gong, Lu Lin, Weihao Song, and Hongning Wang. 2020. JNET: Learning User Representations via Joint Network Embedding and Topic Embedding. In WSDM. 205--213.
[16]
Aditya Grover and Jure Leskovec. 2016. node2vec: Scalable feature learning for networks. In SIGKDD. 855--864.
[17]
Will Hamilton, Zhitao Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. In NeurIPS. 1024--1034.
[18]
Diederik P Kingma and Max Welling. 2013. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 (2013).
[19]
Diederik P Kingma, Max Welling, et al. 2019. An introduction to variational autoencoders. Foundations and Trends® in Machine Learning 12, 4 (2019), 307--392.
[20]
Thomas N Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016).
[21]
Lu Lin, Lin Gong, and Hongning Wang. 2019. Learning Personalized Topical Compositions with Item Response Theory. In WSDM. 609--617.
[22]
Miller McPherson, Lynn Smith-Lovin, and James M Cook. 2001. Birds of a feather: Homophily in social networks. Annual review of sociology 27, 1 (2001), 415--444.
[23]
Ashwin Paranjape, Austin R Benson, and Jure Leskovec. 2017. Motifs in temporal networks. In WSDM. 601--610.
[24]
Heiko Paulheim. 2017. Knowledge graph refinement: A survey of approaches and evaluation methods. Semantic web 8, 3 (2017), 489--508.
[25]
Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. 2014. Deepwalk: Online learning of social representations. In SIGKDD. 701--710.
[26]
Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne Van Den Berg, Ivan Titov, and Max Welling. 2018. Modeling relational data with graph convolutional networks. In ESWC. Springer, 593--607.
[27]
Akash Srivastava and Charles Sutton. 2017. Autoencoding variational inference for topic models. arXiv preprint arXiv:1703.01488 (2017).
[28]
Ben Taskar, Ming-Fai Wong, Pieter Abbeel, and Daphne Koller. 2004. Link prediction in relational data. In NeurIPS. 659--666.
[29]
Cunchao Tu, Han Liu, Zhiyuan Liu, and Maosong Sun. 2017. Cane: Context-aware network embedding for relation modeling. In ACL. 1722--1731.
[30]
Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2017. Graph attention networks. arXiv preprint arXiv:1710.10903 (2017).
[31]
Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. 2014. Knowledge graph embedding by translating on hyperplanes. In AAAI.
[32]
Carl Yang, Jieyu Zhang, Haonan Wang, Sha Li, Myungwan Kim, Matt Walker, Yiou Xiao, and Jiawei Han. 2020. Relation Learning on Social Networks with Multi-Modal Graph Edge Variational Autoencoders. In WSDM. 699--707.
[33]
Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L Hamilton, and Jure Leskovec. 2018. Graph convolutional neural networks for web-scale recommender systems. In SIGKDD. 974--983.
[34]
Hongming Zhang, Liwei Qiu, Lingling Yi, and Yangqiu Song. 2018. Scalable Multiplex Network Embedding. In IJCAI, Vol. 18. 3082--3088.

Cited By

View all
  • (2024)A Multifocal Graph-Based Neural Network Scheme for Topic Event ExtractionACM Transactions on Information Systems10.1145/369635343:1(1-36)Online publication date: 19-Sep-2024
  • (2024)LMACL: Improving Graph Collaborative Filtering with Learnable Model Augmentation Contrastive LearningACM Transactions on Knowledge Discovery from Data10.1145/365730218:7(1-24)Online publication date: 19-Jun-2024
  • (2024)Graph Contrastive Learning via Interventional View GenerationProceedings of the ACM Web Conference 202410.1145/3589334.3645687(1024-1034)Online publication date: 13-May-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
KDD '20: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining
August 2020
3664 pages
ISBN:9781450379984
DOI:10.1145/3394486
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 August 2020

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. graph neural networks
  2. representation learning
  3. topic modeling
  4. variational auto-encoder

Qualifiers

  • Research-article

Funding Sources

Conference

KDD '20
Sponsor:

Acceptance Rates

Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

Upcoming Conference

KDD '25

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)236
  • Downloads (Last 6 weeks)29
Reflects downloads up to 28 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)A Multifocal Graph-Based Neural Network Scheme for Topic Event ExtractionACM Transactions on Information Systems10.1145/369635343:1(1-36)Online publication date: 19-Sep-2024
  • (2024)LMACL: Improving Graph Collaborative Filtering with Learnable Model Augmentation Contrastive LearningACM Transactions on Knowledge Discovery from Data10.1145/365730218:7(1-24)Online publication date: 19-Jun-2024
  • (2024)Graph Contrastive Learning via Interventional View GenerationProceedings of the ACM Web Conference 202410.1145/3589334.3645687(1024-1034)Online publication date: 13-May-2024
  • (2024)Globally Interpretable Graph Learning via Distribution MatchingProceedings of the ACM Web Conference 202410.1145/3589334.3645674(992-1002)Online publication date: 13-May-2024
  • (2024)Graph Multi-Convolution and Attention Pooling for Graph ClassificationIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.344325346:12(10546-10557)Online publication date: Dec-2024
  • (2024)Beyond Homophily and Homogeneity Assumption: Relation-Based Frequency Adaptive Graph Neural NetworksIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2022.323041735:6(8497-8509)Online publication date: Jun-2024
  • (2024)Denoising Item Graph With Disentangled Learning for RecommendationIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.336148236:7(2942-2955)Online publication date: Jul-2024
  • (2024)Syntactic Graph Attention Network for Aspect-Level Sentiment AnalysisIEEE Transactions on Artificial Intelligence10.1109/TAI.2022.32275355:1(140-153)Online publication date: Jan-2024
  • (2024)GTR: An explainable Graph Topic-aware Recommender for scholarly documentElectronic Commerce Research and Applications10.1016/j.elerap.2024.101439(101439)Online publication date: Jul-2024
  • (2023)Learning the Explainable Semantic Relations via Unified Graph Topic-Disentangled Neural NetworksACM Transactions on Knowledge Discovery from Data10.1145/358996417:8(1-23)Online publication date: 12-May-2023
  • Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Login options

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media