Bryan Perozzi
ABSTRACT
Graphs are general data structures that can represent information from a variety of domains (social, biomedical, online transactions, and many more). Graph Neural Networks (GNNs) are quickly becoming the de-facto Machine Learning models for learning from Graph data and hereby infer missing information, such as, predicting labels of nodes or imputing missing edges.
In this tutorial we'll cover essential applications of Graph Machine Learning using TensorFlow GNN1[12], a Python framework that extends TensorFlow [1] with Graph Neural Networks (GNNs): models that leverage graph-structured data. TF-GNN is motivated and informed by years of applying graph representation learning to practical problems at Google [2-8, 10, 11, 13-19, 22-29]. In particular, TF-GNN focuses on the representation of heterogeneous graph data and supports the explicit modeling of an arbitrary number of relationship (edge) types between an arbitrary number of entity (node) types. These relationships can be used in combination with other TensorFlow components, e.g., a TF-GNN model might connect representations from a language model to those of a vision model and fine-tune these features for a node classification task. Many teams at Google run TF-GNN models in production. We believe this to be a direct consequence of TF-GNN's multi-layered API which is designed for accessibility to developers (regardless of their prior experience with machine learning).
- Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, et al. 2016. TensorFlow: A System for Large-Scale Machine Learning. In 12th USENIX symposium on operating systems design and implementation (OSDI 16). 265--283.Google Scholar
Digital Library
- Sami Abu-El-Haija, Amol Kapoor, Bryan Perozzi, and Joonseok Lee. 2020. N-gcn: Multi-scale graph convolution for semi-supervised node classification. In uncertainty in artificial intelligence. PMLR, 841--851.Google Scholar
- Sami Abu-El-Haija, Bryan Perozzi, and Rami Al-Rfou. 2017. Learning edge representations via low-rank asymmetric projections. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management. 1787--1796.Google ScholarDigital Library
- Sami Abu-El-Haija, Bryan Perozzi, Rami Al-Rfou, and Alexander A Alemi. 2018. Watch your step: Learning node embeddings via graph attention. Advances in neural information processing systems, Vol. 31 (2018).Google Scholar
- Sami Abu-El-Haija, Bryan Perozzi, Amol Kapoor, Nazanin Alipourfard, Kristina Lerman, Hrayr Harutyunyan, Greg Ver Steeg, and Aram Galstyan. 2019. Mixhop: Higher-order graph convolutional architectures via sparsified neighborhood mixing. In international conference on machine learning. PMLR, 21--29.Google Scholar
- Rami Al-Rfou, Bryan Perozzi, and Dustin Zelle. 2019. Ddgk: Learning graph representations for deep divergence graph kernels. In The World Wide Web Conference. 37--48.Google ScholarDigital Library
- Aleksandar Bojchevski, Johannes Klicpera, Bryan Perozzi, Amol Kapoor, Martin Blais, Benedek Rózemberczki, Michal Lukasik, and Stephan Günnemann. 2020. Scaling graph neural networks with approximate pagerank. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2464--2473.Google ScholarDigital Library
- Kaidi Cao, Phitchaya Mangpo Phothilimthana, Sami Abu-El-Haija, Dustin Zelle, Yanqi Zhou, Charith Mendis, Jure Leskovec, and Bryan Perozzi. 2023. Learning Large Graph Property Prediction via Graph Segment Training. arXiv preprint arXiv:2305.12322 (2023).Google Scholar
- Ines Chami, Sami Abu-El-Haija, Bryan Perozzi, Christopher Re, and Kevin Murphy. 2022. Machine Learning on Graphs: A Model and Comprehensive Taxonomy. Journal of Machine Learning Research, Vol. 23, 89 (2022), 1--64.Google Scholar
- Alessandro Epasto, Vahab Mirrokni, Bryan Perozzi, Anton Tsitsulin, and Peilin Zhong. 2022. Differentially Private Graph Learning via Sensitivity-Bounded Personalized PageRank. In Advances in Neural Information Processing Systems, Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (Eds.). https://openreview.net/forum?id=Fhty8PgFkDoGoogle Scholar
- Alessandro Epasto and Bryan Perozzi. 2019. Is a single embedding enough? learning node representations that capture multiple social contexts. In The world wide web conference. 394--404.Google Scholar
- Oleksandr Ferludin, Arno Eigenwillig, Martin Blais, Dustin Zelle, Jan Pfeifer, Alvaro Sanchez-Gonzalez, Sibon Li, Sami Abu-El-Haija, Peter Battaglia, Neslihan Bulut, et al. 2022. TF-GNN: graph neural networks in TensorFlow. arXiv preprint arXiv:2207.03522 (2022).Google Scholar
- Kimon Fountoulakis, Dake He, Silvio Lattanzi, Bryan Perozzi, Anton Tsitsulin, and Shenghao Yang. 2022. On Classification Thresholds for Graph Attention with Edge Features. arXiv preprint arXiv:2210.10014 (2022).Google Scholar
- Jonathan Halcrow, Alexandru Mosoi, Sam Ruth, and Bryan Perozzi. 2020. Grale: Designing Networks for Graph Learning. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (Virtual Event, CA, USA) (KDD '20). Association for Computing Machinery, New York, NY, USA, 2523--2532. https://doi.org/10.1145/3394486.3403302Google ScholarDigital Library
- Amol Kapoor, Xue Ben, Luyang Liu, Bryan Perozzi, Matt Barnes, Martin Blais, and Shawn O'Banion. 2020. Examining covid-19 forecasting using spatio-temporal graph neural networks. arXiv preprint arXiv:2007.03113 (2020).Google Scholar
- Seyed Mehran Kazemi, Anton Tsitsulin, Hossein Esfandiari, MohammadHossein Bateni, Deepak Ramachandran, Bryan Perozzi, and Vahab Mirrokni. 2022. Tackling Provably Hard Representative Selection via Graph Neural Networks. arxiv: 2205.10403 [cs.LG]Google Scholar
- Elan Sopher Markowitz, Keshav Balasubramanian, Mehrnoosh Mirtaheri, Sami Abu-El-Haija, Bryan Perozzi, Greg Ver Steeg, and Aram Galstyan. 2021. Graph Traversal with Tensor Functionals: A Meta-Algorithm for Scalable Learning. In International Conference on Learning Representations. https://openreview.net/forum?id=6DOZ8XNNfGNGoogle Scholar
- John Palowitch and Bryan Perozzi. 2020. Debiasing graph representations via metadata-orthogonal training. In 2020 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM). IEEE, 435--442.Google ScholarDigital Library
- John Palowitch, Anton Tsitsulin, Brandon Mayer, and Bryan Perozzi. 2022. GraphWorld: Fake Graphs Bring Real Insights for GNNs. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (Washington DC, USA) (KDD '22). Association for Computing Machinery, New York, NY, USA, 3691--3701. https://doi.org/10.1145/3534678.3539203Google ScholarDigital Library
- Bryan Perozzi. 2016. Local Modeling of Attributed Graphs: Algorithms and Applications. Ph.,D. Dissertation. State University of New York at Stony Brook.Google Scholar
- Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. 2014. DeepWalk: Online Learning of Social Representations. In ACM SIGKDD international conference on Knowledge discovery and data mining (KDD). 701--710.Google ScholarDigital Library
- ?tefan Postua varu, Anton Tsitsulin, Filipe Miguel Goncc alves de Almeida, Yingtao Tian, Silvio Lattanzi, and Bryan Perozzi. 2020. InstantEmbedding: Efficient local node representations. arXiv preprint arXiv:2010.06992 (2020).Google Scholar
- Benedek Rozemberczki, Peter Englert, Amol Kapoor, Martin Blais, and Bryan Perozzi. 2021. Pathfinder discovery networks for neural message passing. In Proceedings of the Web Conference 2021. 2547--2558.Google ScholarDigital Library
- Anton Tsitsulin, Marina Munkhoeva, and Bryan Perozzi. 2020. Just slaq when you approximate: Accurate spectral distances for web-scale graphs. In Proceedings of the Web Conference 2020. 2697--2703.Google ScholarDigital Library
- Anton Tsitsulin, Marina Munkhoeva, and Bryan Perozzi. 2023. Unsupervised Embedding Quality Evaluation. arxiv: 2305.16562 [cs.LG]Google Scholar
- Anton Tsitsulin, Benedek Rozemberczki, John Palowitch, and Bryan Perozzi. 2022. Synthetic Graph Generation to Benchmark Graph Learning. arXiv preprint arXiv:2204.01376 (2022).Google Scholar
- Minji Yoon, John Palowitch, Dustin Zelle, Ziniu Hu, Russ Salakhutdinov, and Bryan Perozzi. 2022. Zero-shot Transfer Learning within a Heterogeneous Graph via Knowledge Transfer Networks. In Advances in Neural Information Processing Systems, Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (Eds.).Google Scholar
- Qi Zhu, Yizhu Jiao, Haonan Wang, Natalia Ponomareva, and Bryan Perozzi. 2023. GNN Domain Adaptation using Optimal Transport. https://openreview.net/forum?id=nGyWzq-703uGoogle Scholar
- Qi Zhu, Natalia Ponomareva, Jiawei Han, and Bryan Perozzi. 2021. Shift-Robust GNNs: Overcoming the Limitations of Localized Graph Training data. In Advances in Neural Information Processing Systems, M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (Eds.), Vol. 34. Curran Associates, Inc., 27965--27977. https://proceedings.neurips.cc/paper/2021/file/eb55e369affa90f77dd7dc9e2cd33b16-Paper.pdfGoogle Scholar
Index Terms
- Graph Neural Networks in TensorFlow
Recommendations
On Graph Learning with Neural Networks
Machine Learning, Optimization, and Data ScienceAbstractGraphs are ideal for modeling natural systems where relations may be intrinsic among data objects. With massive data available, learning graph models from data has become potentially feasible as well as necessary. Yet from the traditional machine ...
Generating a Graph Colouring Heuristic with Deep Q-Learning and Graph Neural Networks
Learning and Intelligent OptimizationAbstractThe graph colouring problem consists of assigning labels, or colours, to the vertices of a graph such that no two adjacent vertices share the same colour. In this work we investigate whether deep reinforcement learning can be used to discover a ...
Graph Neural Networks for Graph Search
GRADES-NDA'20: Proceedings of the 3rd Joint International Workshop on Graph Data Management Experiences & Systems (GRADES) and Network Data Analytics (NDA)Graph neural networks (GNNs) have received more and more attention in past several years, due to the wide applications of graphs and networks, and the superiority of their performance compared to traditional heuristics-driven approaches. However, most ...
Access Granted
Comments