Xiaojie Guo
ABSTRACT
Spatial networks represent crucial data structures where the nodes and edges are embedded in a geometric space. Nowadays, spatial network data is becoming increasingly popular and important, ranging from microscale (e.g., protein structures), to middle-scale (e.g., biological neural networks), to macro-scale (e.g., mobility networks). Although, modeling and understanding the generative process of spatial networks are very important, they remain largely under-explored due to the significant challenges in automatically modeling and distinguishing the independency and correlation among various spatial and network factors. To address these challenges, we first propose a novel objective for joint spatial-network disentanglement from the perspective of information bottleneck as well as a novel optimization algorithm to optimize the intractable objective. Based on this, a spatial-network variational autoencoder (SND-VAE) with a new spatial-network message passing neural network (S-MPNN) is proposed to discover the independent and dependent latent factors of spatial and networks. Qualitative and quantitative experiments on both synthetic and real-world datasets demonstrate the superiority of the proposed model over the state-of-the-arts by up to 66.9% for graph generation and 37.3% for interpretability.
Supplemental Material
- Farras Abdelnour, Michael Dayan, and Orrin et al. Devinsky. 2018. Functional brain connectivity is predictable from anatomic network's Laplacian eigen-structure. NeuroImage, Vol. 172 (2018), 728--739.Google Scholar
Cross Ref
- Alexander A Alemi, Ian Fischer, Joshua V Dillon, and Kevin Murphy. 2016. Deep variational information bottleneck. arXiv preprint arXiv:1612.00410 (2016).Google Scholar
- Namrata Anand and Po-Ssu Huang. 2018. Generative modeling for protein structures. In NeurIPS. 7505--7516.Google Scholar
- Tristan Aumentado-Armstrong, Stavros Tsogkas, Allan Jepson, and Sven Dickinson. 2019. Geometric disentanglement for generative latent shape models. In ICCV. 8181--8190.Google Scholar
- Marc Barthélemy. 2011. Spatial networks. Physics Reports, Vol. 499, 1--3 (2011), 1--101.Google ScholarCross Ref
- Yoshua Bengio, Aaron Courville, and Pascal Vincent. 2013. Representation learning: A review and new perspectives. IEEE transactions on pattern analysis and machine intelligence, Vol. 35, 8 (2013), 1798--1828.Google ScholarDigital Library
- Milan Bradonjić, Aric Hagberg, and Allon G Percus. 2007. Giant component and connectivity in geographical threshold graphs. In International Workshop on Algorithms and Models for the Web-Graph. Springer, 209--216.Google ScholarCross Ref
- Christopher P Burgess, Irina Higgins, and Arka et al. Pal. 2018. Understanding disentangling in β-VAE. arXiv preprint arXiv:1804.03599 (2018).Google Scholar
- Daniel T Chang. 2020. Geometric Graph Representations and Geometric Graph Convolutions for Deep Learning on Three-Dimensional (3D) Graphs. arXiv preprint arXiv:2006.01785 (2020).Google Scholar
- Ricky TQ Chen, Xuechen Li, Roger B Grosse, and David K Duvenaud. 2018. Isolating sources of disentanglement in variational autoencoders. In NeurIPS. 2610--2620.Google Scholar
- Hyeoncheol Cho and Insung S Choi. 2018. Three-dimensionally embedded graph convolutional network (3dgcn) for molecule interpretation. arXiv preprint arXiv:1811.09794 (2018).Google Scholar
- Christopher B Choy, Danfei Xu, and JunYoung et al. Gwak. 2016. 3d-r2n2: A unified approach for single and multi-view 3d object reconstruction. In European conference on computer vision. Springer, 628--644.Google Scholar
- Tomasz Danel, Przemysław Spurek, and Jacek et al Tabor. 2019. Spatial Graph Convolutional Networks. arXiv preprint arXiv:1909.05310 (2019).Google Scholar
- Yuanqi Du, Xiaojie Guo, Amarda Shehu, and Liang Zhao. 2020. Interpretable Molecule Generation via Disentanglement Learning. In Proceedings of the 11th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics. 1--8.Google ScholarDigital Library
- Anastasia Dubrovina, Fei Xia, and Panos et al. Achlioptas. 2019. Composite shape modeling via latent space factorization. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 8140--8149.Google Scholar
- Aditya Grover, Aaron Zweig, and Stefano Ermon. 2019. Graphite: Iterative Generative Modeling of Graphs. In ICML. 2434--2444.Google Scholar
- Xiaojie Guo, Yuanqi Du, and Liang Zhao. 2021. Property Controllable Variational Auto-encoder via Invertible Mutual Dependence. In ICLR.Google Scholar
- Xiaojie Guo, Sivani Tadepalli, Liang Zhao, and Amarda Shehu. 2020 a. Generating tertiary protein structures via an interpretative variational autoencoder. arXiv preprint arXiv:2004.07119 (2020).Google Scholar
- Xiaojie Guo, Lingfei Wu, and Liang Zhao. 2018. Deep graph translation. arXiv preprint arXiv:1805.09980 (2018).Google Scholar
- Xiaojie Guo and Liang Zhao. 2020. A systematic survey on deep generative models for graph generation. arXiv preprint arXiv:2007.06686 (2020).Google Scholar
- Xiaojie Guo, Liang Zhao, and et al. 2019. Deep Multi-attributed Graph Translation with Node-Edge Co-evolution. In ICDM.Google Scholar
- Xiaojie Guo, Liang Zhao, and Zhao et al Qin. 2020 b. Interpretable Deep Graph Generation with Node-Edge Co-Disentanglement. In KDD.Google Scholar
- Irina Higgins, Loic Matthey, and Arka et al. Pal. 2017. beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework. ICLR, Vol. 2, 5 (2017), 6.Google Scholar
- Dou Huang, Xuan Song, and Zipei et al. Fan. 2019. A variational autoencoder based generative model of urban human mobility. In MIPR. IEEE, 425--430.Google Scholar
- John Ingraham, Vikas Garg, and Regina et al Barzilay. 2019. Generative models for graph-based protein design. In NeurIPS. 15820--15831.Google Scholar
- Hyunjik Kim and Andriy Mnih. 2018. Disentangling by factorising. ICML (2018).Google Scholar
- Thomas N Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. ICLR (2016).Google Scholar
- Jon Kleinberg. 2000. The small-world phenomenon: An algorithmic perspective. In Proceedings of the thirty-second annual ACM symposium on Theory of computing. 163--170.Google ScholarDigital Library
- Konstantin Klemmer, Adriano Koshiyama, and Sebastian Flennerhag. 2019. Augmenting correlation structures in spatial data using deep generative models. arXiv preprint arXiv:1905.09796 (2019).Google Scholar
- Johannes Klicpera, Janek Groß, and Stephan Günnemann. 2020. Directional message passing for molecular graphs. arXiv preprint arXiv:2003.03123 (2020).Google Scholar
- Abhishek Kumar, Prasanna Sattigeri, and Avinash Balakrishnan. 2018. Variational Inference of Disentangled Latent Concepts from Unlabeled Observations. In ICLR.Google Scholar
- Jake Levinson, Avneesh Sud, and Ameesh Makadia. 2019. Latent feature disentanglement for 3D meshes. arXiv preprint arXiv:1906.03281 (2019).Google Scholar
- Francesco Locatello, Stefan Bauer, and Mario et al. Lucic. 2019. Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations. In ICML. 4114--4124.Google Scholar
- Jianxin Ma, Peng Cui, and Kun et al. Kuang. 2019. Disentangled graph convolutional networks. In ICML. 4212--4221.Google Scholar
- Olvi L Mangasarian. 1994. Nonlinear programming. SIAM.Google Scholar
- Charlie Nash and Christopher KI Williams. 2017. The shape variational autoencoder: A deep generative model of part-segmented 3D objects. In Computer Graphics Forum, Vol. 36. Wiley Online Library, 1--12.Google Scholar
- Arun Pandey, Michael Fanuel, and Joachim et al. Schreurs. 2020. Disentangled Representation Learning and Generation with Manifold Optimization. arXiv preprint arXiv:2006.07046 (2020).Google Scholar
- Mathew Penrose et al. 2003. Random geometric graphs. Vol. 5. Oxford university press.Google Scholar
- Taseef Rahman, Yuanqi Du, Liang Zhao, and Amarda Shehu. 2021. Generative Adversarial Learning of Protein Tertiary Structures. Molecules, Vol. 26, 5 (2021), 1209.Google ScholarCross Ref
- Scott Reed, Kihyuk Sohn, and Yuting et al Zhang. 2014. Learning to disentangle factors of variation with manifold interaction. In ICML. PMLR, 1431--1439.Google Scholar
- Martin Simonovsky and Nikos Komodakis. 2018. Graphvae: Towards generation of small graphs using variational autoencoders. In ICANN. Springer, 412--422.Google Scholar
- Qingyang Tan, Lin Gao, Yu-Kun Lai, and Shihong Xia. 2018. Variational autoencoders for deforming 3d mesh models. In CVPR. 5841--5850.Google Scholar
- Diego Valsesia, Giulia Fracastoro, and Enrico Magli. 2018. Learning localized generative models for 3d point clouds via graph convolution. In ICLR.Google Scholar
- Xiaoyang Wang, Yao Ma, and Yiqiet al. Wang. 2020. Traffic flow prediction via spatial temporal graph neural network. In Proceedings of The Web Conference 2020. 1082--1092.Google ScholarDigital Library
- Bernard M Waxman. 1988. Routing of multipoint connections. IEEE journal on selected areas in communications, Vol. 6, 9 (1988), 1617--1622.Google ScholarDigital Library
- Sijie Yan, Yuanjun Xiong, and Dahua Lin. 2018. Spatial temporal graph convolutional networks for skeleton-based action recognition. In AAAI, Vol. 32.Google ScholarCross Ref
- Jiaxuan You, Rex Ying, and et al. 2018. GraphRNN: Generating Realistic Graphs with Deep Auto-regressive Models. In ICML. 5708--5717.Google Scholar
- Liang Zhao. 2021. Event Prediction in the Big Data Era: A Systematic Survey. ACM Comput. Surv., Vol. 54, 5, Article 94 (May 2021), bibinfonumpages37 pages.Google ScholarDigital Library
Index Terms
- Deep Generative Models for Spatial Networks
Recommendations
Diversity in Deep Generative Models and Generative AI
Machine Learning, Optimization, and Data ScienceAbstractThe decoder-based machine learning generative algorithms such as Generative Adversarial Networks (GAN), Variational Auto-Encoders (VAE), Transformers show impressive results when constructing objects similar to those in a training ensemble. ...
Algorithms that Get Old: The Case of Generative Deep Neural Networks
Machine Learning, Optimization, and Data ScienceAbstractGenerative deep neural networks used in machine learning, like the Variational Auto-Encoders (VAE), and Generative Adversarial Networks (GANs) produce new objects each time when asked to do so with the constraint that the new objects remain ...
Accelerating Computation of Distance Based Centrality Measures for Spatial Networks
Discovery ScienceAbstractIn this paper, by focusing on spatial networks embedded in the real space, we first extend the conventional step-based closeness and betweenness centralities by incorporating inter-nodes link distances obtained from the positions of nodes. Then, ...
Comments