Abstract
Graph Auto-Encoder(GAE) emerged as a powerful node embedding method, has attracted extensive interests lately. GAE and most of its extensions rely on a series of encoding layers to learn effective node embeddings, while corresponding decoding layers trying to recover the original features. Promising performances on challenging tasks have demonstrated GAE’s powerful ability of representation. On the other hand, Subgraph Convolutional Networks(SCNs), as an extension of Graph Convolutional Networks(GCNs), can aggregate both tagged and local structural features in an artful way. In this paper, we show that SCNs can be improved (AttSCNs) by an attention mechanism to acquire better representational capability, which is competent for the duty of encoder. Then we develop inversed AttSCNs and propose a novel auto-encoder, i.e., Attention-Based Auto-Encoder(ABAE). This architecture utilizes attention mechanism to get insight of the data. We perform experiments on some challenging tasks to show the effectiveness of our models. Moreover, we construct AttSCNs for Node Classification. The results demonstrate that AttSCNs can produce considerable embeddings. Furthermore, we launch Link Prediction task for the proposed ABAE. Experimental results show that our ABAE has its fantastic power and achieves state-of-the-art in Link Prediction.
Y. Li and Y. Sun—Equally Contributed.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Perozzi, B., Al-Rfou, R., Skiena, S.: Deepwalk: online learning of social representations. In: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2014)
Grover, A., Leskovec, J.: node2vec: scalable feature learning for networks. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2016)
Gori, M., Monfardini, G., Scarselli, F.: A new model for learning in graph domains. In: Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005, vol. 2. IEEE (2005)
Joshi, R.B., Mishra, S.: Learning Graph Representations. arXiv preprint arXiv:2102.02026 (2021)
Kipf, T.N., Welling, M.: Variational graph auto-encoders. arXiv preprint arXiv:1611.07308 (2016)
Zhang, Z., et al.: Quantum-based subgraph convolutional neural networks. Pattern Recogn. 88, 38–49 (2019)
Salha, G., Hennequin, R., Vazirgiannis, M.: Simple and effective graph autoencoders with one-hop linear models. arXiv preprint arXiv:2001.07614 (2020)
Li, J., et al.: Graph Autoencoders with Deconvolutional Networks. arXiv preprint arXiv:2012.11898 (2020)
Shi, H., Fan, H., Kwok, J.T.: Effective decoding in graph auto-encoder using triadic closure. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34. no. 01 (2020)
Flam-Shepherd, D., Wu, T., Aspuru-Guzik, A.: Graph deconvolutional generation. arXiv preprint arXiv:2002.07087 (2020)
Bai, L., Cui, L., Bai, X., Hancock, E.R.: Deep depth-based representations of graphs through deep learning networks. Neurocomput. 336, 3–12 (2019)
Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016)
Velikovi, P., et al.: Graph attention networks. arXiv preprint arXiv:1710.10903 (2017)
Gao, H., Shuiwang, J.: Graph u-nets. In: International Conference on Machine Learning. PMLR (2019)
Zhang, J., et al.: Graph-bert: Only attention is needed for learning graph representations. arXiv preprint arXiv:2001.05140 (2020)
Zhang, J.: Get Rid of Suspended Animation Problem: Deep Diffusive Neural Network on Graph Semi-Supervised Classification. arXiv preprint arXiv:2001.07922 (2020)
Dabhi, S., Parmar, M.: NodeNet: A Graph Regularised Neural Network for Node Classification. arXiv preprint arXiv:2006.09022 (2020)
Huang, W., et al.: Adaptive sampling towards fast graph representation learning. arXiv preprint arXiv:1809.05343 (2018)
Davidson, T.R., et al.: Hyperspherical variational auto-encoders. arXiv preprint arXiv:1804.00891 (2018)
Di, X., et al.: Mutual information maximization in graph neural networks. In: 2020 International Joint Conference on Neural Networks (IJCNN). IEEE (2020)
Mavromatis, C., Karypis, G.: Graph InfoClust: Leveraging cluster-level node information for unsupervised graph representation learning. arXiv preprint arXiv:2009.06946 (2020)
Yang, H., et al.: Binarized attributed network embedding. In: 2018 IEEE International Conference on Data Mining (ICDM). IEEE (2018)
Acknowledgments
This work is supported by the National Natural Science Foundation of China (Grant no. T2122020, 61976235 and 61602535), the program for innovation research in Central University of Finance and Economics, the Emerging Interdisciplinary Project of CUFE.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Liu, T., Li, Y., Sun, Y., Cui, L., Bai, L. (2021). ABAE: Utilize Attention to Boost Graph Auto-Encoder. In: Pham, D.N., Theeramunkong, T., Governatori, G., Liu, F. (eds) PRICAI 2021: Trends in Artificial Intelligence. PRICAI 2021. Lecture Notes in Computer Science(), vol 13032. Springer, Cham. https://doi.org/10.1007/978-3-030-89363-7_26
Download citation
DOI: https://doi.org/10.1007/978-3-030-89363-7_26
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-89362-0
Online ISBN: 978-3-030-89363-7
eBook Packages: Computer ScienceComputer Science (R0)