Abstract
In recent years, substantial progress has been made on Graph Convolutional Networks (GCNs). However, the computing of GCN usually requires a large memory space for keeping the entire graph. In consequence, GCN is not flexible enough, especially for large scale graphs in complex real-world applications. Fortunately, for transductive graph representation learning, methods based on Matrix Factorization (MF) naturally support constructing mini-batches, and thus are more friendly to distributed computing compared with GCN. Accordingly, in this paper, we analyze the connections between GCN and MF, and simplify GCN as matrix factorization with unitization and co-training. Furthermore, under the guidance of our analysis, we propose an alternative model to GCN named Unitized and Co-training Matrix Factorization (UCMF). Extensive experiments have been conducted on several real-world datasets. On the task of semi-supervised node classification, the experimental results illustrate that UCMF achieves similar or superior performances compared with GCN. Meanwhile, distributed UCMF significantly outperforms distributed GCN methods, which shows that UCMF can greatly benefit complex real-world applications.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Chen, D., Lin, Y., Li, W., Li, P., Zhou, J., Sun, X.: Measuring and relieving the over-smoothing problem for graph neural networks from the topological view. In: AAAI (2020)
Chen, J., Ma, T., Xiao, C.: FastGCN: fast learning with graph convolutional networks via importance sampling. arXiv preprint arXiv:1801.10247 (2018)
Chiang, W.L., Liu, X., Si, S., Li, Y., Bengio, S., Hsieh, C.J.: Cluster-GCN: an efficient algorithm for training deep and large graph convolutional networks. In: KDD (2019)
Du, L., Wang, Y., Song, G., Lu, Z., Wang, J.: Dynamic network embedding: an extended approach for skip-gram based network embedding. In: IJCAI (2018)
Gemulla, R., Nijkamp, E., Haas, P.J., Sismanis, Y.: Large-scale matrix factorization with distributed stochastic gradient descent. In: KDD (2011)
Grover, A., Leskovec, J.: node2vec: scalable feature learning for networks. In: KDD (2016)
Hamilton, W., Ying, Z., Leskovec, J.: Inductive representation learning on large graphs. In: NeurIPS (2017)
Huang, X., Li, J., Hu, X.: Label informed attributed network embedding. In: WSDM (2017)
Karypis, G., Kumar, V.: A fast and high quality multilevel scheme for partitioning irregular graphs. SIAM J. Sci. Comput. 20(1), 359–392 (1998)
Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. In: ICLR (2017)
Levy, O., Goldberg, Y.: Neural word embedding as implicit matrix factorization. In: NeurIPS (2014)
Li, M., et al.: Scaling distributed machine learning with the parameter server. In: OSDI (2014)
Li, Q., Han, Z., Wu, X.M.: Deeper insights into graph convolutional networks for semi-supervised learning. In: AAAI (2018)
Perozzi, B., Al-Rfou, R., Skiena, S.: DeepWalk: online learning of social representations. In: KDD (2014)
Qiu, J., Dong, Y., Ma, H., Li, J., Wang, K., Tang, J.: Network embedding as matrix factorization: Unifying DeepWalk, LINE, PTE, and node2vec. In: WSDM (2018)
Sen, P., Namata, G., Bilgic, M., Getoor, L., Gallagher, B., Eliassirad, T.: Collective classification in network data. AI Mag. 29(3), 93–106 (2008)
Tang, J., Qu, M., Wang, M., Zhang, M., Yan, J., Mei, Q.: Line: large-scale information network embedding. In: WWW (2015)
Veličković, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., Bengio, Y.: Graph attention networks. In: ICLR (2017)
Wang, X., He, X., Wang, M., Feng, F., Chua, T.S.: Neural graph collaborative filtering. In: SIGIR (2019)
Weston, J., Ratle, F., Mobahi, H., Collobert, R.: Deep learning via semi-supervised embedding. In: ICML (2012)
Wu, F., Zhang, T., Souza Jr, A.H.d., Fifty, C., Yu, T., Weinberger, K.Q.: Simplifying graph convolutional networks. arXiv preprint arXiv:1902.07153 (2019)
Wu, J., He, J., Xu, J.: DEMO-Net: degree-specific graph neural networks for node and graph classification. In: KDD (2019)
Yang, Z., Cohen, W., Salakhutdinov, R.: Revisiting semi-supervised learning with graph embeddings. In: ICML (2016)
Ying, R., He, R., Chen, K., Eksombatchai, P., Hamilton, W.L., Leskovec, J.: Graph convolutional neural networks for web-scale recommender systems. In: KDD (2018)
Yu, H.-F., Hsieh, C.-J., Si, S., Dhillon, I.S.: Parallel matrix factorization for recommender systems. Knowl. Inf. Syst. 41(3), 793–819 (2013). https://doi.org/10.1007/s10115-013-0682-2
Zhang, M., Chen, Y.: Link prediction based on graph neural networks. In: NeurIPS (2018)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Liu, Q., Zhang, H., Liu, Z. (2021). Simplifying Graph Convolutional Networks as Matrix Factorization. In: U, L.H., Spaniol, M., Sakurai, Y., Chen, J. (eds) Web and Big Data. APWeb-WAIM 2021. Lecture Notes in Computer Science(), vol 12858. Springer, Cham. https://doi.org/10.1007/978-3-030-85896-4_3
Download citation
DOI: https://doi.org/10.1007/978-3-030-85896-4_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-85895-7
Online ISBN: 978-3-030-85896-4
eBook Packages: Computer ScienceComputer Science (R0)