skip to main content
10.1145/3459637.3482299acmconferencesArticle/Chapter ViewAbstractPublication PagescikmConference Proceedingsconference-collections
research-article

Speedup Robust Graph Structure Learning with Low-Rank Information

Published: 30 October 2021 Publication History

Abstract

Recent studies have shown that graph neural networks (GNNs) are vulnerable to unnoticeable adversarial perturbations, which largely confines their deployment in many safety-critical domains. Robust graph structure learning has been proposed to improve the GNN performance in the face of adversarial attacks. In particular, the low-rank methods are utilized to purify the perturbed graphs. However, these methods are mostly computationally expensive with O(n3) time complexity and O(n2) space complexity. We propose LRGNN, a fast and robust graph structure learning framework, which exploits the low-rank property as prior knowledge to speed up optimization. To eliminate adversarial perturbation, LRGNN decouples the adjacency matrix into a low-rank component and a sparse one, and learns by minimizing the rank of the first part while suppressing the second part. Its sparse variant is formed to reduce the memory footprint further. Experimental results on various attack settings have shown LRGNN acquires comparable robustness with the state-of-the-art much more efficiently, boasting a significant advantage on large-scale graphs.

Supplementary Material

MP4 File (cikm.mp4)
Video presentation.

References

[1]
Joan Bruna, Wojciech Zaremba, Arthur Szlam, and Yann LeCun. Spectral networks and locally connected networks on graphs. In 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings, 2014.
[2]
Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. Convolutional neural networks on graphs with fast localized spectral filtering. In Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, December 5-10, 2016, Barcelona, Spain, pages 3837--3845, 2016.
[3]
Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings, 2017.
[4]
Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings, 2018.
[5]
Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L Hamilton, and Jure Leskovec. Graph convolutional neural networks for web-scale recommender systems. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 974--983, 2018.
[6]
Ruoyu Li, Sheng Wang, Feiyun Zhu, and Junzhou Huang. Adaptive graph convolutional neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018.
[7]
Saurabh Verma and Zhi-Li Zhang. Stability and generalization of graph convolutional neural networks. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 1539--1548, 2019.
[8]
Bin Lu, Xiaoying Gan, Haiming Jin, Luoyi Fu, and Haisong Zhang. Spatiotemporal adaptive gated graph convolution network for urban traffic flow forecasting. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management, pages 1025--1034, 2020.
[9]
Yangtao Wang, Yanzhao Xie, Yu Liu, Ke Zhou, and Xiaocui Li. Fast graph convolution network based multi-label image recognition via cross-modal fusion. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management, pages 1575--1584, 2020.
[10]
Chenghu Zhou, Hua Wang, Chengshan Wang, Zengqian Hou, Zhiming Zheng, Shuzhong Shen, Qiuming Cheng, Zhiqiang Feng, Xinbing Wang, Hairong Lv, et al. Prospects for the research on geoscience knowledge graph in the big data era. Science China Earth Sciences, pages 1--11, 2021.
[11]
Lichao Sun, Yingtong Dou, Carl Yang, Ji Wang, Philip S. Yu, Lifang He, and Bo Li. Adversarial attack and defense on graph data: A survey. arXiv preprint arXiv:1812.10528, 2018.
[12]
Wei Jin, Yaxin Li, Han Xu, Yiqi Wang, and Jiliang Tang. Adversarial attacks and defenses on graphs: A review and empirical study. arXiv preprint arXiv:2003.00653, 2020.
[13]
Dingyuan Zhu, Ziwei Zhang, Peng Cui, and Wenwu Zhu. Robust graph convolutional networks against adversarial attacks. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 1399--1407, 2019.
[14]
Wei Jin, Tyler Derr, Yiqi Wang, Yao Ma, Zitao Liu, and Jiliang Tang. Node similarity preserving graph convolutional networks. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining, pages 148--156, 2021.
[15]
Huijun Wu, Chen Wang, Yuriy Tyshetskiy, Andrew Docherty, Kai Lu, and Liming Zhu. Adversarial examples on graph data: Deep insights into attack and defense. arXiv preprint arXiv:1903.01610, 2019.
[16]
Negin Entezari, Saba A Al-Sayouri, Amirali Darvishzadeh, and Evangelos E Papalexakis. All you need is low (rank) defending against adversarial attacks on graphs. In Proceedings of the 13th International Conference on Web Search and Data Mining, pages 169--177, 2020.
[17]
Wei Jin, Yao Ma, Xiaorui Liu, Xianfeng Tang, Suhang Wang, and Jiliang Tang. Graph structure learning for robust graph neural networks. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 66--74, 2020.
[18]
Daniel Zü gner and Stephan Gü nnemann. Adversarial attacks on graph neural networks via meta learning. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019, 2019.
[19]
Emmanuel J Candès, Xiaodong Li, Yi Ma, and John Wright. Robust principal component analysis? Journal of the ACM (JACM), 58(3):1--37, 2011.
[20]
Namrata Vaswani and Praneeth Narayanamurthy. Static and dynamic robust pca and matrix completion: A review. Proceedings of the IEEE, 106(8):1359--1379, 2018.
[21]
Shiqian Ma, Donald Goldfarb, and Lifeng Chen. Fixed point and bregman iterative methods for matrix rank minimization. Mathematical Programming, 128(1):321--353, 2011.
[22]
Karthik Mohan and Maryam Fazel. Iterative reweighted algorithms for matrix rank minimization. The Journal of Machine Learning Research, 13(1):3441--3473, 2012.
[23]
Zihan Zhou, Xiaodong Li, John Wright, Emmanuel Candes, and Yi Ma. Stable principal component pursuit. In 2010 IEEE international symposium on information theory, pages 1518--1522. IEEE, 2010.
[24]
Shiqian Ma and Necdet Serhat Aybat. Efficient optimization algorithms for robust principal component analysis and its variants. Proceedings of the IEEE, 106(8):1411--1426, 2018.
[25]
Luong Trung Nguyen, Junhan Kim, and Byonghyo Shim. Low-rank matrix completion: A contemporary survey. IEEE Access, 7:94215--94237, 2019.
[26]
Emmanuel J Candes and Yaniv Plan. Matrix completion with noise. Proceedings of the IEEE, 98(6):925--936, 2010.
[27]
Tom Goldstein, Christoph Studer, and Richard G. Baraniuk. A field guide to forward-backward splitting with a FASTA implementation. CoRR, abs/1411.3406, 2014.
[28]
Stephen Boyd, Stephen P Boyd, and Lieven Vandenberghe. Convex optimization. Cambridge university press, 2004.
[29]
Daniel Zügner, Amir Akbarnejad, and Stephan Günnemann. Adversarial attacks on neural networks for graph data. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 2847--2856, 2018.
[30]
Andrew Kachites McCallum, Kamal Nigam, Jason Rennie, and Kristie Seymore. Automating the construction of internet portals with machine learning. Information Retrieval, 3(2):127--163, 2000.
[31]
C Lee Giles, Kurt D Bollacker, and Steve Lawrence. Citeseer: An automatic citation indexing system. In Proceedings of the third ACM conference on Digital libraries, pages 89--98, 1998.
[32]
Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi-Rad. Collective classification in network data. AI magazine, 29(3):93--93, 2008.
[33]
Lada A Adamic and Natalie Glance. The political blogosphere and the 2004 us election: divided they blog. In Proceedings of the 3rd international workshop on Link discovery, pages 36--43, 2005.
[34]
Yaxin Li, Wei Jin, Han Xu, and Jiliang Tang. Deeprobust: A pytorch library for adversarial attacks and defenses. arXiv preprint arXiv:2005.06149, 2020.

Cited By

View all
  • (2024)MF-GSLAE: A Multi-Factor User Representation Pre-Training Framework for Dual-Target Cross-Domain RecommendationACM Transactions on Information Systems10.1145/369038243:2(1-28)Online publication date: 24-Oct-2024
  • (2024)A Multi-View Graph Contrastive Learning Framework for Defending Against Adversarial AttacksIEEE Transactions on Emerging Topics in Computational Intelligence10.1109/TETCI.2024.33822308:6(4022-4032)Online publication date: Dec-2024
  • (2024)Semi-Supervised Graph Structure Learning via Dual Reinforcement of Label and Prior StructureIEEE Transactions on Cybernetics10.1109/TCYB.2024.341662154:11(6943-6956)Online publication date: Nov-2024
  • Show More Cited By

Index Terms

  1. Speedup Robust Graph Structure Learning with Low-Rank Information

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    CIKM '21: Proceedings of the 30th ACM International Conference on Information & Knowledge Management
    October 2021
    4966 pages
    ISBN:9781450384469
    DOI:10.1145/3459637
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 30 October 2021

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. graph convolutional networks
    2. matrix rank minimization
    3. robustness

    Qualifiers

    • Research-article

    Funding Sources

    • NSF China
    • the Science and Technology Innovation Program of Shanghai

    Conference

    CIKM '21
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

    Upcoming Conference

    CIKM '25

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)84
    • Downloads (Last 6 weeks)3
    Reflects downloads up to 17 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)MF-GSLAE: A Multi-Factor User Representation Pre-Training Framework for Dual-Target Cross-Domain RecommendationACM Transactions on Information Systems10.1145/369038243:2(1-28)Online publication date: 24-Oct-2024
    • (2024)A Multi-View Graph Contrastive Learning Framework for Defending Against Adversarial AttacksIEEE Transactions on Emerging Topics in Computational Intelligence10.1109/TETCI.2024.33822308:6(4022-4032)Online publication date: Dec-2024
    • (2024)Semi-Supervised Graph Structure Learning via Dual Reinforcement of Label and Prior StructureIEEE Transactions on Cybernetics10.1109/TCYB.2024.341662154:11(6943-6956)Online publication date: Nov-2024
    • (2024)Progressive structure enhancement graph convolutional network for face clusteringEngineering Applications of Artificial Intelligence10.1016/j.engappai.2023.107274127:PAOnline publication date: 1-Feb-2024
    • (2023)GSLBProceedings of the 37th International Conference on Neural Information Processing Systems10.5555/3666122.3667441(30306-30318)Online publication date: 10-Dec-2023
    • (2023)APGL4SR: A Generic Framework with Adaptive and Personalized Global Collaborative Information in Sequential RecommendationProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3614781(3009-3019)Online publication date: 21-Oct-2023
    • (2023)Minimum Topology Attacks for Graph Neural NetworksProceedings of the ACM Web Conference 202310.1145/3543507.3583509(630-640)Online publication date: 30-Apr-2023
    • (2023)Adversarial Attack and Defense on Graph Data: A SurveyIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2022.320124335:8(7693-7711)Online publication date: 1-Aug-2023
    • (2023)Meta-path infomax joint structure enhancement for multiplex network representation learningKnowledge-Based Systems10.1016/j.knosys.2023.110701275:COnline publication date: 5-Sep-2023
    • (2022)Are defenses for graph neural networks robust?Proceedings of the 36th International Conference on Neural Information Processing Systems10.5555/3600270.3600921(8954-8968)Online publication date: 28-Nov-2022
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media