skip to main content
10.1145/3548606.3559358acmconferencesArticle/Chapter ViewAbstractPublication PagesccsConference Proceedingsconference-collections
research-article
Public Access

Finding MNEMON: Reviving Memories of Node Embeddings

Published: 07 November 2022 Publication History

Abstract

Previous security research efforts orbiting around graphs have been exclusively focusing on either (de-)anonymizing the graphs or understanding the security and privacy issues of graph neural networks. Little attention has been paid to understand the privacy risks of integrating the output from graph embedding models (e.g., node embeddings) with complex downstream machine learning pipelines. In this paper, we fill this gap and propose a novel model-agnostic graph recovery attack that exploits the implicit graph structural information preserved in the embeddings of graph nodes. We show that an adversary can recover edges with decent accuracy by only gaining access to the node embedding matrix of the original graph without interactions with the node embedding models. We demonstrate the effectiveness and applicability of our graph recovery attack through extensive experiments.

References

[1]
Mikkel Abrahamsen, Greg Bodwin, Eva Rotenberg, and Morten Stöckel. 2016. Graph Reconstruction with a Betweenness Oracle. In STACS.
[2]
Seyed Ali Alhosseini, Raad Bin Tareaf, Pejman Najafi, and Christoph Meinel. 2019. Detect Me If You Can: Spam Bot Detection Using Inductive Representation Learning. In WWW.
[3]
Dana Angluin and Jiang Chen. 2008. Learning a Hidden Graph Using O(log n) Queries Per Edge. J. Comput. System Sci., Vol. 74, 4 (2008).
[4]
Mikhail Belkin and Partha Niyogi. 2001. Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering. In NeurIPS.
[5]
Mathilde Bouvel, Vladimir Grebinski, and Gregory Kucherov. 2005. Combinatorial Search on Graphs Motivated by Bioinformatics Applications: A Brief Survey. In WG.
[6]
Hongyun Cai, Vincent W Zheng, and Kevin Chen-Chuan Chang. 2018. A Comprehensive Survey of Graph Embedding: Problems, Techniques and Applications. IEEE Transactions on Knowledge and Data Engineering, Vol. 30, 9 (2018).
[7]
Yukuo Cen, Zhenyu Hou, Yan Wang, Qibin Chen, Yizhen Luo, Xingcheng Yao, Aohan Zeng, Shiguang Guo, Peng Zhang, Guohao Dai, Yu Wang, Chang Zhou, Hongxia Yang, and Jie Tang. 2021. CogDL: Toolkit for Deep Learning on Graphs. arXiv preprint arXiv:2103.00959 (2021).
[8]
Jianxin Chang, Chen Gao, Yu Zheng, Yiqun Hui, Yanan Niu, Yang Song, Depeng Jin, and Yong Li. 2021. Sequential Recommendation with Graph Neural Networks. In SIGIR.
[9]
Sudhanshu Chanpuriya, Cameron Musco, Konstantinos Sotiropoulos, and Charalampos E Tsourakakis. 2021. DeepWalking Backwards: From Embeddings Back to Graphs. In NeurIPS.
[10]
Min Chen, Zhikun Zhang, Tianhao Wang, Michael Backes, Mathias Humbert, and Yang Zhang. 2021. When Machine Unlearning Jeopardize Privacy. In ACM CCS.
[11]
Min Chen, Zhikun Zhang, Tianhao Wang, Michael Backes, Mathias Humbert, and Yang Zhang. 2022. Graph Unlearning. In ACM CCS.
[12]
Yu Chen, Lingfei Wu, and Mohammed Zaki. 2020. Iterative Deep Graph Learning for Graph Neural Networks: Better and Robust Node Embeddings. In NeurIPS.
[13]
Aaron Clauset, Cristopher Moore, and Mark EJ Newman. 2008. Hierarchical Structure and the Prediction of Missing Links in Networks. Nature (2008).
[14]
Andrea Continella, Mario Polino, Marcello Pogliani, and Stefano Zanero. 2018. There's a Hole in that Bucket! A Large-scale Analysis of Misconfigured S3 Buckets. In ACSAC.
[15]
Anirban Dasgupta, Ravi Kumar, and Tamas Sarlos. 2014. On Estimating the Average Degree. In WWW.
[16]
Vasisht Duddu, Antoine Boutet, and Virat Shejwalkar. 2020. Quantifying Privacy Leakage in Graph Embedding. arXiv preprint arXiv:1912.10979 (2020).
[17]
Daniel M Dunlavy, Tamara G Kolda, and Evrim Acar. 2011. Temporal Link Prediction using Matrix and Tensor Factorizations. TKDD, Vol. 5, 2 (2011).
[18]
Ming Fan, Xiapu Luo, Jun Liu, Meng Wang, Chunyin Nong, Qinghua Zheng, and Ting Liu. 2019. Graph Embedding based Familial Analysis of Android Malware using Unsupervised Learning. In ICSE.
[19]
Luca Franceschi, Mathias Niepert, Massimiliano Pontil, and Xiao He. 2019. Learning Discrete Structures for Graph Neural Networks. In ICML.
[20]
Jian Gao, Xin Yang, Ying Fu, Yu Jiang, and Jiaguang Sun. 2018. VulSeeker: A Semantic Learning Based Vulnerability Seeker for Cross-Platform Binary. In ASE.
[21]
Palash Goyal and Emilio Ferrara. 2018. Graph Embedding Techniques, Applications, and Performance: A Survey. Knowledge-Based Systems, Vol. 151 (2018).
[22]
Aditya Grover and Jure Leskovec. 2016. node2vec: Scalable Feature Learning for Networks. In KDD.
[23]
Roger Guimerà and Marta Sales-Pardo. 2009. Missing and Spurious Interactions and the Reconstruction of Complex Networks. PNAS (2009).
[24]
Emil Julius Gumbel. 1954. Statistical Theory of Extreme Values and Some Practical Applications: A Series of Lectures.
[25]
William L Hamilton, Rex Ying, and Jure Leskovec. 2017. Inductive Representation Learning on Large Graphs. In NeurIPS.
[26]
Stephen J Hardiman and Liran Katzir. 2013. Estimating Clustering Coefficients and Size of Social Networks via Random Walk. In WWW.
[27]
Michael Hay, Chao Li, Gerome Miklau, and David Jensen. 2009. Accurate Estimation of the Degree Distribution of Private Networks. In ICDM.
[28]
Xinlei He, Jinyuan Jia, Michael Backes, Neil Zhenqiang Gong, and Yang Zhang. 2021. Stealing Links from Graph Neural Networks. In USENIX Security.
[29]
Danny Hermelin, Avivit Levy, Oren Weimann, and Raphael Yuster. 2011. Distance Oracles for Vertex-Labeled Graphs. In ICALP.
[30]
Darko Hric, Tiago P Peixoto, and Santo Fortunato. 2016. Network Structure, Metadata, and the Prediction of Missing Nodes and Annotations. Physical Review X (2016).
[31]
Shouling Ji, Weiqing Li, Prateek Mittal, Xin Hu, and Raheem Beyah. 2015. Secgraph: A uniform and open-source evaluation system for graph data anonymization and de-anonymization. In USENIX Security.
[32]
Shouling Ji, Weiqing Li, Mudhakar Srivatsa, and Raheem Beyah. 2014. Structural Data De-anonymization: Quantification, Practice, and Implications. In ACM CCS.
[33]
Shouling Ji, Prateek Mittal, and Raheem Beyah. 2016. Graph Data Anonymization, De-Anonymization Attacks, and De-Anonymizability Quantification: A Survey. IEEE Communications Surveys & Tutorials (2016).
[34]
Ian T Jolliffe and Jorge Cadima. 2016. Principal Component Analysis: A Review and Recent Developments. Philos. Trans. A Math. Phys. Eng Sci. (2016).
[35]
Vassilis Kalofolias. 2016. How to Learn a Graph from Smooth Signals. In AISTATS.
[36]
Anees Kazi, Luca Cosmo, Nassir Navab, and Michael Bronstein. 2020. Differentiable Graph Module (DGM) for Graph Convolutional Networks. arXiv preprint arXiv:2002.04999 (2020).
[37]
Steven Kearnes, Kevin McCloskey, Marc Berndl, Vijay Pande, and Patrick Riley. 2016. Molecular Graph Convolutions: Moving Beyond Fingerprints. Journal of Computer-Aided Molecular Design (2016).
[38]
Thomas N Kipf and Max Welling. 2016a. Semi-Supervised Classification with Graph Convolutional Networks. In ICLR.
[39]
Thomas N Kipf and Max Welling. 2016b. Variational Graph Auto-Encoders. arXiv preprint arXiv:1611.07308 (2016).
[40]
Jon Kleinberg. 2000. The Small-World Phenomenon: An Algorithmic Perspective. In ACM STOC.
[41]
Wouter Kool, Herke Van Hoof, and Max Welling. 2019. Stochastic Beams and Where to Find Them: The Gumbel-Top-k Trick for Sampling Sequences Without Replacement. In ICML.
[42]
Josef Lauri and Raffaele Scapellato. 2016. Topics in Graph Automorphisms and Reconstruction. Cambridge University Press.
[43]
Jure Leskovec and Christos Faloutsos. 2006. Sampling from Large Graphs. In KDD.
[44]
Xiaoxiao Li, Joao Saúde, Prashant Reddy, and Manuela Veloso. 2020. Classifying and Understanding Financial Data Using Graph Neural Network. In The AAAI Workshop on Knowledge Discovery from Unstructured Data in Financial Services (KDF). AAAI.
[45]
David Liben-Nowell and Jon Kleinberg. 2007. The Link-Prediction Problem for Social Networks. JAIST, Vol. 58, 7 (2007), 1019--1031.
[46]
Jenny Liu, Aviral Kumar, Jimmy Ba, Jamie Kiros, and Kevin Swersky. 2019. Graph normalizing flows. NeurIPS, Vol. 32 (2019).
[47]
Kun Liu and Evimaria Terzi. 2008. Towards Identity Anonymization on Graphs. In SIGMOD.
[48]
Linyuan Lü and Tao Zhou. 2011. Link Prediction in Complex Networks: A Survey. Physica A, Vol. 390, 6 (2011).
[49]
Mohammad Malekzadeh, Anastasia Borovykh, and Deniz Gündüz. 2021. Honest-but-Curious Nets: Sensitive Attributes of Private Inputs can be Secretly Coded into the Entropy of Classifiers' Outputs. In ACM CCS.
[50]
Fragkiskos D Malliaros and Michalis Vazirgiannis. 2013. Clustering and Community Detection in Directed Networks: A Survey. Physics reports, Vol. 533, 4 (2013).
[51]
Julian J McAuley and Jure Leskovec. 2012. Learning to Discover Social Circles in Ego Networks. In NeurIPS.
[52]
Arvind Narayanan and Vitaly Shmatikov. 2009. De-anonymizing social networks. In IEEE S&P.
[53]
Hongbin Pei, Bingzhe Wei, Kevin Chen-Chuan Chang, Yu Lei, and Bo Yang. 2020. Geom-GCN: Geometric Graph Convolutional Networks. arXiv preprint arXiv:2002.05287 (2020).
[54]
Abdurrahman Pektacs and Tankut Acarman. 2020. Deep Learning for Effective Android Malware Detection using API Call Graph Embeddings. Soft Computing, Vol. 24, 2 (2020).
[55]
Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. 2014. Deepwalk: Online Learning of Social Representations. In KDD.
[56]
Jiezhong Qiu, Yuxiao Dong, Hao Ma, Jian Li, Chi Wang, Kuansan Wang, and Jie Tang. 2019. NetSMF: Large-Scale Network Embedding as Sparse Matrix Factorization. In WWW.
[57]
Jiezhong Qiu, Yuxiao Dong, Hao Ma, Jian Li, Kuansan Wang, and Jie Tang. 2018a. Network Embedding as Matrix Factorization: Unifying DeepWalk, LINE, PTE, and node2vec. In WSDM.
[58]
Jiezhong Qiu, Jian Tang, Hao Ma, Yuxiao Dong, Kuansan Wang, and Jie Tang. 2018b. DeepInf: Social Influence Prediction with Deep Learning. In KDD.
[59]
Veronica Red, Eric D Kelsic, Peter J Mucha, and Mason A Porter. 2011. Comparing Community Structure to Characteristics in Online Collegiate Social Networks. SIAM review, Vol. 53, 3 (2011).
[60]
Benjamin Ricaud, Nicolas Aspert, and Volodymyr Miz. 2020. Spikyball Sampling: Exploring Large Networks via an Inhomogeneous Filtered Diffusion. Algorithms (2020).
[61]
Ryan A. Rossi and Nesreen K. Ahmed. 2015. The Network Data Repository with Interactive Graph Analytics and Visualization. In AAAI.
[62]
Benedek Rozemberczki, Oliver Kiss, and Rik Sarkar. 2020. Little Ball of Fur: A Python Library for Graph Sampling. In CIKM.
[63]
Guillaume Salha, Romain Hennequin, and Michalis Vazirgiannis. 2020. Simple and Effective Graph Autoencoders with One-Hop Linear Models. In ECML-PKDD.
[64]
Yun Shen, Xinlei He, Yufei Han, and Yang Zhang. 2022. Model Stealing Attacks Against Inductive Graph Neural Networks. In IEEE S&P.
[65]
Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. 2017. Membership Inference Attacks against Machine Learning Models. In IEEE S&P.
[66]
Sigal Sina, Avi Rosenfeld, and Sarit Kraus. 2013. Solving the Missing Node Problem using Structure and Attribute Information. In ASONAM.
[67]
Congzheng Song, Thomas Ristenpart, and Vitaly Shmatikov. 2017. Machine Learning Models that Remember Too Much. In ACM CCS.
[68]
Lichao Sun, Yingtong Dou, Carl Yang, Ji Wang, Philip S Yu, Lifang He, and Bo Li. 2018. Adversarial Attack and Defense on Graph Data: A Survey. arXiv preprint arXiv:1812.10528 (2018).
[69]
Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier, and Guillaume Bouchard. 2016. Complex Embeddings for Simple Link Prediction. In ICML.
[70]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is All you Need. In NeurIPS.
[71]
Binghui Wang, Jinyuan Jia, and Neil Zhenqiang Gong. 2019a. Graph-based Security and Privacy Analytics via Collective Classification with Joint Weight Learning and Propagation. In NDSS.
[72]
Jianyu Wang, Rui Wen, Chunming Wu, Yu Huang, and Jian Xion. 2019b. FdGars: Fraudster Detection via Graph Convolutional Networks in Online App Review System. In WWW.
[73]
Fan Wu, Yunhui Long, Ce Zhang, and Bo Li. 2022. LinkTeller: Recovering Private Edges from Graph Neural Networks via Influence Analysis. In IEEE S&P.
[74]
Zhaohan Xi, Ren Pang, Shouling Ji, and Ting Wang. 2021. Graph Backdoor. In USENIX Security.
[75]
Liu Yang and Rong Jin. 2006. Distance Metric Learning: A Comprehensive Survey. Michigan State Universiy, Vol. 2, 2 (2006).
[76]
Liang Yang, Zesheng Kang, Xiaochun Cao, Di Jin, Bo Yang, and Yuanfang Guo. 2019a. Topology Optimization based Graph Convolutional Network. In IJCAI.
[77]
Qiang Yang, Yang Liu, Tianjian Chen, and Yongxin Tong. 2019b. Federated Machine Learning: Concept and Applications. ACM Transactions on Intelligent Systems and Technology (TIST) (2019).
[78]
Zhilin Yang, William Cohen, and Ruslan Salakhudinov. 2016. Revisiting Semi-Supervised Learning with Graph Embeddings. In ICML.
[79]
Daokun Zhang, Jie Yin, Xingquan Zhu, and Chengqi Zhang. 2018. Network Representation Learning: A Survey. IEEE transactions on Big Data (2018).
[80]
Muhan Zhang and Yixin Chen. 2018. Link Prediction Based on Graph Neural Networks. In NeurIPS.
[81]
Minxing Zhang, Zhaochun Ren, Zihan Wang, Pengjie Ren, Zhunmin Chen, Pengfei Hu, and Yang Zhang. 2021b. Membership Inference Attacks Against Recommender Systems. In ACM CCS.
[82]
Zhikun Zhang, Min Chen, Michael Backes, Yun Shen, and Yang Zhang. 2022. Inference Attacks Against Graph Neural Networks. In USENIX Security.
[83]
Zaixi Zhang, Jinyuan Jia, Binghui Wang, and Neil Zhenqiang Gong. 2021a. Backdoor Attacks to Graph Neural Nnetworks. In SACMAT.
[84]
Zhikun Zhang, Tianhao Wang, Ninghui Li, Jean Honorio, Michael Backes, Shibo He, Jiming Chen, and Yang Zhang. 2021c. PrivSyn: Differentially Private Data Synthesis. In USENIX Security.
[85]
Gang Zhao and Jeff Huang. 2018. Deepsim: Deep Learning Code Functional Similarity. In ESEC/FSE.
[86]
Jianan Zhao, Xiao Wang, Chuan Shi, Binbin Hu, Guojie Song, and Yanfang Ye. 2021. Heterogeneous Graph Structure Learning for Graph Neural Networks. In AAAI.
[87]
Bin Zhou, Jian Pei, and WoShun Luk. 2008. A Brief Survey on Anonymization Techniques for Privacy Preserving Publishing of Social Network Data. ACM SIGKDD Explorations Newsletter (2008).
[88]
Yaqin Zhou, Shangqing Liu, Jingkai Siow, Xiaoning Du, and Yang Liu. 2019. Devign: Effective Vulnerability Identification by Learning Comprehensive Program Semantics via Graph Neural Networks. In NeurIPS.
[89]
Yanqiao Zhu, Weizhi Xu, Jinghao Zhang, Qiang Liu, Shu Wu, and Liang Wang. 2021. Deep Graph Structure Learning for Robust Representations: A Survey. arXiv preprint arXiv:2103.03036 (2021).

Cited By

View all
  • (2024)Adversarial Reconstruction of Trajectories: Privacy Risks and Attack Models in Trajectory EmbeddingProceedings of the 32nd ACM International Conference on Advances in Geographic Information Systems10.1145/3678717.3691274(259-269)Online publication date: 29-Oct-2024
  • (2024)Privacy-Preserving Network Embedding Against Private Link Inference AttacksIEEE Transactions on Dependable and Secure Computing10.1109/TDSC.2023.326411021:2(847-859)Online publication date: Mar-2024
  • (2023)PrivGraphProceedings of the 32nd USENIX Conference on Security Symposium10.5555/3620237.3620419(3241-3258)Online publication date: 9-Aug-2023
  • Show More Cited By

Index Terms

  1. Finding MNEMON: Reviving Memories of Node Embeddings

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      CCS '22: Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security
      November 2022
      3598 pages
      ISBN:9781450394505
      DOI:10.1145/3548606
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 07 November 2022

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. graph embedding
      2. machine learning security and privacy

      Qualifiers

      • Research-article

      Funding Sources

      Conference

      CCS '22
      Sponsor:

      Acceptance Rates

      Overall Acceptance Rate 1,261 of 6,999 submissions, 18%

      Upcoming Conference

      CCS '25

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)206
      • Downloads (Last 6 weeks)33
      Reflects downloads up to 20 Jan 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)Adversarial Reconstruction of Trajectories: Privacy Risks and Attack Models in Trajectory EmbeddingProceedings of the 32nd ACM International Conference on Advances in Geographic Information Systems10.1145/3678717.3691274(259-269)Online publication date: 29-Oct-2024
      • (2024)Privacy-Preserving Network Embedding Against Private Link Inference AttacksIEEE Transactions on Dependable and Secure Computing10.1109/TDSC.2023.326411021:2(847-859)Online publication date: Mar-2024
      • (2023)PrivGraphProceedings of the 32nd USENIX Conference on Security Symposium10.5555/3620237.3620419(3241-3258)Online publication date: 9-Aug-2023
      • (2023)Devil in Disguise: Breaching Graph Neural Networks Privacy through InfiltrationProceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security10.1145/3576915.3623173(1153-1167)Online publication date: 15-Nov-2023
      • (2023)"Get in Researchers; We're Measuring Reproducibility": A Reproducibility Study of Machine Learning Papers in Tier 1 Security ConferencesProceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security10.1145/3576915.3623130(3433-3459)Online publication date: 15-Nov-2023
      • (2023)Making Watermark Survive Model Extraction Attacks in Graph Neural NetworksICC 2023 - IEEE International Conference on Communications10.1109/ICC45041.2023.10278974(57-62)Online publication date: 28-May-2023
      • (2023)Predicting merchant future performance using privacy-safe network-based featuresScientific Reports10.1038/s41598-023-36624-013:1Online publication date: 21-Jun-2023

      View Options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Login options

      Media

      Figures

      Other

      Tables

      Share

      Share

      Share this Publication link

      Share on social media