skip to main content
10.1145/3543873.3587313acmconferencesArticle/Chapter ViewAbstractPublication PagesthewebconfConference Proceedingsconference-collections
poster

Knowledge Distillation on Cross-Modal Adversarial Reprogramming for Data-Limited Attribute Inference

Published: 30 April 2023 Publication History

Abstract

Social media generates a rich source of text data with intrinsic user attributes (e.g., age, gender), where different parties benefit from disclosing them. Attribute inference can be cast as a text classification problem, which, however, suffers from labeled data scarcity. To address this challenge, we propose a data-limited learning model to distill knowledge on adversarial reprogramming of a visual transformer (ViT) for attribute inferences. Not only does this novel cross-modal model transfers the powerful learning capability from ViT, but also leverages unlabeled texts to reduce the demand on labeled data. Experiments on social media datasets demonstrate the state-of-the-art performance of our model on data-limited attribute inferences.

References

[1]
Lingwei Chen, Yujie Fan, and Yanfang Ye. 2021. Adversarial Reprogramming of Pretrained Neural Networks for Fraud Detection. In CIKM. 2935–2939.
[2]
Lingwei Chen, Xiaoting Li, and Dinghao Wu. 2022. Adversarially Reprogramming Pretrained Neural Networks for Data-limited and Cost-efficient Malware Detection. In SDM. SIAM, 693–701.
[3]
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. Proceedings of NAACL-HLT (2019).
[4]
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, 2021. An image is worth 16x16 words: Transformers for image recognition at scale. ICLR (2021).
[5]
Gamaleldin F Elsayed, Ian Goodfellow, and Jascha Sohl-Dickstein. 2018. Adversarial reprogramming of neural networks. ICLR (2018).
[6]
Chelsea Finn, Pieter Abbeel, and Sergey Levine. 2017. Model-agnostic meta-learning for fast adaptation of deep networks. In ICML. 1126–1135.
[7]
Victor Garcia and Joan Bruna. 2018. Few-shot learning with graph neural networks. International Conference on Learning Representations (2018).
[8]
Karen Hambardzumyan, Hrant Khachatrian, and Jonathan May. 2021. Warp: Word-level adversarial reprogramming. ACL (2021).
[9]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In CVPR. 770–778.
[10]
Geoffrey Hinton, Oriol Vinyals, Jeff Dean, 2015. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 2, 7 (2015).
[11]
Lianzhe Huang, Dehong Ma, Sujian Li, Xiaodong Zhang, and Houfeng Wang. 2019. Text level graph neural network for text classification. EMNLP (2019).
[12]
Forrest Iandola, Matt Moskewicz, Sergey Karayev, Ross Girshick, Trevor Darrell, and Kurt Keutzer. 2014. Densenet: Implementing efficient convnet descriptor pyramids. arXiv preprint arXiv:1404.1869 (2014).
[13]
Ashraful Islam, Chun-Fu Richard Chen, Rameswar Panda, Leonid Karlinsky, Rogerio Feris, and Richard J Radke. 2021. Dynamic distillation network for cross-domain few-shot recognition with unlabeled data. NeurIPS (2021).
[14]
Jinyuan Jia and Neil Zhenqiang Gong. 2018. Attriguard: A practical defense against attribute inference attacks via adversarial machine learning. In 27th USENIX Security Symposium (USENIX Security 18). 513–529.
[15]
Quan Li, Xiaoting Li, Lingwei Chen, and Dinghao Wu. 2022. Distilling Knowledge on Text Graph for Social Media Attribute Inference. In SIGIR. 2024–2028.
[16]
Xiaoting Li, Lingwei Chen, and Dinghao Wu. 2021. Turning Attacks into Protection: Social Media Privacy Protection Using Adversarial Attacks. In SDM. 208–216.
[17]
Chung-Ying Lin. 2020. Social reaction toward the 2019 novel coronavirus (COVID-19). Social Health and Behavior 3, 1 (2020), 1.
[18]
Hu Linmei, Tianchi Yang, Chuan Shi, Houye Ji, and Xiaoli Li. 2019. Heterogeneous graph attention networks for semi-supervised short text classification. In EMNLP-IJCNLP. 4821–4830.
[19]
Yanbin Liu, Juho Lee, Minseop Park, Saehoon Kim, Eunho Yang, Sung Ju Hwang, and Yi Yang. 2018. Learning to propagate labels: Transductive propagation network for few-shot learning. arXiv preprint arXiv:1805.10002 (2018).
[20]
Nikola Mrkšić, Diarmuid O Séaghdha, Blaise Thomson, Milica Gašić, Lina Rojas-Barahona, Pei-Hao Su, David Vandyke, Tsung-Hsien Wen, and Steve Young. 2016. Counter-fitting word vectors to linguistic constraints. NAACL (2016).
[21]
Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. Glove: Global vectors for word representation. In EMNLP. 1532–1543.
[22]
Matthew E Peters, Sebastian Ruder, and Noah A Smith. 2019. To tune or not to tune? adapting pretrained representations to diverse tasks. arXiv preprint arXiv:1903.05987 (2019).
[23]
Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language understanding by generative pre-training. (2018).
[24]
Nils Reimers. 2019. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. In ACL. Association for Computational Linguistics.
[25]
Jonathan Schler, Moshe Koppel, Shlomo Argamon, and James W Pennebaker. 2006. Effects of age and gender on blogging. In AAAI spring symposium: Computational approaches to analyzing weblogs, Vol. 6. 199–205.
[26]
Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. 2016. Rethinking the inception architecture for computer vision. In CVPR. 2818–2826.
[27]
Risto Vuorio, Shao-Hua Sun, Hexiang Hu, and Joseph J Lim. 2019. Multimodal model-agnostic meta-learning via task-aware modulation. Advances in Neural Information Processing Systems 32 (2019).
[28]
Yaqing Wang, Song Wang, Quanming Yao, and Dejing Dou. 2021. Hierarchical Heterogeneous Graph Representation Learning for Short Text Classification. arXiv preprint arXiv:2111.00180 (2021).
[29]
Huaxiu Yao, Ying Wei, Junzhou Huang, and Zhenhui Li. 2019. Hierarchically structured meta-learning. In International Conference on Machine Learning. PMLR, 7045–7054.
[30]
Huaxiu Yao, Ying Wei, Long-Kai Huang, Ding Xue, Junzhou Huang, and Zhenhui Jessie Li. 2021. Functionally Regionalized Knowledge Transfer for Low-resource Drug Discovery. NeurIPS 34 (2021).
[31]
Liang Yao, Chengsheng Mao, and Yuan Luo. 2019. Graph convolutional networks for text classification. In AAAI, Vol. 33. 7370–7377.
[32]
Yanfang Ye, Shifu Hou, Yujie Fan, Yiyue Qian, Yiming Zhang, Shiyu Sun, Qian Peng, and Kenneth Laparo. 2020. α -Satellite: An AI-driven System and Benchmark Datasets for Hierarchical Community-level Risk Assessment to Help Combat COVID-19. arXiv preprint arXiv:2003.12232 (2020).
[33]
Sixie Yu, Yevgeniy Vorobeychik, and Scott Alfeld. 2018. Adversarial classification on social networks. In International Conference on Autonomous Agents and MultiAgent Systems. 211–219.
[34]
Yufeng Zhang, Xueli Yu, Zeyu Cui, Shu Wu, Zhongzhen Wen, and Liang Wang. 2020. Every document owns its structure: Inductive text classification via graph neural networks. arXiv preprint arXiv:2004.13826 (2020).

Cited By

View all
  • (2023)Adversary for Social Good: Leveraging Adversarial Attacks to Protect Personal Attribute PrivacyACM Transactions on Knowledge Discovery from Data10.1145/361409818:2(1-24)Online publication date: 13-Nov-2023
  • (2023)Pseudo-Labeling with Graph Active Learning for Few-shot Node Classification2023 IEEE International Conference on Data Mining (ICDM)10.1109/ICDM58522.2023.00133(1115-1120)Online publication date: 1-Dec-2023
  • (2023)DISCERN: Leveraging Knowledge Distillation to Generate High Resolution Soil Moisture Estimation from Coarse Satellite Data2023 IEEE International Conference on Big Data (BigData)10.1109/BigData59044.2023.10386179(1222-1229)Online publication date: 15-Dec-2023

Index Terms

  1. Knowledge Distillation on Cross-Modal Adversarial Reprogramming for Data-Limited Attribute Inference

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      WWW '23 Companion: Companion Proceedings of the ACM Web Conference 2023
      April 2023
      1567 pages
      ISBN:9781450394192
      DOI:10.1145/3543873
      Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 30 April 2023

      Check for updates

      Author Tags

      1. Adversarial Reprogramming
      2. Attribute Inference
      3. Data-limited Learning
      4. Knowledge Distillation.

      Qualifiers

      • Poster
      • Research
      • Refereed limited

      Conference

      WWW '23
      Sponsor:
      WWW '23: The ACM Web Conference 2023
      April 30 - May 4, 2023
      TX, Austin, USA

      Acceptance Rates

      Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)55
      • Downloads (Last 6 weeks)1
      Reflects downloads up to 20 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2023)Adversary for Social Good: Leveraging Adversarial Attacks to Protect Personal Attribute PrivacyACM Transactions on Knowledge Discovery from Data10.1145/361409818:2(1-24)Online publication date: 13-Nov-2023
      • (2023)Pseudo-Labeling with Graph Active Learning for Few-shot Node Classification2023 IEEE International Conference on Data Mining (ICDM)10.1109/ICDM58522.2023.00133(1115-1120)Online publication date: 1-Dec-2023
      • (2023)DISCERN: Leveraging Knowledge Distillation to Generate High Resolution Soil Moisture Estimation from Coarse Satellite Data2023 IEEE International Conference on Big Data (BigData)10.1109/BigData59044.2023.10386179(1222-1229)Online publication date: 15-Dec-2023

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      HTML Format

      View this article in HTML Format.

      HTML Format

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media