Skip to main content
SpringerLink
Log in

Adversarial Attack and Defense on Natural Language Processing in Deep Learning: A Survey and Perspective

  • Conference paper
  • First Online:
Machine Learning for Cyber Security (ML4CS 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13655))

Included in the following conference series:

Abstract

Natural language processing (NLP) presently has become a new paradigm and enables a variety of applications such as text classification, information retrieval, and natural language generation by leveraging deep learning techniques. However, recent studies have shown that the NLP models based on deep neural network are susceptible to maliciously designed adversarial examples. Therefore, the main challenges lie in improving the robustness and ensuring the security of the system. In this paper, we first introduce common NLP tasks and quality measures on adversarial example. Next, we present a comprehensive review on literature in terms of both adversarial attack and defense methods, based upon the granularity and type. Our work is also the first of its kind to provide a brief overview of the adversarial examples on Chinese texts as a result of the language difference. Finally, we summarize this study by providing directions for future research.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015)

    Article  Google Scholar 

  2. Szegedy, C., Zaremba, W., Sutskever, I., et al.: Intriguing properties of neural networks. In: Proceedings of the 2nd International Conference on Learning Representations, pp. 1–10 (2014)

    Google Scholar 

  3. Wenqi, W., Lina, W., Benxiao, T., et al.: Towards a robust deep neural network in text domain: a survey. arXiv preprint arXiv:1902.07285 (2019)

  4. Zhang, W., Sheng, Q., Alhazmi, A., et al.: Adversarial attacks on deep-learning models in natural language processing: a survey. ACM Trans. Intell. Syst. Technol. 11(3), 24:1–24:41 (2020)

    Google Scholar 

  5. Kusner, M., Sun, Y., Kolkin, N., et al.: From word embeddings to document distances. In: Proceedings of the International Conference on Machine Learning, Lille, pp. 957–966. ACM (2015)

    Google Scholar 

  6. Gao, J., Lanchantin, J., Soffa, M.L., et al.: Black-box generation of adversarial text sequences to evade deep learning classifiers. In: Proceedings of the 2018 IEEE Security and Privacy Workshops, San Francisco, pp. 50–56. IEEE (2018)

    Google Scholar 

  7. He, X., Lyu, L., Xu, Q., et al.: Model extraction and adversarial transferability, your BERT is vulnerable!. In: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 2006–2012. NAACL (2021)

    Google Scholar 

  8. Ebrahimi, J., Rao, A., Lowd, D., et al.: HotFlip: white-box adversarial examples for text classification. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, Melbourne, pp. 31–36. ACL (2018)

    Google Scholar 

  9. Gil, Y., Chai, Y., Gorodissky, O., et al.: White-to black: efficient distillation of black-box adversarial attacks. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, pp. 1373–1379. NAACL (2019)

    Google Scholar 

  10. Ebrahimi, J., Lowd, D., Dou, D.: On adversarial examples for character-level neural machine translation. In: Proceedings of the 27th International Conference on Computational Linguistics, Santa Fe, pp. 653–663. ACM (2018)

    Google Scholar 

  11. Ren, S., Deng, Y., He, K., et al.: Generating natural language adversarial examples through probability weighted word saliency. In: Proceedings of the 57th Conference of the Association for Computational Linguistics, Florence, pp. 1085–1097. ACL (2019)

    Google Scholar 

  12. Jin, D., Jin, Z., Zhou, J.T., et al.: Is BERT really robust? A strong baseline for natural language attack on text classification and entailment. In: Proceedings of the Thirty-Fourth AAAI Conference on Artificial Intelligence, New York, pp. 8018–8025. AAAI (2020)

    Google Scholar 

  13. Emmery, C., Kadar, A., Chrupala, G.: Adversarial stylometry in the wild: transferable lexical substitution attacks on author profiling. In: Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pp. 2388–2402. ACL (2021)

    Google Scholar 

  14. Maheshwary, R., Maheshwary, S., Pudi, V.: A strong baseline for query efficient attacks in a black box setting. In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, Punta Cana, pp. 8396–8409. ACL (2021)

    Google Scholar 

  15. Zhang, X., Zhang, J., Chen, Z., et al.: Crafting adversarial examples for neural machine translation. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics, pp. 1967–1977. ACL (2021)

    Google Scholar 

  16. Zeng, Z., Xiong, D.: An empirical study on adversarial attack on NMT: languages and positions matter. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics, pp. 454–460. ACL (2021)

    Google Scholar 

  17. Emelin, D., Titov, I., Sennrich, R.: Detecting word sense disambiguation biases in machine translation for model-agnostic adversarial attacks. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pp. 7635–7653. ACL (2020)

    Google Scholar 

  18. Cheng, Y., Jiang, L., Macherey, W.: Robust neural machine translation with doubly adversarial inputs. In: Proceedings of the 57th Conference of the Association for Computational Linguistics, Florence, pp. 4324–4333. ACL (2019)

    Google Scholar 

  19. Meng, Z., Wattenhofer, R.: A geometry-inspired attack for generating natural language adversarial examples. In: Proceedings of the 28th International Conference on Computational Linguistics, Barcelona, pp. 6679–6689. ACM (2020)

    Google Scholar 

  20. Lin, J., Zou, J., Ding, N.: Using adversarial attacks to reveal the statistical bias in machine reading comprehension models. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pp. 333–342. ACL (2021)

    Google Scholar 

  21. Wang, T., Wang, X., Qin, Y., et al.: CAT-Gen: improving robustness in NLP models via controlled adversarial text generation. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pp. 5141–5146. ACL (2020)

    Google Scholar 

  22. Jia, R., Liang, P.: Adversarial examples for evaluating reading comprehension systems. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, Copenhagen, pp. 2021–2031. ACL (2017)

    Google Scholar 

  23. Wang, B., Pei, H., Pan, B., et al.: T3: tree autoencoder constrained adversarial text generation for targeted attack. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pp. 6134–6150. ACL (2020)

    Google Scholar 

  24. Tan, S., Joty, S.R.: Code-mixing on sesame street: dawn of the adversarial polyglots. In: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 3596–3616. NAACL (2021)

    Google Scholar 

  25. Belinkov, Y., Bisk, Y.: Synthetic and natural noise both break neural machine translation. In: Proceedings of the 6th International Conference on Learning Representations, Vancouver, pp. 1–13. ACM (2018)

    Google Scholar 

  26. Wang, X., Jin, H., Yang, Y., et al.: Natural language adversarial defense through synonym encoding. In: Proceedings of the Thirty-Senventh Conference on Uncertainty in Artificial Intelligence. AUAI (2021)

    Google Scholar 

  27. Zhou, Y., Zheng, X., Hsieh, C.J., et al.: Defense against synonym substitution-based adversarial attacks via Dirichlet neighborhood ensemble. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pp. 5482–5492. ACL (2021)

    Google Scholar 

  28. Bao, R., Wang, J., Zhao, H.: Defending pre-trained language models from adversarial word substitution without performance sacrifice. In: Proceedings of the Findings of the Association for Computational Linguistics: ACL/IJCNLP 2021, pp. 3248–3258. ACL (2021)

    Google Scholar 

  29. Si, C., Zhang, Z., Qi, F., et al.: Better robustness by more coverage: adversarial and mixup data augmentation for robust finetuning. In: Proceedings of the Findings of the Association for Computational Linguistics: ACL/IJCNLP 2021, pp. 1569–1576. ACL (2021)

    Google Scholar 

  30. Wang, X., Yang, Y., Deng, Y., et al.: Adversarial training with fast gradient projection method against synonym substitution based text attacks. In: Proceedings of the Thirty-Fifth AAAI Conference on Artificial Intelligence, pp. 13997–14005. AAAI (2021)

    Google Scholar 

  31. Mozes, M., Stenetorp, P., Kleinberg, B., et al.: Frequency-guided word substitutions for detecting textual adversarial examples. In: Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pp. 171–186. EACL (2021)

    Google Scholar 

  32. Keller, Y., Mackensen, J., Eger, S.: BERT-defense: a probabilistic model based on BERT to combat cognitively inspired orthographic adversarial attacks. In: Proceedings of the Findings of the Association for Computational Linguistics: ACL/IJCNLP 2021, pp. 1616–1629. ACL (2021)

    Google Scholar 

  33. Wang, B., Wang, S., Cheng, Y., et al.: InfoBERT: improving robustness of language models from an information theoretic perspective. In: Proceedings of the 9th International Conference on Learning Representation (2021)

    Google Scholar 

  34. Le, T., Park, N., Lee, D.: A sweet rabbit hole by DARCY: using honeypots to detect universal trigger’s adversarial attacks. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pp. 3831–3844. ACL (2021)

    Google Scholar 

  35. Wang, W., Tang, P., Lou, J., et al.: Certified robustness to word substitution attack with differential privacy. In: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 1102–1112. NAACL (2021)

    Google Scholar 

  36. Ye, M., Gong, C., Liu, Q.: SAFER: a structure-free approach for certified robustness to adversarial word substitution. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 3465–3475. ACL (2020)

    Google Scholar 

  37. Xu, K., Shi, Z., Zhang, H., et al.: Automatic perturbation analysis for scalable certified robustness and beyond. In: Proceedings of the Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020. MIT Press (2020)

    Google Scholar 

  38. Wang, W., Wang, R., Wang, L., Tang, B.: Adversarial examples generation approach for tendency classification on Chinese texts. Ruan Jian Xue Bao/J. Softw. 30(8), 2415–2427 (2019)

    Google Scholar 

  39. Cheng, N., Chang, G., Gao, H., et al.: WordChange: adversarial examples generation approach for Chinese text classification. IEEE Access 8, 79561–79572 (2020)

    Article  Google Scholar 

  40. Yeh, J.F., Lu, Y.Y., Lee, C.H., et al.: Chinese word spelling correction based on rule induction. In: Proceedings of the Third CIPS-SIGHAN Joint Conference on Chinese Language Processing, Wuhan, pp. 139–145. ACL (2014)

    Google Scholar 

  41. Li, J., Du, T., Ji, S., et al.: TextShield: robust text classification based on multimodal embedding and neural machine translation. In: Proceedings of the 29th USENIX Security Symposium, pp. 1381–1398. USENIX Association (2020)

    Google Scholar 

  42. Ian, J.G., Jonathon, S., Christian, S.: Expaining and harnessing adversarial examples. In 3rd International Conference on Learning Representations, San Diego. ICLR (2015)

    Google Scholar 

  43. Ilyas, A., Santurkar, S., Tsipras, D., et al.: Adversarial examples are not bugs, they are features. In: Proceedings of the Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, Vancouver, pp. 125–136. MIT Press (2019)

    Google Scholar 

Download references

Acknowledgements

The work is supported by the National Natural Science Foundation of China under Grant 61972148.

Author information

Authors and Affiliations

  1. School of Control and Computer Engineering, North China Electric Power University, Beijing, 102206, China

    Huoyuan Dong, Jialiang Dong & Zhitao Guan

  2. Department of Finance, Operations, and Information Systems (FOIS), Brock University, St. Catharines, ON, Canada

    Shuai Yuan

Authors
  1. Huoyuan Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jialiang Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shuai Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhitao Guan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhitao Guan .

Editor information

Editors and Affiliations

  1. School of Computing and Informatics, University of Louisiana at Lafayette, Lafayette, IN, USA

    Yuan Xu

  2. Institute of Artificial Intelligence and Blockchain, Guangzhou University, Guangzhou, China

    Hongyang Yan

  3. Institute of Artificial Intelligence and Blockchain, Guangzhou University, Guangzhou, China

    Huang Teng

  4. Guangdong Polytechnic Normal University, Guangzhou, China

    Jun Cai

  5. Institute of Artificial Intelligence and Blockchain, Guangzhou University, Guangzhou, China

    Jin Li

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Dong, H., Dong, J., Yuan, S., Guan, Z. (2023). Adversarial Attack and Defense on Natural Language Processing in Deep Learning: A Survey and Perspective. In: Xu, Y., Yan, H., Teng, H., Cai, J., Li, J. (eds) Machine Learning for Cyber Security. ML4CS 2022. Lecture Notes in Computer Science, vol 13655. Springer, Cham. https://doi.org/10.1007/978-3-031-20096-0_31

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-20096-0_31

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-20095-3

  • Online ISBN: 978-3-031-20096-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation