Elsevier

Information Sciences

Volume 613, October 2022, Pages 50-68
Information Sciences

Compound adversarial examples in deep neural networks

https://doi.org/10.1016/j.ins.2022.08.031Get rights and content

Highlights

  • We propose two algorithms to optimize adversarial perturbation and patch.

  • We discover that CAEs can quickly decrease the classification accuracy.

  • We find that two weak attack patterns can be combined to perform stronger attack.

  • We demonstrate the effectiveness and robustness of the proposed method.

Abstract

Although deep learning has made great progress in many fields, they are still vulnerable to adversarial examples. Many methods for generating adversarial examples have been proposed, which either contain adversarial perturbation or patch. In this paper, we explore the method that creates compound adversarial examples including both perturbation and patch. We show that fusing two weak attack modes can produce more powerful adversarial examples, where the patch covers only 1% of the pixels at random location in the image, and the perturbation changes only by 2/255 in the original pixel value (scale to 0–1). For both targeted attack and untargeted attack, compound attack can improve the generative efficiency of adversarial examples, and can attain higher attack success rate with fewer iteration steps. The compound adversarial examples successfully attack the models with defensive mechanisms that previously can defend perturbation attack or patch attack. Furthermore, the compound adversarial examples show good transferability on normal trained classifiers and adversarial trained classifiers. Experimental results on a series of widely used classifiers and defense models show that the proposed compound adversarial examples have strong robustness, high effectiveness, and good transferability.

Introduction

As one of the key technologies of artificial intelligence, deep learning technology has been extensively applied in many fields, and has achieved remarkable results. When applying deep neural networks (DNNs) in security sensitive tasks such as face recognition [34], social analysis [49], automatic driving and medical imaging [48], the security problem of deep learning has become increasingly prominent. The adversarial examples [41], [5] are carefully designed to deceive the deep neural network and can lead the deep neural network to output wrong results, which has raised considerable security concerns in the development of deep learning.

Adversarial examples are suitable for various scenarios. The digital images 1 from the Internet may suffer from attack of adversarial examples, and the printed adversarial examples can perform real-world attack to systems based on DNNs. The examples generated by adversarial attacks will not affect humans’ understanding the semantic information of the images, but will make the state-of-the-art deep learning systems classification or recognition errors. Adversarial examples have drawn attention to the security applications of deep models and have provided insights into the advantages, disadvantages, and blind spots of these ubiquitous classification models [43], [15].

Adversarial examples can be generally divided into two categories: one is the adversarial perturbation [41] that is added to the whole original image and is inperceptible to human eyes; the other one is the adversarial patch [5] that covers only a small area of the original image and is visible. Both adversarial patch and adversarial perturbation can fool the classifiers, but they usually have different application scenarios. With rich colors and small size, adversarial patch can be printed and attached to any real images to implement physical world attack. Adversarial perturbation, being invisible, is more prone to attacking digital images. However, the recent PhysGAN [21] has generated physical-world-resilient perturbations that have successfully misled the autonomous driving systems. Both adversarial patch and adversarial perturbation can attack the digital images and real world images, prompting us to consider compound attack with perturbation and patch in the same image.

In recent years, many adversarial example generation methods have been proposed, where the adversarial perturbations and patches are studied independently, and produce adversarial perturbations or adversarial patches. In order to attack successfully, the value of perturbation is usually set to such a high level as 16/255, 8/255 [29], [14], which makes the adversarial examples obviously different from the normal samples. When the perturbation value is small, the number of iterations will reach hundreds or thousands, resulting in extra consumption of time and computing resources. For producing a successful adversarial patch, the size of the patch is either too large to cover about 15% of the image [5], or too many iteration steps are required [20].

In this paper, to improve the efficiency and efficacy of the previous adversarial example generation methods, we propose a method that produces compound adversarial example with perturbation and patch at the same time. In order to solve the problem of high value perturbation and large patch size, we constrain the upper bound of perturbation to 2/255, and the patch size to 1% of the image size. We generate compound adversarial examples on two datasets with different resolution and categories for both targeted attack and untargeted attack. For untargeted attack, we first study the relationship between attack intensity and perturbation value and patch size. We then present the attack success rates (ASR) of the three modes with different iteration steps on the popular classifiers, and find that compound attack is an integration of superiority of perturbation and patch attack, and its ASR is greater than the sum of the two. Moreover, we compare the performance of three types of adversarial examples on various classifiers and datasets under the same iteration steps for targeted attack. On CIFAR-10 [22], we study the performance of compound adversarial examples in decreasing the classification accuracy. We also analyze the attack performance of compound adversarial examples on two certification defensive models and several adversarial trained models. Finally, we study the transferability of the compound adversarial examples on normal trained classifiers and adversarial trained classifiers.

In summary, the main contributions are as follows:

  • We present a novel framework for the generation process of compound adversarial examples and propose two new algorithms to simultaneously optimize adversarial perturbation and patch for targeted attack and untargeted attack, respectively. We also find that these two weak attack patterns can be combined to perform stronger attack, leading to better generation efficiency. To the best of our knowledge, we are the first to generate compound adversarial examples.

  • We discover that compound adversarial examples can quickly decrease the classification accuracy of both normal classifiers and defense classifiers. For normal classifiers, compound attack can quickly generate compound adversarial examples with the highest success rate; for defense classifiers, compound attack breaks through the certification methods that are beneficial for patch attack or perturbation attack.

  • We validate that the compound adversarial examples can perform black-box attack with moderate success rate for normal trained classifiers and adversarial trained classifiers.

  • We demonstrate the effectiveness and robustness of the proposed method on a set of state-of-the-art classifiers and several datasets.

The rest of the paper is organized as follows. In Section 2, we briefly review the generation and defense methods of adversarial attack as well as the methods that enhance the success rate of black-box attack. Section 3 introduces the compound attack and provides the algorithms for targeted attack and untargeted attack. In Section 4, sufficient experiments on untargeted attack, targeted attack, defense models and black-box attack are conducted to verify the superiority of compound adversarial examples, where we also analyze deeply on the experimental results. We conclude our research and discuss possible future work in Section 5.

Section snippets

Related Works

In recent years, a particularly compelling challenge has emerged in the field of computer vision: making very small changes to the input image can deceive the most advanced deep neural networks with high confidence. The adversarial examples have revealed the vulnerability and sensitivity of DNNs, but they can also provide valuable insights for deep models. As our method is based on perturbation attack and patch attack, we first focus on outlining some generation methods of these two attack

Compound Attack

In this section, we introduce our compound attack in details. We first introduce the assumptions and notations to clarify the attack scenarios and the mathematical symbols. Then the untargeted attack and targeted attack algorithms of compound attack are introduced, including constraints, optimization functions and pseudo codes. Finally, the compound attack processes on several defensive models are introduced.

Experiment

In this section, we first clarify the settings in the experiments, including attack models and datasets. Next, we evaluate the white-box attack of compound adversarial example on several widely-used deep neural network classifiers and popular datasets. We conduct both untargeted attack and targeted attack. We then study the performance of compound adversarial examples on several defense models. Finally we study the black-box attack performance.

Conclusions

In this paper, we propose two algorithms to generate compound adversarial examples for both untargeted attack and targeted attack. The proposed algorithm is based on the momentum gradient and the random initialization noise. Compared with perturbation attack and patch attack, the compound attack has a higher attack success rate, better generation efficiency and stronger transferability. We also assess the aggression of compound adversarial examples on models with defensive mechanism, and the

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Yanchun Li obtained the B.S., M.S. degrees from the College of Information Engineering of Xiangtan University, Hunan, China, and obtained the Ph.D. degree from the School of Computer Science and Engineering, South China University of Technology, Guangzhou, China. She is now a senior lecturer in School of computer, Xiangtan University. Her research interests include deep learning, computer vision and image processing. She is a member of Chinese Computer Federation (CCF).

References (50)

  • J.Y. Chen et al.

    Black-box adversarial attack on license plate recognition system

    Acta Automatica Sinica

    (2021)
  • P.Y. Chen et al.

    Zoo: Zeroth order optimization based black-box attacks to deep neural networks without training substitute models

  • J.M. Cohen, E. Rosenfeld, J.Z. Kolter, Certified adversarial robustness via randomized smoothing,...
  • K. Deb et al.

    A fast and elitist multiobjective genetic algorithm: Nsga-ii

    IEEE Transactions on Evolutionary Computation

    (2002)
  • J. Deng et al.

    Imagenet: A large-scale hierarchical image database

    Proc of IEEE Computer Vision and Pattern Recognition

    (2009)
  • Dong, Y., Liao, F., Pang, T., Hu, X., Zhu, J., 2017. Discovering adversarial examples with momentum. CoRR...
  • Goodfellow, I.J., Pouget-Abadie, J., Mirza, M., Bing, X., Bengio, Y., 2014. Generative adversarial nets, in: Advances...
  • Goodfellow, I.J., Shlens, J., Szegedy, C., 2015. Explaining and harnessing adversarial examples, in:...
  • Hamm, J., 2017. Machine vs machine: Defending classifiers against learning-based adversarial attacks. CoRR...
  • K. He et al.

    Deep residual learning for image recognition

  • He, K., Zhang, X., Ren, S., Sun, J., 2016b. Identity mappings in deep residual networks, in: Computer Vision – ECCV...
  • G. Huang et al.

    Densely connected convolutional networks

  • G. Huang et al.

    Convolutional networks with dense connectivity

    IEEE Transactions on Pattern Analysis and Machine

    (2019)
  • D. Karmon et al.

    LaVAN: Localized and visible adversarial noise, in

  • Z. Kong et al.

    Physgan: Generating physical-world-resilient adversarial examples for autonomous driving

  • Cited by (5)

    Yanchun Li obtained the B.S., M.S. degrees from the College of Information Engineering of Xiangtan University, Hunan, China, and obtained the Ph.D. degree from the School of Computer Science and Engineering, South China University of Technology, Guangzhou, China. She is now a senior lecturer in School of computer, Xiangtan University. Her research interests include deep learning, computer vision and image processing. She is a member of Chinese Computer Federation (CCF).

    Zhetao Li (Member, IEEE) received the B.S. degree from Xiangtan University, Xiangtan, China, in 2002, the M.S. degree from Beihang University, Beijing, China, in 2005, and the Ph.D. degree in computer application technology from Hunan University, Changsha, China, in 2010. He is currently a professor with the Key Laboratory of Hunan Province for Internet of Things and Information Security and School of Computer Science, Xiangtan University, China. He was a visiting researcher with Ajou University, South Korea from May to August 2012. His research interests include cloud computing, artificial intelligence, and multimedia signal processing.

    Li Zeng received a bachelor’s degree in software engineering and a master’s degree in computer science and technology from Xiangtan University in 2013 and 2016, respectively. She has been working at Xiangtan University since she graduated with a master’s degree. Her current research interests include information security, cryptanalysis, and bioinformatics.

    Saiqin Long received the PhD degree in computer applications technology from the South China University of Technology, Guangzhou, China, in 2014. She is currently a professor with the College of Information Science and Technology, Jinan University, China. Her research interests include cloud computing, edge computing, parallel and distributed systems, and Internet of things. She has published 20 + refereed papers in these areas, most of which are published in premium conferences and journals, including IEEE TSC, IEEE TPDS, IEEE TMC, etc. She is a member of Chinese Computer Federation (CCF).

    Feiran Huang (Member, IEEE) received the B.Sc. degree from Central South University, Changsha, China, in 2011, and the Ph.D. degree in computer software and theory from the School of Computer Science and Engineering, Beihang University, Beijing, China, in 2019. He is currently a Lecturer with the School of Information Science and Technology and College of Cyber Security, Jinan University, Guangzhou, China. His research interests include social media analysis and multimodal learning.

    Kui Ren (Fellow, IEEE) is a professor and associate dean of College of Computer Science and Technology at Zhejiang University, where he also directs the Institute of Cyber Science and Technology. Before that, he was with State University of New York at Buffalo. He received his Ph.D. degree in Electrical and Computer Engineering from Worcester Polytechnic Institute. Kui’s current research interests include Data Security, IoT Security, AI Security, and Privacy. Kui is a Fellow of IEEE, a Distinguished Member of ACM and a Clarivate Highly-Cited Researcher. He is a frequent reviewer for funding agencies internationally and serves on the editorial boards of many IEEE and ACM journals. He currently serves as Chair of SIGSAC of ACM China.

    This work is supported in part by National Key Research and Development Program of China under Grant 2021YFB3101201, Hunan Provincial Natural Science Foundation under Grant 2021JJ30685, Natural Science Foundation of China under Grant 62032020, Natural Science Foundation of China under Grant 62172349, Hunan Province Department of Education under Grant 21B0120, and Hunan Science and Technology Planning Project under Grant No.2019RS3019.

    View full text