Compound adversarial examples in deep neural networks
Introduction
As one of the key technologies of artificial intelligence, deep learning technology has been extensively applied in many fields, and has achieved remarkable results. When applying deep neural networks (DNNs) in security sensitive tasks such as face recognition [34], social analysis [49], automatic driving and medical imaging [48], the security problem of deep learning has become increasingly prominent. The adversarial examples [41], [5] are carefully designed to deceive the deep neural network and can lead the deep neural network to output wrong results, which has raised considerable security concerns in the development of deep learning.
Adversarial examples are suitable for various scenarios. The digital images 1 from the Internet may suffer from attack of adversarial examples, and the printed adversarial examples can perform real-world attack to systems based on DNNs. The examples generated by adversarial attacks will not affect humans’ understanding the semantic information of the images, but will make the state-of-the-art deep learning systems classification or recognition errors. Adversarial examples have drawn attention to the security applications of deep models and have provided insights into the advantages, disadvantages, and blind spots of these ubiquitous classification models [43], [15].
Adversarial examples can be generally divided into two categories: one is the adversarial perturbation [41] that is added to the whole original image and is inperceptible to human eyes; the other one is the adversarial patch [5] that covers only a small area of the original image and is visible. Both adversarial patch and adversarial perturbation can fool the classifiers, but they usually have different application scenarios. With rich colors and small size, adversarial patch can be printed and attached to any real images to implement physical world attack. Adversarial perturbation, being invisible, is more prone to attacking digital images. However, the recent PhysGAN [21] has generated physical-world-resilient perturbations that have successfully misled the autonomous driving systems. Both adversarial patch and adversarial perturbation can attack the digital images and real world images, prompting us to consider compound attack with perturbation and patch in the same image.
In recent years, many adversarial example generation methods have been proposed, where the adversarial perturbations and patches are studied independently, and produce adversarial perturbations or adversarial patches. In order to attack successfully, the value of perturbation is usually set to such a high level as 16/255, 8/255 [29], [14], which makes the adversarial examples obviously different from the normal samples. When the perturbation value is small, the number of iterations will reach hundreds or thousands, resulting in extra consumption of time and computing resources. For producing a successful adversarial patch, the size of the patch is either too large to cover about 15% of the image [5], or too many iteration steps are required [20].
In this paper, to improve the efficiency and efficacy of the previous adversarial example generation methods, we propose a method that produces compound adversarial example with perturbation and patch at the same time. In order to solve the problem of high value perturbation and large patch size, we constrain the upper bound of perturbation to 2/255, and the patch size to 1% of the image size. We generate compound adversarial examples on two datasets with different resolution and categories for both targeted attack and untargeted attack. For untargeted attack, we first study the relationship between attack intensity and perturbation value and patch size. We then present the attack success rates (ASR) of the three modes with different iteration steps on the popular classifiers, and find that compound attack is an integration of superiority of perturbation and patch attack, and its ASR is greater than the sum of the two. Moreover, we compare the performance of three types of adversarial examples on various classifiers and datasets under the same iteration steps for targeted attack. On CIFAR-10 [22], we study the performance of compound adversarial examples in decreasing the classification accuracy. We also analyze the attack performance of compound adversarial examples on two certification defensive models and several adversarial trained models. Finally, we study the transferability of the compound adversarial examples on normal trained classifiers and adversarial trained classifiers.
In summary, the main contributions are as follows:
- •
We present a novel framework for the generation process of compound adversarial examples and propose two new algorithms to simultaneously optimize adversarial perturbation and patch for targeted attack and untargeted attack, respectively. We also find that these two weak attack patterns can be combined to perform stronger attack, leading to better generation efficiency. To the best of our knowledge, we are the first to generate compound adversarial examples.
- •
We discover that compound adversarial examples can quickly decrease the classification accuracy of both normal classifiers and defense classifiers. For normal classifiers, compound attack can quickly generate compound adversarial examples with the highest success rate; for defense classifiers, compound attack breaks through the certification methods that are beneficial for patch attack or perturbation attack.
- •
We validate that the compound adversarial examples can perform black-box attack with moderate success rate for normal trained classifiers and adversarial trained classifiers.
- •
We demonstrate the effectiveness and robustness of the proposed method on a set of state-of-the-art classifiers and several datasets.
Section snippets
Related Works
In recent years, a particularly compelling challenge has emerged in the field of computer vision: making very small changes to the input image can deceive the most advanced deep neural networks with high confidence. The adversarial examples have revealed the vulnerability and sensitivity of DNNs, but they can also provide valuable insights for deep models. As our method is based on perturbation attack and patch attack, we first focus on outlining some generation methods of these two attack
Compound Attack
In this section, we introduce our compound attack in details. We first introduce the assumptions and notations to clarify the attack scenarios and the mathematical symbols. Then the untargeted attack and targeted attack algorithms of compound attack are introduced, including constraints, optimization functions and pseudo codes. Finally, the compound attack processes on several defensive models are introduced.
Experiment
In this section, we first clarify the settings in the experiments, including attack models and datasets. Next, we evaluate the white-box attack of compound adversarial example on several widely-used deep neural network classifiers and popular datasets. We conduct both untargeted attack and targeted attack. We then study the performance of compound adversarial examples on several defense models. Finally we study the black-box attack performance.
Conclusions
In this paper, we propose two algorithms to generate compound adversarial examples for both untargeted attack and targeted attack. The proposed algorithm is based on the momentum gradient and the random initialization noise. Compared with perturbation attack and patch attack, the compound attack has a higher attack success rate, better generation efficiency and stronger transferability. We also assess the aggression of compound adversarial examples on models with defensive mechanism, and the
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Yanchun Li obtained the B.S., M.S. degrees from the College of Information Engineering of Xiangtan University, Hunan, China, and obtained the Ph.D. degree from the School of Computer Science and Engineering, South China University of Technology, Guangzhou, China. She is now a senior lecturer in School of computer, Xiangtan University. Her research interests include deep learning, computer vision and image processing. She is a member of Chinese Computer Federation (CCF).
References (50)
- et al.
A face recognition framework based on a pool of techniques and differential evolution
Information Sciences
(2021) - et al.
A kernel correlation-based approach to adaptively acquire local features for learning 3d point clouds
Computer-Aided Design
(2022) - et al.
Adversarial example generation with adaptive gradient search for single and ensemble deep neural network
Information Sciences
(2020) - et al.
A gpu-based residual network for medical image classification in smart medicine
Information Sciences
(2020) - et al.
Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples
- S. Baluja, I. Fischer, Adversarial transformation networks: Learning to generate adversarial examples, 2017. CoRR...
- L. Bottou, Large-Scale Machine Learning with Stochastic Gradient Descent,...
- et al.
Decision-based adversarial attacks: Reliable attacks against black-box machine learning models
- T.B. Brown, D. Mané, A. Roy, M. Abadi, J. Gilmer, Adversarial patch, 2017. ArXiv...
- et al.
Look and think twice: Capturing top-down visual attention with feedback convolutional neural networks
Black-box adversarial attack on license plate recognition system
Acta Automatica Sinica
Zoo: Zeroth order optimization based black-box attacks to deep neural networks without training substitute models
A fast and elitist multiobjective genetic algorithm: Nsga-ii
IEEE Transactions on Evolutionary Computation
Imagenet: A large-scale hierarchical image database
Proc of IEEE Computer Vision and Pattern Recognition
Deep residual learning for image recognition
Densely connected convolutional networks
Convolutional networks with dense connectivity
IEEE Transactions on Pattern Analysis and Machine
LaVAN: Localized and visible adversarial noise, in
Physgan: Generating physical-world-resilient adversarial examples for autonomous driving
Cited by (5)
CGN: Class gradient network for the construction of adversarial samples
2024, Information SciencesAttacking the tracker with a universal and attractive patch as fake target
2023, Information SciencesCrafting transferable adversarial examples via contaminating the salient feature variance
2023, Information SciencesImproving the invisibility of adversarial examples with perceptually adaptive perturbation
2023, Information SciencesEvaluating the Robustness of Deep Learning Models against Adversarial Attacks: An Analysis with FGSM, PGD and CW
2024, Big Data and Cognitive Computing
Yanchun Li obtained the B.S., M.S. degrees from the College of Information Engineering of Xiangtan University, Hunan, China, and obtained the Ph.D. degree from the School of Computer Science and Engineering, South China University of Technology, Guangzhou, China. She is now a senior lecturer in School of computer, Xiangtan University. Her research interests include deep learning, computer vision and image processing. She is a member of Chinese Computer Federation (CCF).
Zhetao Li (Member, IEEE) received the B.S. degree from Xiangtan University, Xiangtan, China, in 2002, the M.S. degree from Beihang University, Beijing, China, in 2005, and the Ph.D. degree in computer application technology from Hunan University, Changsha, China, in 2010. He is currently a professor with the Key Laboratory of Hunan Province for Internet of Things and Information Security and School of Computer Science, Xiangtan University, China. He was a visiting researcher with Ajou University, South Korea from May to August 2012. His research interests include cloud computing, artificial intelligence, and multimedia signal processing.
Li Zeng received a bachelor’s degree in software engineering and a master’s degree in computer science and technology from Xiangtan University in 2013 and 2016, respectively. She has been working at Xiangtan University since she graduated with a master’s degree. Her current research interests include information security, cryptanalysis, and bioinformatics.
Saiqin Long received the PhD degree in computer applications technology from the South China University of Technology, Guangzhou, China, in 2014. She is currently a professor with the College of Information Science and Technology, Jinan University, China. Her research interests include cloud computing, edge computing, parallel and distributed systems, and Internet of things. She has published 20 + refereed papers in these areas, most of which are published in premium conferences and journals, including IEEE TSC, IEEE TPDS, IEEE TMC, etc. She is a member of Chinese Computer Federation (CCF).
Feiran Huang (Member, IEEE) received the B.Sc. degree from Central South University, Changsha, China, in 2011, and the Ph.D. degree in computer software and theory from the School of Computer Science and Engineering, Beihang University, Beijing, China, in 2019. He is currently a Lecturer with the School of Information Science and Technology and College of Cyber Security, Jinan University, Guangzhou, China. His research interests include social media analysis and multimodal learning.
Kui Ren (Fellow, IEEE) is a professor and associate dean of College of Computer Science and Technology at Zhejiang University, where he also directs the Institute of Cyber Science and Technology. Before that, he was with State University of New York at Buffalo. He received his Ph.D. degree in Electrical and Computer Engineering from Worcester Polytechnic Institute. Kui’s current research interests include Data Security, IoT Security, AI Security, and Privacy. Kui is a Fellow of IEEE, a Distinguished Member of ACM and a Clarivate Highly-Cited Researcher. He is a frequent reviewer for funding agencies internationally and serves on the editorial boards of many IEEE and ACM journals. He currently serves as Chair of SIGSAC of ACM China.
This work is supported in part by National Key Research and Development Program of China under Grant 2021YFB3101201, Hunan Provincial Natural Science Foundation under Grant 2021JJ30685, Natural Science Foundation of China under Grant 62032020, Natural Science Foundation of China under Grant 62172349, Hunan Province Department of Education under Grant 21B0120, and Hunan Science and Technology Planning Project under Grant No.2019RS3019.