ABSTRACT
Convolutional neural networks(CNN) are vulnerable to adversarial samples, which poses a threat to application in some scenes. This paper proposes a novel loss function to improve the robustness of CNN—CIraCLoss, which combines the intra-class distance loss function (IntraCLoss) and the cross-entropy loss function (CELoss). In the training stage, the IntraCLoss encourages each feature extracted by CNN to be close to its intra-class center. With this feature space distribution, the adversarial sample needs a larger intensity of attack so that its feature keeps away from the intra-class center. Therefore, IntraCLoss can make CNN more robust to defend against adversarial attacks. The results on the CIFAR10 and MNIST datasets show that CIraCLoss, which is mainly affected by IntraCLoss, can reduce the DBI index of the feature space and also reduce the fooling rates of models. In addition, our method can be applied to different network structures and has good generalization.
- He K , Zhang X , Ren S , Deep Residual Learning for Image Recognition[C]// IEEE Conference on Computer Vision & Pattern Recognition. IEEE Computer Society, 2016.Google Scholar
- Goodfellow I J , Shlens J , Szegedy C . Explaining and harnessing adversarial examples[C]// ICML. 2015.Google Scholar
- Xu H , Ma Y , Liu H C , Adversarial Attacks and Defenses in Images, Graphs and Text: A Review[J]. International Journal of Automation and Computing, 2020, 17(2):151-178.Google ScholarCross Ref
- K. Eykholt , "Robust Physical-World Attacks on Deep Learning Visual Classification," 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, 2018, pp. 1625-1634, doi: 10.1109/CVPR.2018.00175.Google ScholarCross Ref
- Ross, A. and Finale Doshi-Velez. “Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing their Input Gradients.” AAAI (2018).Google ScholarCross Ref
- Drucker H , Le Cun Y . Improving generalization performance using double backpropagation[J]. IEEE Transactions on Neural Networks, 1992, 3(6):P.991-997.Google ScholarDigital Library
- Papernot N , Mcdaniel P , Wu X , Distillation as a Defense to Adversarial Perturbations Against Deep Neural Networks[J]. 2015.Google Scholar
- Hinton G , Vinyals O , Dean J . Distilling the Knowledge in a Neural Network[J]. Computer ence, 2015, 14(7):38-39.Google Scholar
- Szegedy C , Zaremba W , Sutskever I , Intriguing properties of neural networks. 2013.Google Scholar
- Moosavi-Dezfooli S M , Fawzi A , Frossard P . DeepFool: a simple and accurate method to fool deep neural networks[C]// Computer Vision & Pattern Recognition. IEEE, 2016.Google Scholar
- Carlini N , Wagner D . Towards Evaluating the Robustness of Neural Networks[J]. 2016.Google Scholar
- Su J , Vargas D V , Kouichi S . One pixel attack for fooling deep neural networks[J]. IEEE Transactions on Evolutionary Computation, 2017.Google Scholar
- Chen P Y , Zhang H , Sharma Y , ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Models[J]. ACM, 2017.Google Scholar
- Ru, Binxin, “BayesOpt Adversarial Attack.” ICLR 2020: Eighth International Conference on Learning Representations, 2020.Google Scholar
- S. Chen, Z. He, C. Sun, J. Yang and X. Huang, "Universal Adversarial Attack on Attention and the Resulting Dataset DAmageNet," in IEEE Transactions on Pattern Analysis and Machine Intelligence, doi: 10.1109/TPAMI.2020.3033291.Google Scholar
- Akhtar N , Mian A . Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey[J]. IEEE Access, 2018, 6:14410-14430.Google ScholarCross Ref
- Tramèr, Florian, Kurakin A , Papernot N , Ensemble Adversarial Training: Attacks and Defenses[J]. 2017.Google Scholar
- Dziugaite G K , Ghahramani Z , Roy D M . A study of the effect of JPG compression on adversarial images[J]. 2016.Google Scholar
- Lyu, C., Huang, K., & Liang, H. (2015). A Unified Gradient Regularization Family for Adversarial Examples. 2015 IEEE International Conference on Data Mining, 301-309.Google ScholarDigital Library
- Xu W , Evans D , Qi Y . Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks[C]// Network and Distributed System Security Symposium. 2018.Google Scholar
- Lee H , Han S , Lee J . Generative Adversarial Trainer: Defense to Adversarial Perturbations with GAN[J]. 2017.Google Scholar
- Liu H , Tian Y , Wang Y , Deep Relative Distance Learning: Tell the Difference between Similar Vehicles[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2016.Google Scholar
- Gu S , Rigazio L . Towards Deep Neural Network Architectures Robust to Adversarial Examples[J]. Computer ence, 2015.Google Scholar
- Bengio Y . Learning Deep Architectures for AI[J]. Foundations & Trends in Machine Learning, 2009, 2(1):1-127.Google ScholarDigital Library
- Shaham U , Yamada Y , Negahban S . Understanding Adversarial Training: Increasing Local Stability of Neural Nets through Robust Optimization[J]. Computer ence, 2015.Google Scholar
- Nguyen L , Wang S , Sinha A . A Learning and Masking Approach to Secure Learning[J]. 2017.Google Scholar
- Narodytska N , Kasiviswanathan S P . Simple Black-Box Adversarial Perturbations for Deep Networks[J]. 2016.Google Scholar
- Wen Y , Zhang K , Li Z , A Discriminative Feature Learning Approach for Deep Face Recognition[C]// European Conference on Computer Vision. Springer, Cham, 2016.Google Scholar
Index Terms
CIraCLoss: Intra-class Distance Loss Makes CNN Robust
Recommendations
A Novel CNN Training Framework: Loss Transferring
As one of the indispensable components in convolutional neural network (CNN), loss function assists in updating parameters of CNN models during the training phase. Generally, different loss functions can assist convolutional neural network (CNN) to learn ...
Face Recognition Using 3D CNN and Hardmining Loss Function
AbstractResearch in face recognition has achieved new heights after the advent of deep learning, particularly 2D Convolution Neural Network (2D CNN). However, to solve the real-world challenges in face recognition, researchers have started using 3D CNN ...
A robust deformed convolutional neural network (CNN) for image denoising
AbstractDue to strong learning ability, convolutional neural networks (CNNs) have been developed in image denoising. However, convolutional operations may change original distributions of noise in corrupted images, which may increase training difficulty ...
Comments