Abstract
The data augmentation method has been demonstrated as a ploy for enhancing model accuracy and adversarial robustness. However, it is well known that the traditional data augmentation methods have limited ability to defend against adversarial attacks. The most effective way to defend against adversarial attacks is still adversarial training. In addition, the trade-off between classification accuracy and robustness in adversarial training is also a popular research direction. In this paper, we propose a more effective method to combine one mixup algorithm with adversarial training to further enhance the robustness and accuracy of the model. Specifically, we align images in the feature space before the adversarial training. This method adds the features of another image on the basis of retaining the outline of one image. The images are trained by adversarial attack afterward. To verify the effectiveness of our method, we compare several other adversarial training methods. The experiments show our method achieves significant robustness and accuracy gains. Especially, our method makes an impressive trade-off between robustness and accuracy.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Goodfellow, I.J., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572 (2014)
Zhang, H., Cisse, H., Dauphin, Y.N., Lopez-Paz, D.: mixup: beyond empirical risk minimization. arXiv preprint arXiv:1710.09412 (2017)
DeVries, T., Taylor, G.W.: Improved regularization of convolutional neural networks with cutout. arXiv preprint arXiv:1708.04552 (2017)
Inoue, H.: Data augmentation by pairing samples for images classification. arXiv preprint arXiv:1801.02929 (2018)
Yun, S., et al.: CutMix: regularization strategy to train strong classifiers with localizable features. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 6023–6032 (2019)
Verma, V., et al.: Manifold mixup: better representations by interpolating hidden states. In: International Conference on Machine Learning, pp. 6438–6447. PMLR (2019)
Uddin, A.F.M., et al. SaliencyMix: a saliency guided data augmentation strategy for better regularization. arXiv preprint arXiv:2006.01791 (2020)
Kim, J.-H., Choo, W., Song, H.O.: Puzzle mix: exploiting saliency and local statistics for optimal mixup. In: International Conference on Machine Learning, pp. 5275–5285. PMLR (2020)
Kim, J.-H., Choo, W., Jeong, H., Song, H.O.: Co-mixup: saliency guided joint mixup with supermodular diversity. arXiv preprint arXiv:2102.03065 (2021)
Hong, M., Choi, J., Kim, G.: StyleMix: separating content and style for enhanced data augmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14862–14870 (2021)
Venkataramanan, S., Avrithis, Y., Kijak, E., Amsaleg, L.: AlignMix: improving representation by interpolating aligned features. arXiv preprint arXiv:2103.15375 (2021)
Hendrycks, D., Mu, N., Cubuk, E.D., Zoph, B., Gilmer, J., Lakshminarayanan, B.: AugMix: a simple data processing method to improve robustness and uncertainty. arXiv preprint arXiv:1912.02781 (2019)
Goodfellow, I., et al.: Generative adversarial nets. In: 27th Proceedings Conference on Advances in Neural Information Processing Systems (2014)
Cubuk, E.D., Zoph, B., Mane, D., Vasudevan, V., Le, Q.V.: AutoAugment: learning augmentation policies from data. arXiv preprint arXiv:1805.09501 (2018)
Müller, S.G., Hutter, F.: TrivialAugment: tuning-free yet state-of-the-art data augamentation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 774–782 (2021)
Madry, A.,Makelov, A., Schmidt, L., Tsipras, D., Vladu, A.: Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083 (2017)
Shafahi, A., et al.: Adversarial training for free! In: 32nd Proceedings of the Conference on Advances in Neural Information Processing Systems (2019)
Zhu, C., Cheng, Y., Gan, Z., Sun, S., Goldstein, T., Liu, J.: FreeLB: enhanced adversarial training for natural language understanding. arXiv preprint arXiv:1909.11764 (2019)
Zhang, H., Yu, Y., Jiao, J., Xing, E., El Ghaoui, L., Jordan, M.: Theoretically principled trade-off between robustness and accuracy. In International Conference on Machine Learning, pp. 7472–7482. PMLR (2019)
Wang, Y., Zou, D., Yi, J., Bailey, J., Ma, X., Gu, Q.: Improving adversarial robustness requires revisiting misclassified examples. In: International Conference on Learning Representations (2019)
Shafahi, A., Ghiasi, A., Huang, F., Goldstein, T.: Label smoothing and logit squeezing: a replacement for adversarial training? arXiv preprint arXiv:1910.11585 (2019)
Smith, L.N., Topin, N.: Super-convergence: very fast training of residual networks using large learning rates. In: ICLR (2018)
Micikevicius, P., et al.: Mixed precision training. arXiv preprint arXiv:1710.03740 (2017)
Tramèr, F., Kurakin, A., Papernot, N., Goodfellow, I., Boneh, D., McDaniel P.: Ensemble adversarial training: attacks and defenses. arXiv preprint arXiv:1705.07204 (2017)
Jakubovitz, D., Giryes, R.: Improving DNN robustness to adversarial attacks using jacobian regularization. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11216, pp. 525–541. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01258-8_32
Wong, E., Rice, L., Kolter, J.Z.: Fast is better than free: revisiting adversarial training. arXiv preprint arXiv:2001.03994 (2020)
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)
Krizhevsky, A., et al.: Learning multiple layers of features from tiny images (2009)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Zagoruyko, S., Komodakis, N.: Wide residual networks. arXiv preprint arXiv:1605.07146 (2016)
Acknowledgements
This work is supported by the National Natural Science Foundation of China (No. 62072109, No. U1804263) and Natural Science Foundation of Fujian Province (No. 2021J06013).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Lin, Y., Liu, X., Jiang, N. (2023). AMAD: Improving Adversarial Robustness Without Reducing Accuracy. In: Xu, Y., Yan, H., Teng, H., Cai, J., Li, J. (eds) Machine Learning for Cyber Security. ML4CS 2022. Lecture Notes in Computer Science, vol 13656. Springer, Cham. https://doi.org/10.1007/978-3-031-20099-1_1
Download citation
DOI: https://doi.org/10.1007/978-3-031-20099-1_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-20098-4
Online ISBN: 978-3-031-20099-1
eBook Packages: Computer ScienceComputer Science (R0)