Abstract
Several deep neural networks are vulnerable to adversarial samples that are imperceptible to humans. To address this challenge, a range of techniques have been proposed to design more robust model architectures. However, previous research has primarily focused on identifying atomic structures that are more resilient, while our work focuses on adapting the model in two spatial dimensions: width and depth. In this paper, we present a multi-objective neural architecture search (NAS) method that searches for optimal widths for different layers in spatial dimensions, referred to as DW-Net. We also propose a novel adversarial sample generation technique for one-shot that enhances search space diversity and promotes search efficiency. Our experimental results demonstrate that the proposed optimal neural architecture outperforms state-of-the-art NAS-based networks widely used in the literature in terms of adversarial accuracy, under different adversarial attacks and for different-sized tasks.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Athalye, A., Carlini, N., Wagner, D.: Obfuscated gradients give a false sense of security: circumventing defenses to adversarial examples. In: International Conference on Machine Learning, pp. 274–283. PMLR (2018)
Bai, Y., et al.: Improving adversarial robustness via channel-wise activation suppressing (2021)
Das, N., et al.: Compression to the rescue: Defending from adversarial attacks across modalities. In: ACM SIGKDD Conference on Knowledge Discovery and Data Mining (2018)
Devaguptapu, C., Agarwal, D., Mittal, G., Gopalani, P., Balasubramanian, V.N.: On adversarial robustness: a neural architecture search perspective http://arxiv.org/abs/2007.08428
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: pre-training of deep bidirectional transformers for language understanding (2018)
Eykholt, K., et al.: Robust physical-world attacks on deep learning visual classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1625–1634 (2018)
Goodfellow, I.J., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial examples (2017)
Goodfellow, I.J., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial examples (2014)
Gu, S., Rigazio, L.: Towards deep neural network architectures robust to adversarial examples. arXiv preprint arXiv:1412.5068 (2014)
Guo, M., Yang, Y., Xu, R., Liu, Z., Lin, D.: When NAS meets robustness: in search of robust architectures against adversarial attacks. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 628–637 (2020). https://doi.org/10.1109/CVPR42600.2020.00071, ISSN: 2575-7075
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition (2016). http://arxiv.org/abs/1512.03385
Kurakin, A., Goodfellow, I., Bengio, S.: Adversarial machine learning at scale (2016)
Liao, F., Liang, M., Dong, Y., Pang, T., Hu, X., Zhu, J.: Defense against adversarial attacks using high-level representation guided denoiser. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1778–1787 (2018)
Liu, H., Simonyan, K., Yang, Y.: DARTS: differentiable architecture search (2018). http://arxiv.org/abs/1806.09055
Lu, Z., Whalen, I., Boddeti, V., Dhebar, Y., Deb, K., Goodman, E., Banzhaf, W.: NSGA-net: neural architecture search using multi-objective genetic algorithm. In: Proceedings of the Genetic and Evolutionary Computation Conference, pp. 419–427 (2019)
Madry, A., Makelov, A., Schmidt, L., Tsipras, D., Vladu, A.: Towards deep learning models resistant to adversarial attacks (2017)
Ning, X., Zhao, J., Li, W., Zhao, T., Yang, H., Wang, Y.: Multi-shot NAS for discovering adversarially robust convolutional neural architectures at targeted capacities. arXiv preprint arXiv:2012.11835 (2020)
Papernot, N., McDaniel, P., Wu, X., Jha, S., Swami, A.: Distillation as a defense to adversarial perturbations against deep neural networks. In: 2016 IEEE Symposium on Security and Privacy (SP), pp. 582–597. IEEE (2016)
Pham, H., Guan, M.Y., Zoph, B., Le, Q.V., Dean, J.: Efficient neural architecture search via parameter sharing (2018)
Qian, Y., Huang, S., Wang, Y., Li, S.: RNAS: robust network architecture search beyond DARTS (2021)
Szegedy, C., et al.: Intriguing properties of neural networks (2013)
Tramr, F., Kurakin, A., Papernot, N., Goodfellow, I., Boneh, D., McDaniel, P.: Ensemble adversarial training: attacks and defenses (2017). http://arxiv.org/abs/1705.07204, issue: arXiv:1705.07204
Wang, Y., Deng, X., Pu, S., Huang, Z.: Residual convolutional CTC networks for automatic speech recognition (2017)
Wang, Y., Ma, X., Bailey, J., Yi, J., Zhou, B., Gu, Q.: On the convergence and robustness of adversarial training (2021)
Wang, Y., Ma, X., Bailey, J., Yi, J., Zhou, B., Gu, Q.: On the convergence and robustness of adversarial training (2021). arXiv preprint arXiv:2112.08304
Xie, G., Wang, J., Yu, G., Zheng, F., Jin, Y.: Tiny adversarial mulit-objective oneshot neural architecture search (2023)
Zhang, D., Zhang, T., Lu, Y., Zhu, Z., Dong, B.: You only propagate once: accelerating adversarial training via maximal principle, vol. 32 (2019)
Zhang, H., Yu, Y., Jiao, J., Xing, E., El Ghaoui, L., Jordan, M.: Theoretically principled trade-off between robustness and accuracy. In: International Conference on Machine Learning, pp. 7472–7482. PMLR (2019)
Zhang, J., et al.: Attacks which do not kill training make adversarial learning stronger. In: International Conference on Machine Learning, pp. 11278–11287. PMLR (2020)
Zoph, B., Le, Q.V.: Neural architecture search with reinforcement learning (2016)
Acknowledgment
This work is supported by Chinese Government Key R &D Project - High Security Isolated Execution Technology and Application based on Reconfigurable Technology in Specific Areas of Information Security
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Niu, R., Zhu, Z., Leng, T., Li, C., Liu, Y., Meng, D. (2023). Improving Adversarial Robustness via Channel and Depth Compatibility. In: Yang, X., et al. Advanced Data Mining and Applications. ADMA 2023. Lecture Notes in Computer Science(), vol 14180. Springer, Cham. https://doi.org/10.1007/978-3-031-46677-9_10
Download citation
DOI: https://doi.org/10.1007/978-3-031-46677-9_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-46676-2
Online ISBN: 978-3-031-46677-9
eBook Packages: Computer ScienceComputer Science (R0)