Menghua Cao
ABSTRACT
Adversarial training is one of the most promising methods to improve the model's robustness, while the expensive training cost keeps a huge problem for this method. Recent researchers have made great effort to improve its performance by reducing the inner adversarial sample construction cost. Their works have alleviated this problem to some extent while the overall performance is still expensive and not interpretable. In this work, we propose AAT (Adaptive Adversarial Training) algorithm utilizing the inherent relationship between the model's robustness and the effects of the adversarial samples to accelerate the overall performance. Our method offers more interpretable robustness improvement while achieving higher efficiency than the state-of-the-art works on standard datasets. We have reduced more than 56% training time than traditional adversarial training on CIFAR10.
- Goodfellow, I.J., Shlens, J. and Szegedy, C., 2015. Explaining and harnessing adversarial examples. In 3rd International Conference on Learning Representations.Google Scholar
- Biggio, B., Corona, I., Maiorca, D., Nelson, B., Šrndić, N., Laskov, P., Giacinto, G. and Roli, F., 2013, September. Evasion attacks against machine learning at test time. In Joint European conference on machine learning and knowledge discovery in databases (pp. 387-402). Springer, Berlin, Heidelberg.Google Scholar
- Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I. and Fergus, R., 2014. Intriguing properties of neural networks. In 2nd International Conference on Learning Representations.Google Scholar
- Carlini, N. and Wagner, D., 2017, May. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp) (pp. 39-57). IEEE.Google Scholar
- Kurakin, A., Goodfellow, I. and Bengio, S., 2017. Adversarial examples in the physical world. In 5th International Conference on Learning Representations.Google Scholar
- Deng, Y., Zheng, X., Zhang, T., Chen, C., Lou, G. and Kim, M., 2020, March. An analysis of adversarial attacks and defenses on autonomous driving models. In 2020 IEEE International Conference on Pervasive Computing and Communications (PerCom) (pp. 1-10). IEEE.Google Scholar
- Bojarski, Mariusz, "End to end learning for self-driving cars." arXiv preprint arXiv:1604.07316 (2016).Google Scholar
- Dong, Y., Su, H., Wu, B., Li, Z., Liu, W., Zhang, T. and Zhu, J., 2019. Efficient decision-based black-box adversarial attacks on face recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 7714-7722).Google Scholar
- Silva, S. H., Rad, P., Beebe, N., Choo, K. K. R., & Umapathy, M. (2019). Cooperative unmanned aerial vehicles with privacy preserving deep vision for real-time object identification and tracking. Journal of Parallel and Distributed Computing, 131, 147-160.Google ScholarDigital Library
- Gu, S., Holly, E., Lillicrap, T., & Levine, S. (2017, May). Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates. In 2017 IEEE international conference on robotics and automation (ICRA) (pp. 3389-3396). IEEE.Google ScholarDigital Library
- Madry, A., Makelov, A., Schmidt, L., Tsipras, D. and Vladu, A., 2018. Towards deep learning models resistant to adversarial attacks. In 6th International Conference on Learning Representations.Google Scholar
- Tramèr, F., Kurakin, A., Papernot, N., Goodfellow, I., Boneh, D. and McDaniel, P., 2018. Ensemble adversarial training: Attacks and defenses. In 6th International Conference on Learning Representations.Google Scholar
- Rice, L., Wong, E. and Kolter, Z., 2020, November. Overfitting in adversarially robust deep learning. In International Conference on Machine Learning (pp. 8093-8104). PMLR.Google Scholar
- Zhang, H., Yu, Y., Jiao, J., Xing, E., El Ghaoui, L. and Jordan, M., 2019, May. Theoretically principled trade-off between robustness and accuracy. In International Conference on Machine Learning (pp. 7472-7482). PMLR.Google Scholar
- Bai, T., Luo, J., Zhao, J., Wen, B. and Wang, Q., 2021. Recent Advances in Adversarial Training for Adversarial Robustness. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence.Google Scholar
- Zhang, J., Xu, X., Han, B., Niu, G., Cui, L., Sugiyama, M. and Kankanhalli, M., 2020, November. Attacks which do not kill training make adversarial learning stronger. In International Conference on Machine Learning (pp. 11278-11287). PMLR.Google Scholar
- Ye, N., Li, Q., Zhou, X.Y. and Zhu, Z., 2021. An Annealing Mechanism for Adversarial Training Acceleration. IEEE Transactions on Neural Networks and Learning SystemsGoogle Scholar
- Shamir, A., Safran, I., Ronen, E. and Dunkelman, O., 2019. A simple explanation for the existence of adversarial examples with small hamming distance. arXiv preprint arXiv:1901.10861.Google Scholar
- Mustafa, A., Khan, S., Hayat, M., Goecke, R., Shen, J. and Shao, L., 2019. Adversarial defense by restricting the hidden space of deep neural networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 3385-3394).Google Scholar
- Wong, E., Rice, L. and Kolter, J.Z., 2020. Fast is better than free: Revisiting adversarial training. In 8th International Conference on Learning Representations.Google Scholar
- He, K., Zhang, X., Ren, S. and Sun, J., 2016, October. Identity mappings in deep residual networks. In European conference on computer vision (pp. 630-645). Springer, Cham.Google Scholar
Recommendations
GAN-Based Fusion Adversarial Training
Knowledge Science, Engineering and ManagementAbstractIn the field of artificial intelligence security, adversarial machine learning has made breakthroughs. However, it is still vulnerable to attacks under a wide variety of adversarial samples, and adversarial training is a very effective method ...
Nrat: towards adversarial training with inherent label noise
AbstractAdversarial training (AT) has been widely recognized as the most effective defense approach against adversarial attacks on deep neural networks and it is formulated as a min-max optimization. Most AT algorithms are geared towards research-oriented ...
A hybrid adversarial training for deep learning model and denoising network resistant to adversarial examples
AbstractDeep neural networks (DNNs) are vulnerable to adversarial attacks that generate adversarial examples by adding small perturbations to the clean images. To combat adversarial attacks, the two main defense methods used are denoising and adversarial ...
Comments