Abstract
Deep neural networks (DNNs) are deemed vulnerable to adversarial examples (AEs). Transfer-based attacks enable attackers to craft adversarial images based on local surrogate models without feedback from remote ones. One of the promising attacks is to distract the attention map of the surrogate model that is likely to be shared among remote models. However, we find that the attention maps calculated from a local model are usually over-focus on the most critical area, which limits the transferability of the attacks. In response to this challenge, we propose an enhanced image transformation method (EIT), which guides adversarial perturbations to distract not only the most critical area but also other relevant regions. The proposed EIT effectively mitigates the differences in attention maps between multiple models and better neutralizes model-specific features, thereby avoiding getting stuck in local optima specific to the surrogate model. Experiments confirm the superiority of our approach to the state-of-the-art benchmarks. Our implementation is available at: github.com/britney-code/EIT-attack.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Goodfellow, I.J., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial examples. In: International Conference on Learning Representations (ICLR) (2015)
Madry, A., Makelov, A., Schmidt, L., et al.: Towards deep learning models resistant to adversarial attacks. arXiv:1706.06083 (2017)
Carlini, N., Wagner, D.: Towards evaluating the robustness of neural networks. In: IEEE Symposium on Security and Privacy (SP), pp. 39–57 (2017)
Huang, Y., Kong, A.W.: Transferable adversarial attack based on integrated gradients. In: International Conference on Learning Representations (ICLR) (2022)
Chen, S., He, Z., Sun, C., et al.: Universal adversarial attack on attention and the resulting dataset DAmageNet. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) (2020)
Xie, C., Zhang, Z., Zhou, Y., et al.: Improving transferability of adversarial examples with input diversity. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2730–2739 (2019)
Dong, Y., Pang, T., Su, H., et al.: Evading defenses to transferable adversarial examples by translation-invariant attacks. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4312–4321 (2019)
Wang, X., He, K.: Enhancing the transferability of adversarial attacks through variance tuning. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1924–1933 (2021)
Kurakin, A., Goodfellow, I., Bengio, S.: Adversarial examples in the physical world. In: International Conference on Learning Representations (ICLR) (2017)
Dong, Y., Liao, F., Pang, T., et al.: Boosting adversarial attacks with momentum. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9185–9193 (2018)
Papernot, N., McDaniel, P., Jha, S., et al.: The limitations of deep learning in adversarial settings. In: IEEE European Symposium on Security and Privacy, pp. 372–387 (2016)
Wu, W., Su, Y., Chen, X., et al.: Boosting the transferability of adversarial samples via attention. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1161–1170 (2020)
Zhou, W., Hou, X., Chen, Y., et al.: Transferable Adversarial Perturbations. In: European Conference on Computer Vision (ECCV), pp. 471–486 (2018)
Huang, Q., Katsman, I., Gu, Z., et al.: Enhancing adversarial example transferability with an intermediate level attack. In: IEEE/CVF International Conference on Computer Vision (ICCV), pp. 4732–4741 (2019)
Iwana, B.K., Kuroki, R., Uchida, S.: Explaining convolutional neural networks using softmax gradient layer-wise relevance propagation. arXiv:1908.04351 (2019)
Sundararajan, M., Taly, A., Yan, Q.: Axiomatic attribution for deep networks. In: International Conference on Machine Learning (ICML), pp. 3319–3328 (2017)
Lin, J., Song, C., He, K., et al.: Nesterov accelerated gradient and scale invariance for adversarial attacks. In: International Conference on Learning Representations (ICLR) (2020)
Wang, Z., Guo, H., Zhang, Z., Liu, W., Qin, Z., Ren, K.: Feature importance-aware transferable adversarial attacks. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 7639–7648 (2021)
Ge, Z., Shang, F., Liu, H., Liu, Y., Wang, X.: Boosting adversarial transferability by achieving flat local maxima. arXiv:2306.05225 (2023)
Long, Y., Zhang, Q., Zeng, B., et al.: Frequency domain model augmentation for adversarial attack. In: European Conference on Computer Vision (ECCV), pp. 549–566 (2022)
Selvaraju, R.R., Cogswell, M., Das, A., et al.: Grad-CAM: Visual explanations from deep networks via gradient-based localization. In: IEEE International Conference on Computer Vision (ICCV) (2017)
Singh, K.K., Lee, Y.J.: Hide-and-seek: Forcing a network to be meticulous for weakly-supervised object and action localization. In: IEEE International Conference on Computer Vision (ICCV) (2017)
Smilkov, D., Thorat, N., Kim, B., et al.: SmoothGrad: removing noise by adding noise. arXiv:1706.03825 (2017)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: International Conference on Learning Representations (ICLR) (2015)
Szegedy, C., Vanhoucke, V., Ioffe, S., et al.: Rethinking the inception architecture for computer vision. In: IEEE International Conference on Computer Vision (ICCV) (2016)
He, K., Zhang, X., Ren, S., et al.: Deep residual learning for image recognition. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778, (2016)
Huang, G., Liu, Z., Van Der Maaten, L.V., et al.: Densely connected convolutional networks. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4700–4708 (2017)
Szegedy, C., Ioffe, S., Vanhoucke, V., et al.: Inception-v4, Inception-ResNet and the impact of residual connections on learning. In: AAAI Conference on Artificial Intelligence (2017)
Tramèr, F., Kurakin, A., Papernot, N., et al.: Ensemble adversarial training: Attacks and defenses. In: International Conference on Learning Representations (ICLR) (2017)
Heo, B., Yun, S., Han, D., et al.: Rethinking spatial dimensions of vision transformers. In: IEEE/CVF International Conference on Computer Vision (ICCV) (2021)
Touvron, H., Cord, M., Sablayrolles, A., et al.: Going deeper with image transformers. arXiv:2103.17239 (2021)
Touvron, H., Cord, M., Douze, M., et al.: Training data-efficient image transformers & distillation through attention. In: International Conference on Machine Learning (ICML) (2021)
Guo, C., Rana, M., Cisse, M., et al.: Countering adversarial images using input transformations. In: International Conference on Learning Representations (ICLR) (2018)
Xu, W., Evans, D., Qi, Y.: Feature squeezing: Detecting adversarial examples in deep neural networks. In: Network and Distributed System Security Symposium (NDSS) (2018)
Liu, Z., Liu, Q., Liu, T., et al.: Feature distillation: DNN-oriented JPEG compression against adversarial examples. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 860–868 (2019)
Naseer, M., Khan, S., Hayat, M., et al.: A self-supervised approach for adversarial robustness. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 262–271 (2020)
Ge, Z., Shang, F., Liu, H., et al.: Improving the transferability of adversarial examples with arbitrary style transfer. In: Proceedings of the ACM International Conference on Multimedia (2023)
Ilyas, A., Santurkar, S., Engstrom, L., et al.: Adversarial examples are not bugs, they are features. In: Annual Conference on Neural Information Processing Systems (2019)
Zhang, Y., Tan, Y.A., Chen, T., et al.: Enhancing the transferability of adversarial examples with random patch. In: Proceedings of the 31th International Joint Conference on Artificial Intelligence (IJCAI), pp. 1672–1678 (2022)
Liu, Y., Chen, X., Liu, C., et al.: Delving into transferable adversarial examples and black-box attacks. In: International Conference on Learning Representations (ICLR) (2017)
Acknowledgements
This work was supported by the Opening Project of Guangdong Province Key Laboratory of Information Security Technology (no. 2023B1212060026) and the network emergency management research special topic (no. WLYJGL2023ZD003).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Wan, Q., Chen, B., Peng, A., Zeng, H. (2025). A Whale Falls, All Thrive: Mitigating Attention Gap to Improve Adversarial Transferability. In: Antonacopoulos, A., Chaudhuri, S., Chellappa, R., Liu, CL., Bhattacharya, S., Pal, U. (eds) Pattern Recognition. ICPR 2024. Lecture Notes in Computer Science, vol 15322. Springer, Cham. https://doi.org/10.1007/978-3-031-78312-8_23
Download citation
DOI: https://doi.org/10.1007/978-3-031-78312-8_23
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-78311-1
Online ISBN: 978-3-031-78312-8
eBook Packages: Computer ScienceComputer Science (R0)
