Skip to main content
Springer Nature Link
Log in

Unsupervised Multi-criteria Adversarial Detection in Deep Image Retrieval

  • Conference paper
  • First Online:
Security and Privacy in Communication Networks (SecureComm 2023)

  • 146 Accesses

Abstract

The vulnerability in the algorithm supply chain of deep learning has imposed new challenges to image retrieval systems in the downstream. Among a variety of techniques, deep hashing is gaining popularity. As it inherits the algorithmic backend from deep learning, a handful of attacks are recently proposed to disrupt normal image retrieval. Unfortunately, the defense strategies in softmax classification are not readily available to be applied in the image retrieval domain. In this paper, we propose an efficient and unsupervised scheme to identify unique adversarial behaviors in the hamming space. In particular, we design three criteria from the perspectives of hamming distance, quantization loss and denoising to defend against both untargeted and targeted attacks, which collectively limit the adversarial space. The extensive experiments on four datasets demonstrate 2–23% improvements of detection rates with minimum computational overhead for real-time image queries.

The work was supported by the U.S. National Science Foundation under Grant 2245055.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    The CW attack is initially designed for softmax classification thus only a few logits are optimized. But for deep hashing, all hashing bits need to be optimized which rarely succeed for targeted CW attacks. Thus, we focus on the untargeted CW attack.

References

  1. Alibaba’s pailitao. http://www.pailitao.com. Accessed 03 Feb 2022

  2. Bing image search. https://www.bing.com. Accessed 03 Feb 2022

  3. Google image search. https://www.google.com/imghp. Accessed 03 Feb 2022

  4. Pinterest visual search tool. https://www.pinterest.com. Accessed 03 Feb 2022

  5. Athalye, A., Carlini, N., Wagner, D.: Obfuscated gradients give a false sense of security: circumventing defenses to adversarial examples. In: International Conference on Machine Learning, pp. 274–283. PMLR (2018)

    Google Scholar 

  6. Bai, J., et al.: Targeted attack for deep hashing based retrieval. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12346, pp. 618–634. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58452-8_36

    Chapter  Google Scholar 

  7. Cao, Y., Long, M., Liu, B., Wang, J.: Deep Cauchy hashing for hamming space retrieval. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1229–1237 (2018)

    Google Scholar 

  8. Cao, Z., Long, M., Wang, J., Yu, P.S.: HashNet: deep learning to hash by continuation. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 5608–5617 (2017)

    Google Scholar 

  9. Carlini, N., Wagner, D.: Defensive distillation is not robust to adversarial examples. arXiv preprint arXiv:1607.04311 (2016)

  10. Carlini, N., Wagner, D.: Towards evaluating the robustness of neural networks. In: 2017 IEEE Symposium on Security and Privacy (SP), pp. 39–57. IEEE (2017)

    Google Scholar 

  11. Carrara, F., Becarelli, R., Caldelli, R., Falchi, F., Amato, G.: Adversarial examples detection in features distance spaces. In: Leal-Taixé, L., Roth, S. (eds.) ECCV 2018. LNCS, vol. 11130, pp. 313–327. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-11012-3_26

    Chapter  Google Scholar 

  12. Chen, M., Lu, J., Wang, Y., Qin, J., Wang, W.: DAIR: a query-efficient decision-based attack on image retrieval systems. In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1064–1073 (2021)

    Google Scholar 

  13. Croce, F., Hein, M.: Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In: International Conference on Machine Learning, pp. 2206–2216. PMLR (2020)

    Google Scholar 

  14. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2005), vol. 1, pp. 886–893. IEEE (2005)

    Google Scholar 

  15. Gong, Z., Wang, W., Ku, W.-S.: Adversarial and clean data are not twins. arXiv preprint arXiv:1704.04960 (2017)

  16. Goodfellow, I.J., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572 (2014)

  17. Grosse, K., Manoharan, P., Papernot, N., Backes, M., McDaniel, P.: On the (statistical) detection of adversarial examples. arXiv preprint arXiv:1702.06280 (2017)

  18. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

    Google Scholar 

  19. Hendrycks, D., Gimpel, K.: A baseline for detecting misclassified and out-of-distribution examples in neural networks. arXiv preprint arXiv:1610.02136 (2016)

  20. Hoe, J.T., Ng, K.W., Zhang, T., Chan, C.S., Song, Y.-Z., Xiang, T.: One loss for all: deep hashing with a single cosine similarity based learning objective. In: Advances in Neural Information Processing Systems, vol. 34 (2021)

    Google Scholar 

  21. Hu, S., Yu, T., Guo, C., Chao, W.-L., Weinberger, K.Q.: A new defense against adversarial images: turning a weakness into a strength. In: Advances in Neural Information Processing Systems, vol. 32 (2019)

    Google Scholar 

  22. Kurakin, A., Goodfellow, I.J., Bengio, S.: Adversarial examples in the physical world. arXiv preprint arXiv:1607.02533 (2016)

  23. Li, X., Li, F.: Adversarial examples detection in deep networks with convolutional filter statistics. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 5764–5772 (2017)

    Google Scholar 

  24. Li, X., et al.: QAIR: practical query-efficient black-box attacks for image retrieval. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3330–3339 (2021)

    Google Scholar 

  25. Liang, B., Li, H., Miaoqiang, S., Li, X., Shi, W., Wang, X.: Detecting adversarial image examples in deep neural networks with adaptive noise reduction. IEEE Trans. Dependable Secure Comput. 18(1), 72–85 (2018)

    Article  Google Scholar 

  26. Lin, K., Yang, H.-F., Hsiao, J.-H., Chen, C.-S.: Deep learning of binary hash codes for fast image retrieval. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 27–35 (2015)

    Google Scholar 

  27. Liu, H., Wang, R., Shan, S., Chen, X.: Deep supervised hashing for fast image retrieval. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2064–2072 (2016)

    Google Scholar 

  28. Ma, X., et al.: Characterizing adversarial subspaces using local intrinsic dimensionality. arXiv preprint arXiv:1801.02613 (2018)

  29. Madry, A., Makelov, A., Schmidt, L., Tsipras, D., Vladu, A.: Towards deep learning models resistant to adversarial attacks. In: International Conference on Learning Representations (2018)

    Google Scholar 

  30. Meng, D., Chen, H.: Magnet: a two-pronged defense against adversarial examples. In: Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, pp. 135–147 (2017)

    Google Scholar 

  31. Oliva, A., Torralba, A.: Modeling the shape of the scene: a holistic representation of the spatial envelope. Int. J. Comput. Vis. 42(3), 145–175 (2001)

    Article  Google Scholar 

  32. Papernot, N., McDaniel, P., Goodfellow, I., Jha, S., Celik, Z.B., Swami, A.: Practical black-box attacks against machine learning. In: Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security, pp. 506–519 (2017)

    Google Scholar 

  33. Papernot, N., McDaniel, P., Wu, X., Jha, S., Swami, A.: Distillation as a defense to adversarial perturbations against deep neural networks. In: 2016 IEEE Symposium on Security and Privacy (SP), pp. 582–597. IEEE (2016)

    Google Scholar 

  34. Ross, S.M.: A First Course in Probability, vol. 6. Prentice Hall, Upper Saddle River (2002)

    Google Scholar 

  35. Shafahi, A., et al.: Adversarial training for free! arXiv preprint arXiv:1904.12843 (2019)

  36. Szegedy, C., et al.: Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199 (2013)

  37. Tieleman, T., Hinton, G., et al.: Lecture 6.5-rmsprop: divide the gradient by a running average of its recent magnitude. COURSERA: Neural Netw. Mach. Learn. 4(2), 26–31 (2012)

    Google Scholar 

  38. Tramèr, F., Papernot, N., Goodfellow, I., Boneh, D., McDaniel, P.: The space of transferable adversarial examples. arXiv preprint arXiv:1704.03453 (2017)

  39. Wang, H., Wu, X., Huang, Z., Xing, E.P.: High-frequency component helps explain the generalization of convolutional neural networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8684–8694 (2020)

    Google Scholar 

  40. Wang, X., Zhang, Z., Lu, G., Xu, Y.: Targeted attack and defense for deep hashing. In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 2298–2302 (2021)

    Google Scholar 

  41. Wong, E., Rice, L., Zico Kolter, J.: Fast is better than free: Revisiting adversarial training. arXiv preprint arXiv:2001.03994 (2020)

  42. Xiao, Y., Wang, C.: You see what I want you to see: exploring targeted black-box transferability attack for hash-based image retrieval systems. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1934–1943 (2021)

    Google Scholar 

  43. Xiao, Y., Wang, C., Gao, X.: Evade deep image retrieval by stashing private images in the hash space. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9651–9660 (2020)

    Google Scholar 

  44. Xu, W., Evans, D., Qi, Y.: Feature squeezing: detecting adversarial examples in deep neural networks. arXiv preprint arXiv:1704.01155 (2017)

  45. Yang, E., Liu, T., Deng, C., Tao, D.: Adversarial examples for hamming space search. IEEE Trans. Cybern. 50, 1473–1484 (2018)

    Article  Google Scholar 

  46. Yuan, L., et al.: Central similarity quantization for efficient image and video retrieval. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3083–3092 (2020)

    Google Scholar 

  47. Zhang, H., Yu, Y., Jiao, J., Xing, E., El Ghaoui, L., Jordan, M.: Theoretically principled trade-off between robustness and accuracy. In: International Conference on Machine Learning, pp. 7472–7482. PMLR (2019)

    Google Scholar 

  48. Zhu, H., Long, M., Wang, J., Cao, Y.: Deep hashing network for efficient similarity retrieval. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 30 (2016)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Old Dominion University, Norfolk, VA, USA

    Yanru Xiao

  2. Zhejiang University, Hangzhou, China

    Cong Wang

  3. University of Delaware, Newark, DE, USA

    Xing Gao

Authors
  1. Yanru Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Cong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xing Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Cong Wang .

Editor information

Editors and Affiliations

  1. Tsinghua University, Beijing, China

    Haixin Duan

  2. Concordia University, Montreal, QC, Canada

    Mourad Debbabi

  3. Hong Kong Polytechnic University, Kowloon, Hong Kong

    Xavier de Carné de Carnavalet

  4. Hong Kong Polytechnic University, Kowloon, Hong Kong

    Xiapu Luo

  5. Stevens Institute of Technology, Hoboken, USA

    Xiaojiang Du

  6. Hong Kong Polytechnic University, Kowloon, Hong Kong

    Man Ho Allen Au

Rights and permissions

Reprints and permissions

Copyright information

© 2025 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Xiao, Y., Wang, C., Gao, X. (2025). Unsupervised Multi-criteria Adversarial Detection in Deep Image Retrieval. In: Duan, H., Debbabi, M., de Carné de Carnavalet, X., Luo, X., Du, X., Au, M.H.A. (eds) Security and Privacy in Communication Networks. SecureComm 2023. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 567. Springer, Cham. https://doi.org/10.1007/978-3-031-64948-6_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-64948-6_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-64947-9

  • Online ISBN: 978-3-031-64948-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics