Abstract
Recent studies show that DNNs are vulnerable to adversarial attacks, in which carefully chosen imperceptible modifications to the inputs lead to incorrect predictions. However most existing attacks focus on closed-set classification, and adversarial attack of open-set recognition has been less investigated. In this paper, we systematically investigate the adversarial robustness of widely used open-set recognition models, namely person re-identification (ReID) and face recognition (FR) models. Specifically, we compare two categories of black-box attacks: transfer-based extensions of standard closed-set attacks and several direct random-search based attacks proposed here. Extensive experiments demonstrate that ReID and FR models are also vulnerable to adversarial attack, and highlight a potential AI trustworthiness problem for these socially important applications.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bai, S., Li, Y., Zhou, Y., Li, Q., Torr, P.H.S.: Metric attack and defense for person re-identification (2019)
Cao, K., Rong, Y., Li, C., Tang, X., Loy, C.C.: Pose-robust face recognition via deep residual equivariant mapping. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5187–5196 (2018)
Carlini, N., Wagner, D.A.: Towards evaluating the robustness of neural networks. In: 2017 IEEE Symposium on Security and Privacy (SP), pp. 39–57 (2017)
Chen, P.Y., Zhang, H., Sharma, Y., Yi, J., Hsieh, C.J.: ZOO: zeroth order optimization based black-box attacks to deep neural networks without training substitute models. In: Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, pp. 15–26 (2017)
Chen, S., Liu, Y., Gao, X., Han, Z.: MobileFaceNets: efficient CNNs for accurate real-time face verification on mobile devices. In: Zhou, J., et al. (eds.) CCBR 2018. LNCS, vol. 10996, pp. 428–438. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-97909-0_46
Cissé, M., Adi, Y., Neverova, N., Keshet, J.: Houdini: fooling deep structured prediction models. CoRR abs/1707.05373 (2017)
Deb, D., Zhang, J., Jain, A.K.: AdvFaces: adversarial face synthesis. CoRR abs/1908.05008 (2019)
Deng, J., Guo, J., Xue, N., Zafeiriou, S.: ArcFace: additive angular margin loss for deep face recognition. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4690–4699 (2019)
Ding, W., Wei, X., Hong, X., Ji, R., Gong, Y.: Universal adversarial perturbations against person re-identification. CoRR abs/1910.14184 (2019)
Gafni, O., Wolf, L., Taigman, Y.: Live face de-identification in video. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 9377–9386 (2019)
Goodfellow, I.J., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial examples. In: 3rd International Conference on Learning Representations (ICLR) (2015)
Goswami, G., Ratha, N.K., Agarwal, A., Singh, R., Vatsa, M.: Unravelling robustness of deep learning based face recognition against adversarial attacks. In: Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, pp. 6829–6836 (2018)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778 (2016)
Huang, G.B., Ramesh, M., Berg, T., Learned-Miller, E.: Labeled faces in the wild: a database for studying face recognition in unconstrained environments. Technical report (2007)
Kurakin, A., Goodfellow, I.J., Bengio, S.: Adversarial examples in the physical world. In: 5th International Conference on Learning Representations (ICLR) (2017)
Li, D., Chen, X., Zhang, Z., Huang, K.: Learning deep context-aware features over body and latent parts for person re-identification. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7398–7407 (2017)
Li, X., Wu, A., Zheng, W.-S.: Adversarial open-world person re-identification. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11206, pp. 287–303. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01216-8_18
Liu, W., Wen, Y., Yu, Z., Li, M., Raj, B., Song, L.: SphereFace: deep hypersphere embedding for face recognition. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6738–6746 (2017)
Liu, Y., Li, H., Wang, X.: Rethinking feature discrimination and polymerization for large-scale recognition. CoRR abs/1710.00870 (2017)
Madry, A., Makelov, A., Schmidt, L., Tsipras, D., Vladu, A.: Towards deep learning models resistant to adversarial attacks. In: 6th International Conference on Learning Representations (ICLR) (2018)
Moosavi-Dezfooli, S., Fawzi, A., Frossard, P.: DeepFool: a simple and accurate method to fool deep neural networks. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2574–2582 (2016)
Papernot, N., McDaniel, P.D., Jha, S., Fredrikson, M., Celik, Z.B., Swami, A.: The limitations of deep learning in adversarial settings. In: IEEE European Symposium on Security and Privacy (EuroS&P), pp. 372–387 (2016)
Sarkar, S., Bansal, A., Mahbub, U., Chellappa, R.: UPSET and ANGRI : breaking high performance image classifiers. CoRR abs/1707.01159 (2017)
Schroff, F., Kalenichenko, D., Philbin, J.: FaceNet: a unified embedding for face recognition and clustering. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 815–823 (2015)
Shen, Y., Xiao, T., Li, H., Yi, S., Wang, X.: End-to-end deep Kronecker-product matching for person re-identification. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6886–6895 (2018)
Si, J., Zhang, H., Li, C., Kuen, J., Kong, X., Kot, A.C., Wang, G.: Dual attention matching network for context-aware feature sequence based person re-identification. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5363–5372 (2018)
Song, C., Huang, Y., Ouyang, W., Wang, L.: Mask-guided contrastive attention model for person re-identification. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1179–1188 (2018)
Su, J., Vargas, D.V., Sakurai, K.: One pixel attack for fooling deep neural networks. IEEE Trans. Evol. Comput. 23(5), 828–841 (2019)
Sun, Y., Wang, X., Tang, X.: Deep learning face representation from predicting 10, 000 classes. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1891–1898 (2014)
Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I.J., Fergus, R.: Intriguing properties of neural networks. In: 2nd International Conference on Learning Representations (ICLR) (2014)
Taigman, Y., Yang, M., Ranzato, M., Wolf, L.: DeepFace: closing the gap to human-level performance in face verification. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1701–1708 (2014)
Tang, D., Wang, X., Zhang, K.: Query-free attacks on industry-grade face recognition systems under resource constraints. CoRR abs/1802.09900 (2018)
Varior, R.R., Haloi, M., Wang, G.: Gated Siamese convolutional neural network architecture for human re-identification. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9912, pp. 791–808. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46484-8_48
Wang, F., Xiang, X., Cheng, J., Yuille, A.L.: Normface: L\({}_{\text{2}}\) hypersphere embedding for face verification. In: Proceedings of the 2017 ACM on Multimedia Conference, pp. 1041–1049 (2017)
Wang, H., Wang, Y., Zhou, Z., Ji, X., Gong, D., Zhou, J., Li, Z., Liu, W.: CosFace: large margin cosine loss for deep face recognition. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5265–5274 (2018)
Wang, Z., Zheng, S., Song, M., Wang, Q., Rahimpour, A., Qi, H.: advPattern: physical-world attacks on deep person re-identification via adversarially transformable patterns. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 8340–8349 (2019)
Wei, L., Zhang, S., Gao, W., Tian, Q.: Person transfer GAN to bridge domain gap for person re-identification. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 79–88 (2018)
Yi, D., Lei, Z., Liao, S., Li, S.Z.: Learning face representation from scratch. CoRR abs/1411.7923 (2014)
Yu, T., Li, D., Yang, Y., Hospedales, T.M., Xiang, T.: Robust person re-identification by modelling feature uncertainty. In: IEEE/CVF International Conference on Computer Vision (ICCV), pp. 552–561 (2019)
Zhang, X., et al.: AlignedReID: surpassing human-level performance in person re-identification. CoRR abs/1711.08184 (2017)
Zhang, Z., Lan, C., Zeng, W., Chen, Z.: Densely semantically aligned person re-identification. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 667–676 (2019)
Zhao, L., Li, X., Zhuang, Y., Wang, J.: Deeply-learned part-aligned representations for person re-identification. In: IEEE International Conference on Computer Vision (ICCV), pp. 3239–3248 (2017)
Zheng, L., Shen, L., Tian, L., Wang, S., Wang, J., Tian, Q.: Scalable person re-identification: a benchmark. In: IEEE International Conference on Computer Vision (ICCV), pp. 1116–1124 (2015)
Zhou, E., Cao, Z., Yin, Q.: Naive-deep face recognition: touching the limit of LFW benchmark or not? CoRR abs/1501.04690 (2015)
Zhou, K., Xiang, T.: Torchreid: a library for deep learning person re-identification in Pytorch. CoRR abs/1910.10093 (2019)
Zhou, K., Yang, Y., Cavallaro, A., Xiang, T.: Omni-scale feature learning for person re-identification. In: IEEE/CVF International Conference on Computer Vision (ICCV), pp. 3701–3711 (2019)
Zhu, Z., Luo, P., Wang, X., Tang, X.: Recover canonical-view faces in the wild with deep neural networks. CoRR abs/1404.3543 (2014)
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Gong, X., Hu, G., Hospedales, T., Yang, Y. (2020). Adversarial Robustness of Open-Set Recognition: Face Recognition and Person Re-identification. In: Bartoli, A., Fusiello, A. (eds) Computer Vision – ECCV 2020 Workshops. ECCV 2020. Lecture Notes in Computer Science(), vol 12535. Springer, Cham. https://doi.org/10.1007/978-3-030-66415-2_9
Download citation
DOI: https://doi.org/10.1007/978-3-030-66415-2_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-66414-5
Online ISBN: 978-3-030-66415-2
eBook Packages: Computer ScienceComputer Science (R0)