Abstract
Deep cross-media computing faces adversarial example attacks, adversarial training is an effective approach to enhance the robustness of machine learning models via adding adversarial examples into the training phase. However, existing adversarial training methods increase the advantage of membership inference attacks, which aim to determine from the model whether an example is in the training dataset. In this paper, we propose an adversarial training framework that guarantees both robustness and membership privacy by introducing a tailor-made example, called reverse-symmetry example. Moreover, our framework reduces the number of required adversarial examples compared with existing adversarial training methods. We implement the framework based on three adversarial training methods on FMNIST and CIFAR10. The experimental results show that our framework outperforms the original adversarial training with respect to the overall performance of accuracy, robustness, privacy, and runtime.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Andor, D., et al.: Globally normalized transition-based neural networks. In: ACL (2016). https://doi.org/10.18653/v1/p16-1231
Carlini, N., Liu, C., Erlingsson, Ú., Kos, J., Song, D.: The secret sharer: evaluating and testing unintended memorization in neural networks. In: USENIX Security Symposium (2019)
Carlini, N., Wagner, D.A.: Towards evaluating the robustness of neural networks. In: S &P (2017). https://doi.org/10.1109/SP.2017.49
Deng, L., Hinton, G.E., Kingsbury, B.: New types of deep neural network learning for speech recognition and related applications: an overview. In: ICASSP (2013). https://doi.org/10.1109/ICASSP.2013.6639344
Gowal, S., et al.: Scalable verified training for provably robust image classification. In: ICCV (2019). https://doi.org/10.1109/ICCV.2019.00494
Krizhevsky, A., Hinton, G.: Learning multiple layers of features from tiny images. Master’s thesis, University of Toronto (2009)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: NIPS (2012)
Kurakin, A., Goodfellow, I.J., Bengio, S.: Adversarial machine learning at scale. CoRR (2016). https://arxiv.org/abs/1611.01236
Leino, K., Fredrikson, M.: Stolen memories: leveraging model memorization for calibrated white-box membership inference. In: USENIX Security Symposium (2020). https://www.usenix.org/conference/usenixsecurity20/presentation/leino
Madry, A., Makelov, A., Schmidt, L., Tsipras, D., Vladu, A.: Towards deep learning models resistant to adversarial attacks. In: ICLR (2018)
Mirman, M., Gehr, T., Vechev, M.T.: Differentiable abstract interpretation for provably robust neural networks. In: ICML (2018)
Papernot, N., McDaniel, P.D., Jha, S., Fredrikson, M., Celik, Z.B., Swami, A.: The limitations of deep learning in adversarial settings. In: EuroS &P (2016). https://doi.org/10.1109/EuroSP.2016.36
Pearson, K.: LIII. on lines and planes of closest fit to systems of points in space. London Edinburgh Dublin Philos. Mag. J. Sci. (1901). https://doi.org/10.1080/14786440109462720
Salem, A., Zhang, Y., Humbert, M., Berrang, P., Fritz, M., Backes, M.: ML-leaks: model and data independent membership inference attacks and defenses on machine learning models. In: NDSS (2019)
Sharif, M., Bhagavatula, S., Bauer, L., Reiter, M.K.: Accessorize to a crime: real and stealthy attacks on state-of-the-art face recognition. In: CCS (2016). https://doi.org/10.1145/2976749.2978392
Shokri, R., Stronati, M., Song, C., Shmatikov, V.: Membership inference attacks against machine learning models. In: S &P (2017). https://doi.org/10.1109/sp.2017.41
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: ICLR (2015). https://arxiv.org/abs/1409.1556
Sinha, A., Namkoong, H., Duchi, J.C.: Certifying some distributional robustness with principled adversarial training. In: ICLR (2018)
Song, L., Mittal, P.: Systematic evaluation of privacy risks of machine learning models. In: USENIX Security Symposium (2021)
Song, L., Shokri, R., Mittal, P.: Privacy risks of securing machine learning models against adversarial examples. In: CCS (2019). https://doi.org/10.1145/3319535.3354211
Xiao, H., Rasul, K., Vollgraf, R.: Fashion-MNIST: a novel image dataset for benchmarking machine learning algorithms. CoRR (2017). https://arxiv.org/abs/1708.07747
Yeom, S., Giacomelli, I., Fredrikson, M., Jha, S.: Privacy risk in machine learning: analyzing the connection to overfitting. In: IEEE CSF (2018). https://doi.org/10.1109/CSF.2018.00027
Zhang, H., Yu, Y., Jiao, J., Xing, E.P., Ghaoui, L.E., Jordan, M.I.: Theoretically principled trade-off between robustness and accuracy. In: ICML (2019). https://proceedings.mlr.press/v97/zhang19p.html
Acknowledgements
This research was supported in part by the National Key R &D Program of China under grant No. 2022YFB3102100, the National Natural Science Foundation of China under grants No. 62076187, 62172303, the Key R &D Program of Hubei Province under grant No. 2022BAA039, and Key R &D Program of Shandong Province under grant No. 2022CXPT055.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Yan, R., Du, R., He, K., Chen, J. (2024). Efficient Adversarial Training with Membership Inference Resistance. In: Liu, Q., et al. Pattern Recognition and Computer Vision. PRCV 2023. Lecture Notes in Computer Science, vol 14425. Springer, Singapore. https://doi.org/10.1007/978-981-99-8429-9_38
Download citation
DOI: https://doi.org/10.1007/978-981-99-8429-9_38
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-8428-2
Online ISBN: 978-981-99-8429-9
eBook Packages: Computer ScienceComputer Science (R0)