Abstract
Adversarial machine learning, i.e., increasing the robustness of machine learning algorithms against so-called adversarial examples, is now an established field. Yet, newly proposed methods are evaluated and compared under unrealistic scenarios where costs for adversary and defender are not considered and either all samples or no samples are adversarially perturbed. We scrutinize these assumptions and propose the advanced adversarial classification game, which incorporates all relevant parameters of an adversary and a defender. Especially, we take into account economic factors on both sides and the fact that all so far proposed countermeasures against adversarial examples reduce accuracy on benign samples. Analyzing the scenario in detail, where both players have two pure strategies, we identify all best responses and conclude that in practical settings, the most influential factor might be the maximum amount of adversarial examples.
All authors are supported by the Austrian Science Fund (FWF) and the Czech Science Foundation (GACR) under grant no. I 4057-N31 (“Game Over Eva(sion)”). Tomas Pevny was additionally supported by Czech Ministry of Education 19-29680L and by the OP VVV project CZ.02.1.01/0.0/0.0/16_019/0000765 “Research Center for Informatics”.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
More than 3500 papers can be found at: https://preview.tinyurl.com/yxenrc4k.
- 2.
Note that this is already a simplification, since in practice none of the parties knows how many samples the adversary can influence.
- 3.
\(\varDelta ^d = \left\{ v\in [0,1]^d \ :v_1 + \dots + v_d = 1 \right\} \) is the \(d-1\) dimensional probability simplex.
- 4.
An alternative formulation of the linear equation for the dotted line in Fig. 3 is: \(\varDelta \mathrm {acc}= \frac{r_{\max }\varDelta \mathrm {rob}}{1-r_{\max }}\) (for \(\varDelta \mu ^\mathrm {def}=0)\).
- 5.
Note that \(\varDelta \mathrm {acc}+\varDelta \mathrm {rob}=\mathrm {acc}_1 - (\mathrm {acc}_2 - \mathrm {rob}_2) - \mathrm {rob}_1 < \mathrm {acc}_1 \le 1\), by Eq. (12).
References
Biggio, B., Roli, F.: Wild patterns: ten years after the rise of adversarial machine learning. Pattern Recogn. 84, 317–331 (2018)
Brendel, W., Rauber, J., Bethge, M.: Decision-based adversarial attacks: reliable attacks against black-box machine learning models. arXiv preprint arXiv:1712.04248 (2017)
Brückner, M., Scheffer, T.: Stackelberg games for adversarial prediction problems. In: Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 547–555 (2011)
Cohen, J., Rosenfeld, E., Kolter, Z.: Certified adversarial robustness via randomized smoothing. In: International Conference on Machine Learning, pp. 1310–1320. PMLR (2019)
Dalvi, N., Domingos, P., Sanghai, S., Verma, D.: Adversarial classification. In: Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 99–108 (2004)
Dritsoula, L., Loiseau, P., Musacchio, J.: A game-theoretic analysis of adversarial classification. IEEE Trans. Inf. Forensics Secur. 12(12), 3094–3109 (2017)
Gilmer, J., Adams, R.P., Goodfellow, I., Andersen, D., Dahl, G.E.: Motivating the rules of the game for adversarial example research. arxiv (2018). https://arxiv.org/abs/1807.06732
Globerson, A., Roweis, S.: Nightmare at test time: robust learning by feature deletion. In: Proceedings of the 23rd International Conference on Machine Learning, pp. 353–360 (2006)
Grosse, K., Papernot, N., Manoharan, P., Backes, M., McDaniel, P.: Adversarial examples for malware detection. In: Foley, S.N., Gollmann, D., Snekkenes, E. (eds.) ESORICS 2017. LNCS, vol. 10493, pp. 62–79. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66399-9_4
Großhans, M., Scheffer, T.: Solving prediction games with parallel batch gradient descent. In: Appice, A., Rodrigues, P.P., Santos Costa, V., Soares, C., Gama, J., Jorge, A. (eds.) ECML PKDD 2015. LNCS (LNAI), vol. 9284, pp. 152–167. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-23528-8_10
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)
Leyton-Brown, K., Shoham, Y.: Essentials of game theory: a concise multidisciplinary introduction. Synth. Lect. Artif. Intell. Mach. Learn. 2(1), 1–88 (2008)
Madry, A., Makelov, A., Schmidt, L., Tsipras, D., Vladu, A.: Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083 (2017)
Nash, J.: Non-cooperative games. Ann. Math. 54, 286–295 (1951)
Papernot, N., McDaniel, P., Goodfellow, I.: Transferability in machine learning: from phenomena to black-box attacks using adversarial samples. arXiv preprint arXiv:1605.07277 (2016)
Papernot, N., McDaniel, P., Goodfellow, I., Jha, S., Celik, Z.B., Swami, A.: Practical black-box attacks against machine learning. In: Proceedings of the 2017 ACM on Asia conference on computer and communications security, pp. 506–519 (2017)
Pinot, R., Ettedgui, R., Rizk, G., Chevaleyre, Y., Atif, J.: Randomization matters how to defend against strong adversarial attacks. In: International Conference on Machine Learning, pp. 7717–7727. PMLR (2020)
Salman, H., et al.: Provably robust deep learning via adversarially trained smoothed classifiers. arXiv preprint arXiv:1906.04584 (2019)
Schuurmans, D., Zinkevich, M.A.: Deep learning games. In: Advances in Neural Information Processing Systems, pp. 1678–1686 (2016)
Shafahi, A., et al.: Adversarial training for free! In: Advances in Neural Information Processing Systems, pp. 3358–3369 (2019)
Szegedy, C., et al.: Intriguing properties of neural networks. In: International Conference on Learning Representations (ICLR) (2014)
Xu, W., Evans, D., Qi, Y.: Feature squeezing: detecting adversarial examples in deep neural networks. In: Network and Distributed System Security Symposium (2018)
Zhang, D., Zhang, T., Lu, Y., Zhu, Z., Dong, B.: You only propagate once: accelerating adversarial training via maximal principle. In: Advances in Neural Information Processing Systems, pp. 227–238 (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Samsinger, M., Merkle, F., Schöttle, P., Pevny, T. (2021). When Should You Defend Your Classifier?. In: Bošanský, B., Gonzalez, C., Rass, S., Sinha, A. (eds) Decision and Game Theory for Security. GameSec 2021. Lecture Notes in Computer Science(), vol 13061. Springer, Cham. https://doi.org/10.1007/978-3-030-90370-1_9
Download citation
DOI: https://doi.org/10.1007/978-3-030-90370-1_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-90369-5
Online ISBN: 978-3-030-90370-1
eBook Packages: Computer ScienceComputer Science (R0)