Abstract
Artificial Neural Networks (ANN) models – a form of discriminative models – are the workhorse of deep learning research, and have resulted in a remarkable performance on a range of applications on a large variety of datasets. On tabular datasets, ANN models are preferable when learning from large quantities of data as non-parametric models such as Random Forest and XGBoost cannot be easily used due to their inherent in-core data processing (i.e., they require loading all the data in memory). The applicability and effectiveness of ANN models, however, come with a price. They have been shown to be susceptible to adversarial attacks, which can greatly compromise their performance and trust in their utilization. There has been a surge in research in developing effective defence strategies for adversarial attacks on ANN models, e.g., Madry, D2A3, etc. Recently, it has been shown that generative models are more robust to adversarial attacks than discriminative models. A natural question is – can generative models be used as a defence for discriminative models against adversarial attacks? This work addresses this question, where we study the power of generative models in warding off adversarial attacks for discriminative models. In this work, we propose an effective defence model – gD2A3 – that exploits the generative-discriminative equivalence of some ANN models. It uses the learned probabilities from a generative model to initialize the input layer parameters of a standard ANN model, and utilizes \(L_2\) regularization of the input layer parameters as a defence mechanism. We show that our proposed model leads to better results than the state-of-the-art method D2A3 by conducting a thorough empirical study on a variety of datasets with two major adversarial attacks.
J. Zhou and N. Zaidi—Equal Contribution.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
By tabular data, we mean, a dataset in tabular format with discrete or categorical features such that the correlation among features is unknown.
- 2.
Note, the training time for KDB, typically with small order values of k due to the computational cost, i.e., \(k=0, k=1\), is negligible compared to training the deep learning model.
References
Ballet, V., Renard, X., Aigrain, J., Laugel, T., Frossard, P., Detyniecki, M.: Imperceptible adversarial attacks on tabular data. arXiv preprint arXiv:1911.03274 (2019)
Kurakin, A., Goodfellow, I., Bengio, S.: Adversarial machine learning at scale. arXiv preprint arXiv:1611.01236 (2016)
Li, Y.: Are generative classifiers more robust to adversarial attacks? CoRR abs/1802.06552 (2018), https://arxiv.org/abs/1802.06552
Liu, X., Hsieh, C.: From adversarial training to generative adversarial networks. CoRR abs/1807.10454 (2018), https://arxiv.org/abs/1807.10454
Madry, A., Makelov, A., Schmidt, L., Tsipras, D., Vladu, A.: Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083 (2017)
Moosavi-Dezfooli, S.M., Fawzi, A., Frossard, P.: Deepfool: a simple and accurate method to fool deep neural networks. In: CVPR (2016)
Ng, A., Jordan, M.: On discriminative vs. generative classifiers: A comparison of logistic regression and naive bayes. In: Dietterich, T., Becker, S., Ghahramani, Z. (eds.) Advances in Neural Information Processing Systems. vol. 14. MIT Press (2001), https://proceedings.neurips.cc/paper/2001/file/7b7a53e239400a13bd6be6c91c4f6c4e-Paper.pdf
Qiu, S., Liu, Q., Zhou, S., Wu, C.: Review of artificial intelligence adversarial attack and defense technologies. Applied Sciences 9(5), 909 (2019)
Schott, L., Rauber, J., Brendel, W., Bethge, M.: Robust perception through analysis by synthesis. CoRR abs/1805.09190 (2018), https://arxiv.org/abs/1805.09190
Zaidi, N.A., Carman, M.J., Cerquides, J., Webb, G.I.: Naive-bayes inspired effective pre-conditioner for speeding-up logistic regression. In: 2014 IEEE International Conference on Data Mining. pp. 1097–1102 (2014). https://doi.org/10.1109/ICDM.2014.53
Zhang, H., Yu, Y., Jiao, J., Xing, E., El Ghaoui, L., Jordan, M.: Theoretically principled trade-off between robustness and accuracy. In: ICML (2019)
Zhou, J., Zaidi, N., Zhang, Y., Li, G.: Discretization inspired defence algorithm against adversarial attacks on tabular data. In: Pacific-Asia Conference on Knowledge Discovery and Data Mining. pp. 367–379. Springer (2022)
Zhou, M., Wu, J., Liu, Y., Liu, S., Zhu, C.: Dast: Data-free substitute training for adversarial attacks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 234–243 (2020)
Acknowledgement
This research is funded through a Defence Science and Technology Group DAIRNet grant.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Zhou, J., Zaidi, N., Zhang, Y., Montague, P., Kim, J., Li, G. (2023). Leveraging Generative Models for Combating Adversarial Attacks on Tabular Datasets. In: Kashima, H., Ide, T., Peng, WC. (eds) Advances in Knowledge Discovery and Data Mining. PAKDD 2023. Lecture Notes in Computer Science(), vol 13935. Springer, Cham. https://doi.org/10.1007/978-3-031-33374-3_12
Download citation
DOI: https://doi.org/10.1007/978-3-031-33374-3_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-33373-6
Online ISBN: 978-3-031-33374-3
eBook Packages: Computer ScienceComputer Science (R0)