Abstract
Semi-Supervised Learning (SSL) has exhibited strong effectiveness in boosting the performance of classification models with the aid of a large amount of unlabeled data. Recently, regularizing the classifier with the help of adversarial examples has proven effective for semi-supervised learning. Existing methods hypothesize that the adversarial examples are based on the pixel-wise perturbation of the original samples. However, other types of adversarial examples (e.g., with spatial transformation) should also be useful for improving the robustness of the classifier. In this paper, we propose a new generalized framework based on adversarial networks, which is able to generate various types of adversarial examples. Our model consists of two modules which are trained in an adversarial process: a generator mapping the original samples to adversarial examples which can fool the classifier, and a classifier that tries to classify the original samples and the adversarial examples consistently. We evaluate our model on several datasets, and the experimental results show that our model outperforms the state-of-the-art methods for semi-supervised learning. The experiments also demonstrate that our model can generate adversarial examples with various types of perturbation such as local spatial transformation, color transformation, and pixel-wise perturbation. Moreover, our model is also applicable to supervised learning, performing as a regularization term to improve the generalization performance of the classifier.
Y. Ma and X. Mao—Contributed equally to this work.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Arjovsky, M., Chintala, S., Bottou, L.: Wasserstein GAN. arXiv:1701.07875 (2017)
Azulay, A., Weiss, Y.: Why do deep convolutional networks generalize so poorly to small image transformations? arXiv:1805.12177 (2018)
Bachman, P., Alsharif, O., Precup, D.: Learning with pseudo-ensembles. In: Proceedings of the Neural Information Processing Systems (NeurIPS), pp. 3365–3373 (2014)
Baluja, S., Fischer, I.: Learning to attack: adversarial transformation networks. In: Association for the Advancement of Artificial Intelligence (AAAI) (2018)
Cisse, M., Bojanowski, P., Grave, E., Dauphin, Y., Usunier, N.: Parseval networks: improving robustness to adversarial examples. In: International Conference on Machine Learning (ICML) (2017)
Collobert, R., Sinz, F.H., Weston, J., Bottou, L.: Large scale transductive SVMs. J. Mach. Learn. Res. (JMLR) (2006)
Dai, Z., Yang, Z., Yang, F., Cohen, W.W., Salakhutdinov, R.: Good semi-supervised learning that requires a bad GAN. In: Advances in Neural Information Processing Systems (NeurIPS) (2017)
Dumoulin, V., et al.: Adversarially learned inference. In: International Conference on Learning Representations (ICLR) (2017)
Gilmer, J., et al.: Adversarial spheres. arXiv:1801.02774 (2018)
Goodfellow, I., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems (NeurIPS), pp. 2672–2680 (2014)
Goodfellow, I.J., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial examples. In: International Conference on Learning Representations (ICLR) (2015)
He, K., Gkioxari, G., Dollár, P., Girshick, R.: Mask R-CNN. In: International Conference on Computer Vision (ICCV) (2017)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Computer Vision and Pattern Recognition (CVPR) (2016)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: Proceedings of the International Conference on Learning Representations (ICLR) (2015)
Kingma, D.P., Rezende, D.J., Mohamed, S., Welling, M.: Semi-supervised learning with deep generative models. In: Advances in Neural Information Processing Systems (NeurIPS) (2014)
Kumar, A., Sattigeri, P., Fletcher, T.: Semi-supervised learning with GANs: manifold invariance with improved inference. In: Proceedings of the Neural Information Processing Systems (NeurIPS), pp. 5534–5544 (2017)
Laine, S., Aila, T.: Temporal ensembling for semi-supervised learning. In: International Conference on Learning Representations (ICLR) (2017)
Li, C., Xu, K., Zhu, J., Zhang, B.: Triple generative adversarial nets. In: Advances in Neural Information Processing Systems (NeurIPS) (2017)
Mao, X., Li, Q., Xie, H., Lau, R.Y., Wang, Z., Smolley, S.P.: Least squares generative adversarial networks. In: International Conference on Computer Vision (ICCV) (2017)
Miyato, T., Maeda, S.I., Koyama, M., Ishii, S.: Virtual adversarial training: a regularization method for supervised and semi-supervised learning. IEEE Trans. Pattern Anal. Mach. Intell. (2018)
Paszke, A., et al.: Automatic differentiation in PyTorch. In: Proceedings of the Advances in Neural Information Processing Systems (NeurIPS) Workshop (2017)
Radford, A., Metz, L., Chintala, S.: Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv:1511.06434 (2015)
Rasmus, A., Valpola, H., Honkala, M., Berglund, M., Raiko, T.: Semi-supervised learning with ladder networks. In: Advances in Neural Information Processing Systems (NeurIPS) (2015)
Sajjadi, M., Javanmardi, M., Tasdizen, T.: Regularization with stochastic transformations and perturbations for deep semi-supervised learning. In: Advances in Neural Information Processing Systems (NeurIPS) (2016)
Salimans, T., et al.: Improved techniques for training GANs. In: Advances in Neural Information Processing Systems (NeurIPS), pp. 2226–2234 (2016)
Springenberg, J.T.: Unsupervised and semi-supervised learning with categorical generative adversarial networks. In: International Conference on Learning Representations (ICLR) (2016)
Szegedy, C., et al.: Intriguing properties of neural networks. In: International Conference on Learning Representations (ICLR) (2014)
Xiao, C., Li, B., Zhu, J.Y., He, W., Liu, M., Song, D.: Generating adversarial examples with adversarial networks. In: International Joint Conference on Artificial Intelligence (IJCAI) (2018)
Xiao, C., Zhu, J.Y., Li, B., He, W., Liu, M., Song, D.: Spatially transformed adversarial examples. In: International Conference on Learning Representations (ICLR) (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Ma, Y., Mao, X., Chen, Y., Li, Q. (2019). Generating Adversarial Examples by Adversarial Networks for Semi-supervised Learning. In: Cheng, R., Mamoulis, N., Sun, Y., Huang, X. (eds) Web Information Systems Engineering – WISE 2019. WISE 2020. Lecture Notes in Computer Science(), vol 11881. Springer, Cham. https://doi.org/10.1007/978-3-030-34223-4_8
Download citation
DOI: https://doi.org/10.1007/978-3-030-34223-4_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-34222-7
Online ISBN: 978-3-030-34223-4
eBook Packages: Computer ScienceComputer Science (R0)