Abstract
Feature selection is one of key problems in machine learning and data mining. It has been widely accepted that adversarial training is an effective strategy to improve the accuracy and robustness of classifiers. In this paper, in order to improve the performance of feature selection, adversarial training is also adopted, and an adversarial training based feature selection framework is proposed. To validate the effectiveness of the proposed feature selection framework, three classical feature selection algorithms, i.e. Relief-F, Fisher Score and minimum Redundancy and maximum Relevance (mRMR) are chosen and two methods are used to generate adversarial examples in experiments. The experimental results on benchmark datasets containing low-dimension and high-dimension datasets demonstrate show that adversarial training is able to improve the performance of classical feature selection methods in most cases.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Li, Y., Li, T., Liu, H.: Recent advances in feature selection and its applications. Knowl. Inf. Syst. 53(3), 551–577 (2017)
Miyato, T., Dai, A.M., Goodfellow, I.: Adversarial training methods for semi-supervised text classification. In: International Conference on Learning Representations (2017)
Graves, A., Schmidhuber, J.: Framewise phoneme classification with bidirectional LSTM and other neural network architectures. Neural Netw. 18(5–6), 602–610 (2005)
Miyato, T., Maeda, S., Koyama, M.: Distributional smoothing with virtual adversarial training. In: International Conference on Learning Representations (2016)
Chivukula, A.S., Liu, W.: Adversarial learning games with deep learning models. In: 2017 International Joint Conference on Neural Networks (IJCNN), pp: 2758–2767. IEEE (2017)
Szegedy, C., Zaremba, W., Sutskever, I.: Intriguing properties of neural networks. Computer Science (2013)
Bartlett, P.L., Wegkamp, M.H.: Classification with a reject option using a hinge loss. J. Mach. Learn. Res. 2008, 1823–1840 (2008)
Cortes, C., Vladimir, V.: Support-vector networks. Mach. Learn. 20(3), 273–297 (1995)
Ruder, S.: An overview of gradient descent optimization algorithms. arXiv preprint (2016). arXiv:1609.04747
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: International Conference on Learning Representations (2015)
Olvi, M.L.: Cancer diagnosis via linear programming. SIAM News 23(5), 1–18 (1990)
Little, M.A., McSharry, P.E., Roberts, S.J.: Exploiting nonlinear recurrence and fractal scaling properties for voice disorder detection. Biomed. Eng. Online 6(1), 23 (2007)
Kononenko, I., Edvard, S.: Induction of decision trees using RELIEFF. In: Proceedings of the ISSEK94 workshop on mathematical and statistical methods in artificial intelligence. Springer, Vienna (1995)
Gu, Q., Li, Z., Han, J.: Generalized fisher score for feature selection. In: Proceedings of the Twenty-Seventh Conference on Uncertainty in Artificial Intelligence (2011)
Peng, H., Long, F., Ding, C.: Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy. IEEE Trans. Pattern Anal. Mach. Intell. 27(8), 1226–1238 (2005)
Dietterich, T.G.: Approximate statistical tests for comparing supervised classification learning algorithms. Neural Comput. 10(7), 1895–1923 (1998)
Donahue, J., Philipp, K., Trevor, D.: Adversarial feature learning. In: International Conference on Learning Representations (2017)
Xu, W., David, E., Yan, Q.: Feature squeezing: detecting adversarial examples in deep neural networks. In: Network and Distributed Systems Security Symposium (NDSS) (2018)
Acknowledgement
This work was partially supported by the National Key Research and Development Program of China 2018YFB1003702 and Natural Science Foundation of China (No. 61603197, 61772284, 41571389).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Liu, B., Han, K., Hang, J., Li, Y. (2019). Adversarial Training Based Feature Selection. In: Liu, F., Xu, J., Xu, S., Yung, M. (eds) Science of Cyber Security. SciSec 2019. Lecture Notes in Computer Science(), vol 11933. Springer, Cham. https://doi.org/10.1007/978-3-030-34637-9_7
Download citation
DOI: https://doi.org/10.1007/978-3-030-34637-9_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-34636-2
Online ISBN: 978-3-030-34637-9
eBook Packages: Computer ScienceComputer Science (R0)