Abstract
In binary classification, to improve the performance for unknown samples, excluding as many unnecessary features representing samples as possible is necessary. Of various methods of feature selection, the filter method calculates indices beforehand for each feature, and the wrapper method finds combinations of features having the maximum performance from all combinations of features. In this paper, we propose a novel feature selection method using distance from the classification boundary and a Monte Carlo simulation. Synthetic sample sets for binary classification were provided, and features determined by random numbers were added to each sample. For these sample sets, the conventional methods and the proposed method were applied, and it was examined whether the feature forming the boundary was selected. Our results demonstrate that feature selection was difficult with the conventional methods but possible with our proposed method.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
John, G.H., Kohavi, R., Pfleger, K.: Irrelevant features and the subset selection problem (1994)
Almuallim, H., Dietterich, T.G.: Learning with many irrelevant features. In: AAAI, vol. 91 (1991)
Blum, A.L., Langley, P.: Selection of relevant features and examples in machine learning. Artif. Intell. 97, 245–271 (1998)
Chandrashekar, G., Sahin, F.: A survey on feature selection methods. Comput. Electr. Eng. 40, 16–28 (2014)
Vergara, J.R., Estévez, P.A.: A review of feature selection methods based on mutual information. Neural Comput. Appl. 24, 175–186 (2015)
Li, Y., Li, T., Liu, H.: Recent advances in feature selection and its applications. Knowl. Inf. Syst. 53, 551–577 (2017)
Guyon, I., Elisseeff, A.: An introduction to variable and feature selection. J. Mach. Learn. Res. 3, 1157–1182 (2003)
Jain, A.K., Duin, R.P.W., Mao, J.: Statistical pattern recognition: a review. IEEE Trans. Pattern Anal. 22, 4–37 (2000)
Shannon, C.: A mathematical theory of communication. ACM SIGMOBILE Mobile Comput. Commun. Rev. 5, 3–55 (2001)
Bennasar, M., Hicks, Y., Setchi, R.: Feature selection using joint mutual information maximisation. Expert Syst. Appl. 42, 8520–8532 (2015)
Zhao, G., Wu, Y., Chen, F., Zhang, J., Bai, J.: Effective feature selection using feature vector graph for classification. Neurocomputing 151, 376–389 (2015)
Pes, B., Dessì, N., Angioni, M.: Exploiting the ensemble paradigm for stable feature selection: a case study on high-dimensional genomic data. Inf. Fusion 35, 132–147 (2017)
Sánchez-Maroño, N., Alonso-Betanzos, A., Tombilla-Sanromán, M.: Filter methods for feature selection – a comparative study. In: Yin, H., Tino, P., Corchado, E., Byrne, W., Yao, X. (eds.) IDEAL 2007. LNCS, vol. 4881, pp. 178–187. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-77226-2_19
Mitchell, T.M.: Machine Learning, vol. 45. McGraw Hill, Burr Ridge (1997)
Quinlan, J.: Induction of decision trees. Mach. Learn. 1, 81–106 (1986)
Kira, K., Rendell, L.A.: The feature selection problem: traditional methods and a new algorithm. In: AAAI, vol. 2 (1992)
Kononenko, I.: Estimating attributes: analysis and extensions of RELIEF. In: Bergadano, F., De Raedt, L. (eds.) ECML 1994. LNCS, vol. 784, pp. 171–182. Springer, Heidelberg (1994). https://doi.org/10.1007/3-540-57868-4_57
Liu, H., Motoda, H., Yu, L.: Feature selection with selective sampling. In: ICML (2002)
Kira, K., Rendell, L.A.: A practical approach to feature selection (1992)
Kohavi, R., John, G.H.: Wrappers for feature subset selection. Artif. Intell. 97, 273–324 (1997)
Panthong, R., Srivihok, A.: Wrapper feature subset selection for dimension reduction based on ensemble learning algorithm. Procedia Comput. Sci. 72, 162–169 (2015)
Mi, H., Petitjean, C., Dubray, B., Vera, P., Ruan, S.: Robust feature selection to predict tumor treatment outcome. Artif. Intell. Med. 64, 195–204 (2015)
Vapnik, V.: Pattern recognition using generalized portrait method. Autom. Remote Control. 24, 774–780 (1963)
Boser, B.E., Guyon, I.M., Vapnik, V.N.: A training algorithm for optimal margin classifiers, pp. 144–152 (1992)
Aizerman, M.A.: Theoretical foundations of the potential function method in pattern recognition learning. Autom. Remote Control 25, 821–837 (1964)
Buhmann, M.D.: Radial Basis Functions: Theory and Implementations, vol. 12. Cambridge University Press, Cambridge (2003)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Koyama, Y., Ikeda, K., Sakumura, Y. (2018). Feature Selection Using Distance from Classification Boundary and Monte Carlo Simulation. In: Cheng, L., Leung, A., Ozawa, S. (eds) Neural Information Processing. ICONIP 2018. Lecture Notes in Computer Science(), vol 11304. Springer, Cham. https://doi.org/10.1007/978-3-030-04212-7_9
Download citation
DOI: https://doi.org/10.1007/978-3-030-04212-7_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-04211-0
Online ISBN: 978-3-030-04212-7
eBook Packages: Computer ScienceComputer Science (R0)