Abstract
This paper presents a new wrapper method able to optimize simultaneously the parameters of the classifier while the size of the subset of features that better describe the input dataset is also being minimized. The search algorithm used for this purpose is based on a co-evolutionary algorithm optimizing several objectives related with different desirable properties for the final solutions, such as its accuracy, its final number of features, and the generalization ability of the classifier. Since these objectives can be sorted according to their priorities, a lexicographic approach has been applied to handle this many-objective problem, which allows the use of a simple evolutionary algorithm to evolve each one of the different sub-populations.
This work was supported by projects TIN2015-67020-P (Spanish “Ministerio de Economía y Competitividad”) and PGC2018-098813-B-C31 (Spanish “Ministerio de Ciencia, Innovación y Universidades”), and by European Regional Development Funds (ERDF).
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Ben-Tal, A.: Characterization of Pareto and lexicographic optimal solutions. In: Fandel, G., Gal, T. (eds.) Multiple Criteria Decision Making Theory and Application. Lecture Notes in Economics and Mathematical System, vol. 177, pp. 1–11. Springer, Berlin (1979). https://doi.org/10.1007/978-3-642-48782-8_1
Caruana, R., Freitag, D.: Greedy attribute selection. In: Cohen, W.W., Hirsh, H. (eds.) Proceedings of the Eleventh International Conference on International Conference on Machine Learning, ICML 1994, pp. 28–36. Morgan Kaufmann, New Brunswick, July 1994
Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Technol. 2(3), 27 (2011). http://www.csie.ntu.edu.tw/~cjlin/libsvm
Cohen, J.: A coefficient of agreement for nominal scales. Educ. Psychol. Measure. 20(1), 37–46 (1960). https://doi.org/10.1037/h0026256
Deb, K.: Multi-Objective Optimization Using Evolutionary Algorithms. Wiley Interscience Series in Systems and Optimization. Wiley, Chichester (2001)
Deb, K., Agrawal, R.B.: Simulated binary crossover for continuous search space. Complex Syst. 9(2), 115–148 (1995). https://www.complex-systems.com/abstracts/v09_i02_a02/
Deb, K., Agrawal, S.: A niched-penalty approach for constraint handling in genetic algorithms. In: Dobnikar, A., Steele, N.C., Pearson, D.W., Albrecht, R.F. (eds.) Artificial Neural Networks and Genetic Algorithms, pp. 235–243. Springer, Portorož (1999). https://doi.org/10.1007/978-3-7091-6384-9_40
Dheeru, D., Karra Taniskidou, E.: UCI machine learning repository. School of Information and Computer Sciences, University of California, Irvine, CA, USA (2017). http://archive.ics.uci.edu/ml
Drechsler, N., Sülflow, A., Drechsler, R.: Incorporating user preferences in many-objective optimization using relation \(\epsilon \)-preferred. Natural Comput. 14(3), 469–483 (2015). https://doi.org/10.1007/s11047-014-9422-0
Farina, M., Amato, P.: On the optimal solution definition for many-criteria optimization problems. In: Keller, J., Nasraoui, O. (eds.) Proceedings of the 2002 Annual Meeting of the North American Fuzzy Information Processing Society, pp. 233–238. IEEE, New Orleans, June 2002. https://doi.org/10.1109/NAFIPS.2002.1018061
Forsyth, R.S.: Zoo dataset. Mapperley Park, Nottingham, UK (1990). https://archive.ics.uci.edu/ml/datasets/Zoo
González, J., Ortega, J., Damas, M., Martín-Smith, P., Gan, J.Q.: A new multi-objective wrapper method for feature selection - accuracy and stability analysis for BCI. Neurocomputing 333(14), 407–418 (2019). https://doi.org/10.1016/j.neucom.2019.01.017
Gutlein, M., Frank, E., Hall, M., Karwath, A.: Large-scale attribute selection using wrappers. In: Smith-Miles, K., Keogh, E., Lee, V.C. (eds.) Proceedings of the 2009 IEEE Symposium on Computational Intelligence and Data Mining, CIDM 2009. IEEE, Nashville, March 2009. https://doi.org/10.1109/CIDM.2009.4938668
Kennedy, J., Eberhart, R.: Particle swarm optimization. In: Proceedings of the IEEE International Conference on Neural Networks, ICNN 1995, vol. 6, pp. 1942–1948. IEEE, Perth, November 1995. https://doi.org/10.1109/ICNN.1995.488968
Khalid, S., Khalil, T., Nasreen, S.: A survey of feature selection and feature extraction techniques in machine learning. In: Arai, K., Mellouk, A. (eds.) Proceedings of the 2014 Science and Information Conference, pp. 372–378. The Science and Information (SAI) Organization, London, August 2014. https://doi.org/10.1109/SAI.2014.6918213
Khare, V.R., Yao, X., Sendhoff, B.: Credit assignment among neurons in co-evolving populations. In: Yao, X., et al. (eds.) PPSN 2004. LNCS, vol. 3242, pp. 882–891. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-30217-9_89
Khosravani, S., Jalali, M., Khajepour, A., Kasaiezadeh, A., Chen, S.K., Litkouhi, B.: Application of lexicographic optimization method to integrated vehicle control systems. IEEE Trans. Ind. Electron. 65(12), 9677–9686 (2018). https://doi.org/10.1109/TIE.2018.2821625
Klepikova, M.G.: The stability of lexicographic optimization problem. USSR Comput. Math. Math. Phys. 25(1), 21–29 (1985). https://doi.org/10.1016/0041-5553(85)90037-0
Kohavi, R., John, G.H.: Wrappers for feature subset selection. Artif. Intell. 97(1–2), 273–324 (1997). https://doi.org/10.1016/S0004-3702(97)00043-X
Li, B., Li, J., Tang, K., Yao, X.: Many-objective evolutionary algorithms: a survey. ACM Comput. Surv. 48(1), 13 (2015). https://doi.org/10.1145/2792984
Luke, S., et al.: ECJ 26. A Java-based evolutionary computation research system. https://cs.gmu.edu/~eclab/projects/ecj/
Madonado, S., Weber, R.: A wrapper method for feature selection using support vector machines. Inf. Sci. 179(13), 2208–2217 (2009). https://doi.org/10.1016/j.ins.2009.02.014
Marill, T., Green, D.: On the effectiveness of receptors in recognition systems. IEEE Trans. Inf. Theory 9(1), 11–17 (1963). https://doi.org/10.1109/TIT.1963.1057810
Michalewicz, Z.: Genetic Algorithms + Data Structures = Evolution Programs, 3rd edn. Springer, Berlin (1998)
Podinovskii, V.V., Gavrilov, V.M.: Optimization with respect to successive criteria (Optimizatsiya po posledovatel’no primenyaemym kriteriyam). Sovetskoe Radio, Moscow, Russia (1975)
Popovici, E., Bucci, A., Wiegand, R.P., De Jong, E.D.: Coevolutionary principles. In: Rozenberg, G., Bäck, T., Kok, J.N. (eds.) Handbook of Natural Computing, pp. 987–1033. Springer, Berlin (2012). https://doi.org/10.1007/978-3-540-92910-9_31
Potter, M.A., De Jong, K.A.: Cooperative coevolution: an architecture for evolving coadapted subcomponents. Evol. Comput. 8(1), 1–29 (2000). https://doi.org/10.1162/106365600568086
Rasekhipour, Y., Fadakar, I., Khajepour, A.: Autonomous driving motion planning with obstacles prioritization using lexicographic optimization. Control Eng. Pract. 77, 235–246 (2018). https://doi.org/10.1016/j.conengprac.2018.04.014
Schmiedle, F., Drechsler, N., Große, D., Drechsler, R.: Priorities in multi-objective optimization for genetic programming. In: Spector, L., Goodman, E.D., Wu, A., Langdon, W.B., Voigt, H.M. (eds.) Proceedings of the 3rd Annual Conference on Genetic and Evolutionary Computation, GECCO 2001, pp. 129–136. Morgan Kaufmann Publishers Inc., San Francisco, July 2001. https://dl.acm.org/citation.cfm?id=2955256
Whitney, A.W.: A direct method of nonparametric measurement selection. EEE Trans. Comput. C–20(9), 1100–1103 (1971). https://doi.org/10.1109/T-C.1971.223410
Xue, B., Zhang, M., Browne, W.N.: New fitness functions in binary particle swarm optimisation for feature selection. In: Abbass, H., Essam, D., Sarker, R. (eds.) Proceedings of the 2012 IEEE Congress on Evolutionary Computation, CEC 2012. IEEE, Brisbane, June 2012. https://doi.org/10.1109/CEC.2012.6256617
Xue, B., Zhang, M., Browne, W.N.: Particle swarm optimization for feature selection in classification: a multi-objective approach. IEEE Trans. Cybern. 43(6), 1656–1671 (2013). https://doi.org/10.1109/TSMCB.2012.2227469
Zykina, A.V.: A lexicographic optimization algorithm. Autom. Remote Control 65(3), 363–368 (2004). https://doi.org/10.1023/B:AURC.0000019366.84601.8e
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
González, J., Ortega, J., Damas, M., Martín-Smith, P. (2019). Many-Objective Cooperative Co-evolutionary Feature Selection: A Lexicographic Approach. In: Rojas, I., Joya, G., Catala, A. (eds) Advances in Computational Intelligence. IWANN 2019. Lecture Notes in Computer Science(), vol 11507. Springer, Cham. https://doi.org/10.1007/978-3-030-20518-8_39
Download citation
DOI: https://doi.org/10.1007/978-3-030-20518-8_39
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-20517-1
Online ISBN: 978-3-030-20518-8
eBook Packages: Computer ScienceComputer Science (R0)