Abstract
Multi-label classification is an important topic in machine learning, where each instance can be classified into more than one category, i.e., have a subset of labels instead of only one. Among existing methods, ML-kNN [25], the direct extension of k-nearest neighbors algorithm to the multi-label scenario, has received much attention due to its conciseness, great interpretability, and good performance. However, ML-kNN usually suffers from a terrible storage cost since all training instances need to be saved in the memory. To address this issue, a natural way is instance selection, intending to save the important instances while deleting the redundant ones. However, previous instance selection methods mainly focus on the single-label scenario, which may have a poor performance when adapted to the multi-label scenario. Recently, few works begin to consider the multi-label scenario, but their performance is limited due to the inapposite modeling. In this paper, we propose to formulate the instance selection problem for ML-kNN as a natural bi-objective optimization problem that considers the accuracy and the number of retained instances simultaneously, and adapt NSGA-II to solve it. Experiments on six real-world data sets show that our proposed method can achieve both not worse prediction accuracy and significantly better compression ratio, compared with state-of-the-art methods.
C. Qian—This work was supported by the National Science Foundation of China (62022039, 62106098).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Arnaiz-González, Á., Díez-Pastor, J., Diez, J.J.R., García-Osorio, C.: Local sets for multi-label instance selection. Appl. Soft Comput. 68, 651–666 (2018)
Ashfaq, R.A.R., He, Y., Chen, D.: Toward an efficient fuzziness based instance selection methodology for intrusion detection system. Int. J. Mach. Learn. Cybern. 8(6), 1767–1776 (2017)
Bäck, T.: Evolutionary Algorithms in Theory and Practice: Evolution Strategies, Evolutionary Programming. Oxford University Press, Genetic Algorithms (1996)
Boutell, M.R., Luo, J., Shen, X., Brown, C.M.: Learning multi-label scene classification. Patt. Recogn. 37(9), 1757–1771 (2004)
Brighton, H., Mellish, C.: On the consistency of information filters for lazy learning algorithms. In: Proceedings of the 3rd Principles of Data Mining and Knowledge Discovery (PKDD 1999), Prague, Czech Republic, pp. 283–288 (1999)
Cai, X., Wang, P., Du, L., Cui, Z., Zhang, W., Chen, J.: Multi-objective three-dimensional dv-hop localization algorithm with NSGA-II. IEEE Sens. J. 19(21), 10003–10015 (2019)
Deb, K., Agrawal, S., Pratap, A., Meyarivan, T.: A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans. Evolut. Comput. 6(2), 182–197 (2002)
Feng, C., Qian, C., Tang, K.: Unsupervised feature selection by Pareto optimization. In: Proceedings of the 33rd AAAI Conference on Artificial Intelligence (AAAI 2019), Honolulu, HI, pp. 3534–3541 (2019)
Fu, Y., Zhu, X., Li, B.: A survey on instance selection for active learning. Knowl. Inf. Syst. 35(2), 249–283 (2013)
García-Pedrajas, N., García, G.C.: Cooperative coevolutionary instance selection for multilabel problems. Knowl.-Based Syst. 234, 107569 (2021)
Holland, J.H.: Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence. MIT Press, Cambridge (1992)
Hong, W., Tang, K., Zhou, A., Ishibuchi, H., Yao, X.: A scalable indicator-based evolutionary algorithm for large-scale multiobjective optimization. IEEE Trans. Evolut. Comput. 23(3), 525–537 (2019)
Hong, W., Yang, P., Tang, K.: Evolutionary computation for large-scale multi-objective optimization: a decade of progresses. Int. J. Autom. Comput. 18(2), 155–169 (2021)
Liu, Y., Chen, Z., Fu, A.W., Wong, R.C., Dai, G.: Optimal location query based on \(k\) nearest neighbours. Front. Comput. Sci. 15(2), 152606 (2021)
Lu, Z., et al.: NSGA-Net: neural architecture search using multi-objective genetic algorithm (extended abstract). In: Proceedings of the 29th International Joint Conference on Artificial Intelligence (IJCAI 2020), Yokohama, Japan, pp. 4750–4754 (2020)
Mukhopadhyay, A., Maulik, U., Bandyopadhyay, S., Coello, C.A.C.: A survey of multiobjective evolutionary algorithms for data mining: Part I. IEEE Trans. Evolut. Comput. 18(1), 4–19 (2014)
Olvera-López, J.A., Carrasco-Ochoa, J.A., Trinidad, J.F.M., Kittler, J.: A review of instance selection methods. Artif. Intell. Rev. 34(2), 133–143 (2010)
Qian, C., Liu, D., Zhou, Z.: Result diversification by multi-objective evolutionary algorithms with theoretical guarantees. Artif. Intell. 309, 103737 (2022)
Qian, C., Yu, Y., Zhou, Z.: Pareto ensemble pruning. In: Proceedings of the 29th AAAI Conference on Artificial Intelligence (AAAI 2015), Austin, TX, pp. 2935–2941 (2015)
Qian, C., Yu, Y., Zhou, Z.: Subset selection by Pareto optimization. In: Advances in Neural Information Processing Systems 28 (NIPS 2015), Montreal, Canada, pp. 1774–1782 (2015)
Qu, B., Zhu, Y., Jiao, Y.C., Wu, M.Y., Suganthan, P.N., Liang, J.J.: A survey on multi-objective evolutionary algorithms for the solution of the environmental/economic dispatch problems. Swarm Evolut. Comput. 38, 1–11 (2018)
Wilson, D.L.: Asymptotic properties of nearest neighbor rules using edited data. IEEE Trans. Syst. Man Cybern. 2(3), 408–421 (1972)
Wu, X., Zhou, Z.: A unified view of multi-label performance measures. In: Proceedings of the 34th International Conference on Machine Learning (ICML 2017), Sydney, Australia, pp. 3780–3788 (2017)
Wu, Y., He, Y., Qian, C., Zhou, Z.: Multi-objective evolutionary ensemble pruning guided by margin distribution. In: Proceedings of the 17th International Conference on Parallel Problem Solving from Nature (PPSN 2022), Dortmund, Germany, pp. 427–441 (2022)
Zhang, M., Zhou, Z.: ML-KNN: a lazy learning approach to multi-label learning. Patt. Recogn. 40(7), 2038–2048 (2007)
Zhang, M., Zhou, Z.: A review on multi-label learning algorithms. IEEE Trans. Knowl. Data Eng. 26(8), 1819–1837 (2014)
Zhou, A., Qu, B., Li, H., Zhao, S., Suganthan, P.N., Zhang, Q.: Multiobjective evolutionary algorithms: a survey of the state of the art. Swarm Evolut. Comput. 1(1), 32–49 (2011)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Liu, D., Shang, H., Hong, W., Qian, C. (2022). Multi-objective Evolutionary Instance Selection for Multi-label Classification. In: Khanna, S., Cao, J., Bai, Q., Xu, G. (eds) PRICAI 2022: Trends in Artificial Intelligence. PRICAI 2022. Lecture Notes in Computer Science, vol 13629. Springer, Cham. https://doi.org/10.1007/978-3-031-20862-1_40
Download citation
DOI: https://doi.org/10.1007/978-3-031-20862-1_40
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-20861-4
Online ISBN: 978-3-031-20862-1
eBook Packages: Computer ScienceComputer Science (R0)