Improved Crowding Distance in Multi-objective Optimization for Feature Selection in Classification

Abstract

Feature selection is an essential preprocessing step in data mining and machine learning. A feature selection task can be treated as a multi-objective optimization problem which simultaneously minimizes the classification error and the number of selected features. Many existing feature selection approaches including multi-objective methods neglect that there exists multiple optimal solutions in feature selection. There can be multiple different optimal feature subsets which achieve the same or similar classification performance. Furthermore, when using evolutionary multi-objective optimization for feature selection, a crowding distance metric is typically used to play a role in environmental selection. However, some existing calculations of crowding metrics based on continuous/numeric values are inappropriate for feature selection since the search space of feature selection is discrete. Therefore, this paper proposes a new environmental selection method to modify the calculation of crowding metrics. The proposed approach is expected to help a multi-objective feature selection algorithm to find multiple potential optimal feature subsets. Experiments on sixteen different datasets of varying difficulty show that the proposed approach can find more diverse feature subsets, achieving the same classification performance without deteriorating performance regarding hypervolume and inverted generational distance.