An efficient binary slime mould algorithm integrated with a novel attacking-feeding strategy for feature selection

Highlights

• Slime mould optimization algorithm is used for the first time for feature selection.

• The new local search algorithm was developed to avoid local optima problem.

• The proposed algorithms show high efficiency over the 28 datasets.

• The proposed algorithms are compared with six well-known optimization algorithms.

• The proposed algorithms outperformed all the competing ones.

Abstract

Feature selection refers to a process used to reduce the dimension of a dataset in order to obtain the optimal features subset for machine learning and data mining algorithms. This aids the achievement of higher classification accuracy in addition to reducing the training time of a learning algorithm as a result of the removal of redundant and less-informative features. In this paper, four binary versions of the slime mould algorithm (SMA) are proposed for feature selection, in which the standard SMA is incorporated with the most appropriate transfer function of eight V-Shaped and S-Shaped transfer functions. The first version converts the standard SMA, which has not been used yet for feature selection to the best of our knowledge, into a binary version (BSMA). The second, abbreviated as TMBSMA, integrates BSMA with two-phase mutation (TM) to further exploit better solutions around the best-so-far. The third version, abbreviated as AFBSMA, combines BSMA with a novel attacking-feeding strategy (AF) that trades off exploration and exploitation based on the memory saving of each particle. Finally, TM and AF are integrated with BSMA to produce better solutions, in a version called FMBSMA. The k-nearest neighbors (KNN) algorithm, one of the common classification and regression algorithms in machine learning, is used to measure the classification accuracy of the selected features. To validate the performance of the four proposed versions of BSMA, 28 well-known datasets are employed from the UCI repository. The experiments confirm the efficacy of the AF method in providing better results. Furthermore, after comparing the four versions, the FMBSMA version is shown to be the best compared with the other three versions and six state-of-art feature selection algorithms.

Introduction

Feature selection (FS) is an essential process in many fields such as biology, finance, and telecommunications to remove insignificant and redundant features that increase training time, and lead to over-fitting when applying machine learning methods (Abdel-Basset et al., 2020, Anter and Ali, 2020. When training a classification approach (such as artificial neural networks (Xu, Qiao, Peng, and Zhang (2004), decision tree (Hssina, Merbouha, Ezzikouri, and Erritali (2014), k-nearest neighbors (Amendolia, Gianfranco Cossu, Ganadu, Masala, and Mura (2003), and support vector machine (Amendolia et al. (2003) on a dataset with high dimensions may lead to the over-fitting problem, and subsequently the trained model is not able to predict the new test instances with sufficient accuracy. So, the FS process is used to remove redundant and noisy features of the original dataset features which then reduces the training time of the machine learning model and improves the classification accuracy. The classification process assigns each of the samples to a specific class. Techniques proposed in the literature for tackling FS problems fall into three categories: wrapper, filter, and embedded methods.

In wrapper methods, a machine learning algorithm is typically used to measure the accuracy of the selected features (Balasaraswathi, Sugumaran, & Hamid, 2017). Unfortunately, those methods waste time in measuring the classification accuracy of the extracted features due to using a machine learning algorithm to be trained on the training dataset based on the extracted subset of features, and then its performance is observed on the test dataset. However, when the number of features is drawn from data sets of large-scale problems, this method becomes unwieldy. As a result, filter methods were developed to measure the classification accuracy of the extracted features based on the properties of the data. Although filter methods reduce the time required, they cannot achieve the same classification accuracy as wrapper methods for the identified features. Embedded or hybrid methods are specialized in machine learning, and therefore have high classification accuracies with reduced time complexity.

Many techniques such as mutual information (Peng, Long, & Ding, 2005), information gain (Uğuz, 2011), relief (Urbanowicz, Meeker, La Cava, Olson, & Moore, 2018), depth search (Dash & Liu, 2003), and breadth search (Dash & Liu, 2003) are used to select the optimal subset of features (Chandrashekar & Sahin, 2014). However, such techniques are hamstrung in their search for the optimal subset for datasets with a very large number of features due to the number of combinations (solutions) that increase with the number of features. For example, for $d$ features, the method needs to observe $2^d$ combinations of the features to determine the optimal combination and practically, this is impossible. So, researchers have moved towards meta-heuristic algorithms that have shown promising performance in many fields and have become the best alternatives to alleviate the drawbacks of traditional search techniques. Despite the superiority of the meta-heuristics algorithms, however, most meta-heuristic algorithms still suffer from problems such as becoming trapped in local minima, the imbalance between the exploration and exploitation operators, and lack of diversity.

Recently, a new meta-heuristic algorithm known as the slime mould algorithm (SMA) has been proposed for solving continuous optimization problems Cheng, Li, Tian, Zhang, Yang, Jin, and Yao (2017). SMA simulates the nature of slime mould in using an oscillation mode to obtain the optimal path for collecting food with significant exploration ability and intensification propensity. The success of SMA in solving continuous optimization problems motivates us to propose a binary version for solving FS. To the best of our knowledge, the binary version of the SMA for the feature selection problem has not so far been proposed. Therefore, in this paper, we propose three variants from SMA in addition to the standard one for tackling the feature selection problem. The first version is the standard continuous SMA converted into a binary technique using selected families of V-Shaped and S-shaped transfer functions, identifying the best transfer function for the dataset. The second version integrates SMA with two-phase mutation to improve SMA’s exploitation ability. The third version is integrated with a novel attacking-feeding strategy to improve both the exploitation and the exploitation properties of the technique. In the final version, both two-phase mutation and attacking-feeding strategy are combined with the SMA. To check the classification accuracy of the extracted features, the k-nearest neighbors (KNN) algorithm is used; this algorithm is classified as one of the common classification and regression algorithms in machine learning. In addition, the four versions are validated using 28 well-known datasets taken from the UCI repository, and compared with a number of well-known existing algorithms. The experimental outcomes show the superiority of the FMBSMA version relative to the other three versions and six state-of-art feature selection algorithms. Finally, the main contributions of this work are as follows:

• Proposing four versions of the standard SMA as binary versions for tackling FS problems

• Investigating the performance of the standard SMA with V-Shaped and S-shaped transfer functions

• Proposing a new strategy called the Attacking-Feeding strategy (AF) to explore and exploit more solutions based on the previous status of each particle.

• After comparing the four versions with the other algorithms, the hybrid version that integrates the binary version of SMA (BSMA) with two-phase mutation (TM), and AF is shown to outperform all other algorithms used in the comparison.

The rest of this paper is organized as follows: Section 1 briefly reviews some of the works proposed for tackling FS, Section 2 describes the SMA, Section 3 explains and discusses the four proposed versions of BSMA which are compared with other algorithms in Section 4. Section 5 draws some conclusions and suggests future work.