Skip to main content
SpringerLink
Log in

A particle swarm optimization based multiobjective memetic algorithm for high-dimensional feature selection

  • Regular research paper
  • Published:
Memetic Computing Aims and scope Submit manuscript

Abstract

Feature selection, as one of the dimension reduction methods, is a crucial processing step in dealing with high-dimensional data. It tries to preserve feature subset representing the whole feature space, which aims to reduce redundancy and increase the classification accuracy. Since the two objectives are usually in conflict with each other, feature selection is modeled as a multi-objective problem. However, the high search space and discrete Pareto front makes it not easy for existing evolutionary multiobjective algorithms. Classic evolutionary computation method, which is often applied to feature selection problem straightforwardly, gradually exposes its inefficiency in searching process. Hence, a particle swarm optimization based multiobjective memetic algorithm for high-dimensional feature selection is designed in this paper to deal with above shortcomings. Its basic idea is to model feature selection as a multiobjective optimization problem by optimizing the number of features and the classification accuracy in supervised condition simultaneously, in which information entropy based initialization and adaptive local search are designed to improve the search efficiency. Moreover, a new particle velocity update rule considering both convergence and diversity of solutions is designed to update particles, and a fast discrete nondominated sorting strategy is designed to rank the Pareto solutions. These strategies enable the proposed algorithm to gain better performance on both the quality and size of feature subset. The experimental results show that the proposed algorithm can improve the quality of Pareto fronts evolved by the state-of-the-art algorithms for feature selection.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Chuang LY, Chang HW, Tu CJ, Yang CH (2008) Improved binary PSO for feature selection using gene expression data. Comput Biol Chem 32(1):29–38

    Article  MATH  Google Scholar 

  2. Deb K, Jain H (2014) An evolutionary many-objective optimization algorithm using reference-point-based nondominated sorting approach, part i: Solving problems with box constraints. IEEE Trans Evol Comput 18(4):577–601

    Article  Google Scholar 

  3. Deb K, Pratap A, Agarwal S, Meyarivan T (2002) A fast and elitist multiobjective genetic algorithm: Nsga-ii. IEEE Trans Evol Comput 6(2):182–197

    Article  Google Scholar 

  4. Demir K, Nguyen BH, Xue B, Zhang M (2020) A decomposition based multi-objective evolutionary algorithm with relieff based local search and solution repair mechanism for feature selection. In: 2020 IEEE congress on evolutionary computation (CEC)

  5. Emmanouilidis C, Hunter A, MacIntyre, J (2000) A multiobjective evolutionary setting for feature selection and a commonality-based crossover operator. In: Proceedings of the 2000 congress on evolutionary computation. CEC00 (Cat. No. 00TH8512), vol 1. IEEE, pp 309–316

  6. Emmanouilidis C, Hunter A, Macintyre J (2002) A multiobjective evolutionary setting for feature selection and a commonality-based crossover operator. In: Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512)

  7. Hamdani TM, Won JM, Alimi AM, Karray F (2007) Multi-objective feature selection with nsga ii. In: International conference on adaptive and natural computing algorithms, Springer, pp 240–247

  8. Han L, Wang H (2021) A random forest assisted evolutionary algorithm using competitive neighborhood search for expensive constrained combinatorial optimization. Memetic Comput 13:19–30

    Article  Google Scholar 

  9. Hou Y, Ong YS, Feng L, Zurada JM (2017) An evolutionary transfer reinforcement learning framework for multiagent systems. IEEE Trans Evol Comput 21(4):601–615. https://doi.org/10.1109/TEVC.2017.2664665

    Article  Google Scholar 

  10. Hu Y, Zhang Y, Gong D (2021) Multiobjective particle swarm optimization for feature selection with fuzzy cost. IEEE Trans Cybern 51(2):874–888

    Article  Google Scholar 

  11. Huang CL, Dun JF (2008) A distributed PSO-SVM hybrid system with feature selection and parameter optimization. Appl Soft Comput 8(4):1381–1391

    Article  Google Scholar 

  12. Kannan SS, Ramaraj N (2010) A novel hybrid feature selection via symmetrical uncertainty ranking based local memetic search algorithm. Knowl-Based Syst 23(6):580–585

    Article  Google Scholar 

  13. Kennedy J, Eberhart R (2002) Particle swarm optimization. In: ICNN95-international Conference on Neural Networks

  14. Li H, He F, Chen Y, Pan Y (2021) MLFS-CCDE: multi-objective large-scale feature selection by cooperative coevolutionary differential evolution. Memetic Comput 13:1–18

    Article  Google Scholar 

  15. Luo J, Jiao L, Liu F, Yang S, Ma W (2019) A pareto-based sparse subspace learning framework. IEEE Trans Cybern 49:3859–3872

    Article  Google Scholar 

  16. Li AD, Xue B, Zhang M (2020) Multi-objective feature selection using hybridization of a genetic algorithm and direct multisearch for key quality characteristic selection. Inf Sci 523:245–265

    Article  MathSciNet  Google Scholar 

  17. Marill T, Green DM (1963) On the effectiveness of receptors in recognition systems. Inf Theory IEEE Trans 9(1):11–17

    Article  Google Scholar 

  18. Moradi P, Gholampour M (2016) A hybrid particle swarm optimization for feature subset selection by integrating a novel local search strategy. Appl Soft Comput 43(C):117–130

  19. Nekkaa M, Boughaci D (2015) A memetic algorithm with support vector machine for feature selection and classification. Memetic Comput 7(1):59–73

    Article  Google Scholar 

  20. Nguyen BH, Xue B, Andreae P, Ishibuchi H, Zhang M (2019) Multiple reference points-based decomposition for multiobjective feature selection in classification: static and dynamic mechanisms. IEEE Trans Evol Comput 24(1):170–184

    Article  Google Scholar 

  21. Nguyen HB, Xue B, Liu I, Andreae P, Zhang M (2015) Gaussian transformation based representation in particle swarm optimisation for feature selection. In: European Conference on the Applications of Evolutionary Computation, Springer, pp 541–553

  22. Shang R, Wang W, Stolkin R, Jiao L (2018) Non-negative spectral learning and sparse regression-based dual-graph regularized feature selection. IEEE Trans Cybern 48(2):793–806

    Article  Google Scholar 

  23. Song Q, Ni J, Wang G (2011) A fast clustering-based feature subset selection algorithm for high-dimensional data. IEEE Trans Knowl Data Eng 25(1):1–14

    Article  Google Scholar 

  24. Storn R, Price K (1997) Differential evolution: a simple and efficient heuristic for global optimization over continuous spaces. J Global Optim 11:341–359

    Article  MathSciNet  MATH  Google Scholar 

  25. Tan KC, Feng L, Jiang M (2021) Evolutionary transfer optimization—a new frontier in evolutionary computation research. IEEE Comput Intell Mag 16(1):22–33. https://doi.org/10.1109/MCI.2020.3039066

    Article  Google Scholar 

  26. Tian J, Li M, Chen F, Feng N (2016) Learning subspace-based RBFNN using coevolutionary algorithm for complex classification tasks. IEEE Trans Neural Netw Learn Syst 27(1):47–61

    Article  MathSciNet  Google Scholar 

  27. Tian Y, Zhang X, Wang C, Jin Y (2020) An evolutionary algorithm for large-scale sparse multiobjective optimization problems. IEEE Trans Evol Comput 24(2):380–393

    Article  Google Scholar 

  28. Tran B, Xue B, Zhang M (2018) A new representation in PSO for discretization-based feature selection. IEEE Trans Cybern 48(6):1733–1746

    Article  Google Scholar 

  29. Tran B, Xue B, Zhang M (2019) Variable-length particle swarm optimization for feature selection on high-dimensional classification. IEEE Trans Evol Comput 23(3):473–487

    Article  Google Scholar 

  30. Tubishat M, Idris N, Shuib L, Abushariah MAM, Mirjalili S (2019) Improved Salp swarm algorithm based on opposition based learning and novel local search algorithm for feature selection. Expert Syst Appl 145:113122

    Article  Google Scholar 

  31. The Memetic Automaton (2019): The Mainspring of Knowledge Transfer in a Data-Driven Optimization Era. Memetic Computation

  32. Wang Z, Zhang Q, Zhou A, Gong M, Jiao L (2016) Adaptive replacement strategies for moea/d. IEEE Trans Cybern 46(2):474–486

    Article  Google Scholar 

  33. Whitney AW (1971) A direct method of nonparametric measurement selection. IEEE Trans Comput 20(9):1100–1103

    Article  MATH  Google Scholar 

  34. Wu X, Xu X, Liu J, Wang H, Nie F (2020) Supervised feature selection with orthogonal regression and feature weighting. IEEE Trans Neural Netw Learn Syst PP(99):2–8

  35. Xue B, Cervante L, Shang L, Browne WN, Zhang M (2014) Binary PSO and rough set theory for feature selection: a multi-objective filter based approach. Int J Comput Intell Appl 13(02):1450009

    Article  Google Scholar 

  36. Xue B, Zhang M, Browne WN (2012) Multi-objective particle swarm optimisation (pso) for feature selection. In: Proceedings of the 14th annual conference on Genetic and evolutionary computation, pp 81–88

  37. Xue B, Zhang M, Browne WN (2012) New fitness functions in binary particle swarm optimisation for feature selection. In: 2012 IEEE congress on evolutionary computation, pp 1–8. IEEE

  38. Xue B, Zhang M, Browne WN (2012) Particle swarm optimization for feature selection in classification: a multi-objective approach. IEEE Trans Cybern 43(6):1656–1671

    Article  Google Scholar 

  39. Xue B, Zhang M, Browne WN (2014) Particle swarm optimisation for feature selection in classification: novel initialisation and updating mechanisms. Appl Soft Comput J 18:261–276

  40. Xue B, Zhang M, Browne WN, Yao X (2016) A survey on evolutionary computation approaches to feature selection. IEEE Trans Evol Comput 20(4):606–626

    Article  Google Scholar 

  41. Xue Y, Xue B, Zhang M (2019) Self-adaptive particle swarm optimization for large-scale feature selection in classification. ACM Trans Knowl Discov Data (TKDD) 13(5):1–27

    Article  Google Scholar 

  42. Zhang Q, Hui L (2008) Moea/d: a multiobjective evolutionary algorithm based on decomposition. IEEE Trans Evolut Comput 11(6):712–731

    Article  Google Scholar 

  43. Zhang Y, Dw Gong, Xz Gao, Tian T, Xy Sun (2020) Binary differential evolution with self-learning for multi-objective feature selection. Inf Sci 507:67–85

    Article  MathSciNet  Google Scholar 

  44. Zhou Y, Kang J, Kwong S, Wang X, Zhang Q (2021) An evolutionary multi-objective optimization framework of discretization-based feature selection for classification. Swarm Evolut Comput 60:100770

    Article  Google Scholar 

  45. Zhu L, Cao L, Yang J (2012) Multiobjective evolutionary algorithm-based soft subspace clustering. In: 2012 IEEE Congress on evolutionary computation, IEEE, pp 1–8

  46. Zhu Z, Ong YS, Dash M (2007) Wrappercfilter feature selection algorithm using a memetic framework. IEEE Trans Syst Man Cybern Part B (Cybernetics) 37(1):70–76

    Article  Google Scholar 

  47. Zitzler E, Laumanns M, Thiele L (2001) Spea2: Improving the strength pareto evolutionary algorithm. Technical Report Gloriastrasse

Download references

Author information

Authors and Affiliations

  1. Beijing Key Laboratory of Intelligent Telecommunication Software and Multimedia, Beijing University of Posts and Telecommunications, Beijing, China

    Juanjuan Luo, Lingling Jiang & Huadong Ma

  2. Northern Institute of Electronic Equipment of China, Beijing, China

    Dongqing Zhou

Authors
  1. Juanjuan Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dongqing Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lingling Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Huadong Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Juanjuan Luo.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work is supported in part by the National Key R&D Program of China under Grant 2018AAA0101201, in part by the National Natural Science Foundation of China under Grant 61806019, and in part by the Fundamental Research Funds for the Central Universities.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Luo, J., Zhou, D., Jiang, L. et al. A particle swarm optimization based multiobjective memetic algorithm for high-dimensional feature selection. Memetic Comp. 14, 77–93 (2022). https://doi.org/10.1007/s12293-022-00354-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12293-022-00354-z

Keywords

Search

Navigation