Skip to main content
SpringerLink
Log in

Particle swarm optimization based on filter-based population initialization method for feature selection in classification

  • Original Research
  • Published:
Journal of Ambient Intelligence and Humanized Computing Aims and scope Submit manuscript

Abstract

In classification problems, the datasets often have many unrelated or redundant features. The unrelated or redundant features may deteriorate the performance of classifier. Feature selection (FS) is an effective approach to solve this kind of problem. In the past research, Particle Swarm Optimization (PSO) has already been used for the FS problems, and some researchers have designed various methods to improve the PSO to efficiently solve the problems. For one thing, it has been verified that a good initial population can significantly enhance the performance of PSO and increase the convergence speed of PSO. For another, lots of traditional filter methods which can provide heuristic information about features have been proposed. However, few studies increase the performance of PSO for FS by improving its initialization method based on the filter methods. Therefore, in this paper, Relief, Information gain and Fisher score are introduced to initialize the population of PSO on FS problems. In the proposed method, for each filter method, it is first employed to evaluate and rank the features. Then, we assign a probability value to each feature according to its rank. After that, the initial population is generated based on the probabilities of each feature. Therefore, three initialization populations are separately obtained by the three filter methods. Finally, they are merged into the final initialization population. The comparative experiments are conducted on nine datasets and the PSO with proposed initialization method is compared with the PSO with other two initialization methods. The results indicate that the proposed initialization method can greatly enhance the search ability and increase the convergence speed of PSO for solving FS problems, especially large scale FS problems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  • Amini F, Hu G (2021) A two-layer feature selection method using genetic algorithm and elastic net. Expert Syst Appl 166(114):072

    Google Scholar 

  • Arora S, Singh S (2019) Butterfly optimization algorithm: a novel approach for global optimization. Soft Comput 23(3):715–734

    Article  Google Scholar 

  • Bansal M, Kumar M, Sachdeva M et al (2021) Transfer learning for image classification using vgg19: Caltech-101 image data set. J Ambient Intell Humaniz Comput 2:1–12

    Google Scholar 

  • Chen K, Zhou F, Wang Y et al (2018) An ameliorated particle swarm optimizer for solving numerical optimization problems. Appl Soft Comput 73:482–496

    Article  Google Scholar 

  • Dash M, Liu H (1997) Feature selection for classification. Intell Data Anal 1(1–4):131–156

    Article  Google Scholar 

  • Davahli A, Shamsi M, Abaei G (2020) Hybridizing genetic algorithm and grey wolf optimizer to advance an intelligent and lightweight intrusion detection system for iot wireless networks. J Ambient Intell Humaniz Comput 11(11):5581–5609

    Article  Google Scholar 

  • Drewek-Ossowicka A, Pietrołaj M, Rumiński J (2021) A survey of neural networks usage for intrusion detection systems. J Ambient Intell Humaniz Comput 12(1):497–514

    Article  Google Scholar 

  • Dutang C, Wuertz D (2009) A note on random number generation. Overview of Random Generation Algorithms p 2

  • Gao W, Liu S, Huang L (2012) Particle swarm optimization with chaotic opposition-based population initialization and stochastic search technique. Commun Nonlinear Sci Numer Simul 17(11):4316–4327

    Article  MathSciNet  MATH  Google Scholar 

  • Hemavathi D, Srimathi H (2021) Effective feature selection technique in an integrated environment using enhanced principal component analysis. J Ambient Intell Humaniz Comput 12(3):3679–3688

    Article  Google Scholar 

  • Hu P, Pan JS, Chu SC et al (2022) Multi-surrogate assisted binary particle swarm optimization algorithm and its application for feature selection. Appl Soft Comput 121(108):736

    Google Scholar 

  • Ibrahim RA, Ewees AA, Oliva D et al (2019) Improved salp swarm algorithm based on particle swarm optimization for feature selection. J Ambient Intell Humaniz Comput 10(8):3155–3169

    Article  Google Scholar 

  • Jabeen H, Jalil Z, Baig AR (2009) Opposition based initialization in particle swarm optimization (O-PSO). In: Proceedings of the 11th Annual Conference Companion on Genetic and Evolutionary Computation Conference: Late Breaking Papers, ACM, pp 2047–2052

  • Kawamura A, Chakraborty B (2017) A hybrid approach for optimal feature subset selection with evolutionary algorithms. In: 2017 IEEE 8th International Conference on Awareness Science and Technology (iCAST), IEEE, pp 564–568

  • Kazimipour B, Li X, Qin AK (2014) A review of population initialization techniques for evolutionary algorithms. In: 2014 IEEE Congress on Evolutionary Computation (CEC), IEEE, pp 2585–2592

  • Kohler M, Vellasco MM, Tanscheit R (2019) Pso+: a new particle swarm optimization algorithm for constrained problems. Appl Soft Comput 85(105):865

    Google Scholar 

  • Li H, Li SJ, Shang J et al (2019) A dynamic scale-free network particle swarm optimization for extracting features on multi-omics data. J Comput Biol 26(8):769–781

    Article  Google Scholar 

  • Li AD, Xue B, Zhang M (2021) Improved binary particle swarm optimization for feature selection with new initialization and search space reduction strategies. Appl Soft Comput 106(107):302

    Google Scholar 

  • Luo J, Zhou D, Jiang L et al (2022) A particle swarm optimization based multiobjective memetic algorithm for high-dimensional feature selection. Memetic Comput 14(1):77–93

    Article  Google Scholar 

  • Mlakar U, Fister I, Brest J et al (2017) Multi-objective differential evolution for feature selection in facial expression recognition systems. Expert Syst Appl 89:129–137

    Article  Google Scholar 

  • Moslehi F, Haeri A (2020) An evolutionary computation-based approach for feature selection. J Ambient Intell Humaniz Comput 11(9):3757–3769

    Article  Google Scholar 

  • Nguyen BH, Xue B, Andreae P et al (2019) Multiple reference points-based decomposition for multiobjective feature selection in classification: Static and dynamic mechanisms. IEEE Trans Evol Comput 24(1):170–184

    Article  Google Scholar 

  • Parsopoulos K, Vrahatis M (2002) Initializing the particle swarm optimizer using the nonlinear simplex method. Adv Intell Syst Fuzzy Syst Evol Comput 216:1–6

    Google Scholar 

  • Rahnamayan S, Tizhoosh H, Salama MM (2007) A novel population initialization method for accelerating evolutionary algorithms. Comput Math Appl 53(10):1605–1614

    Article  MathSciNet  MATH  Google Scholar 

  • Rashno A, Shafipour M, Fadaei S (2022) Particle ranking: An efficient method for multi-objective particle swarm optimization feature selection. Knowl-Based Syst 245(108):640

    Google Scholar 

  • Sayed GI, Darwish A, Hassanien AE (2018) A new chaotic whale optimization algorithm for features selection. J Classif 35(2):300–344

    Article  MathSciNet  MATH  Google Scholar 

  • Shafipour M, Rashno A, Fadaei S (2021) Particle distance rank feature selection by particle swarm optimization. Expert Syst Appl 185(115):620

    Google Scholar 

  • Song X, Zhang Y, Gong D et al (2022) Surrogate sample-assisted particle swarm optimization for feature selection on high-dimensional data. IEEE Trans Evol Comput 2:2

    Google Scholar 

  • Wang H, Wu Z, Wang J, et al (2009) A new population initialization method based on space transformation search. In: 2009 Fifth International Conference on Natural Computation, IEEE, pp 332–336

  • Xue B, Zhang M, Browne W (2014) Particle swarm optimisation for feature selection in classification: novel initialisation and updating mechanisms. Appl Soft Comput 18:261–276

    Article  Google Scholar 

  • Xue B, Zhang M, Browne WN et al (2015) A survey on evolutionary computation approaches to feature selection. IEEE Trans Evol Comput 20(4):606–626

    Article  Google Scholar 

  • Xue Y, Xue B, Zhang M (2019) Self-adaptive particle swarm optimization for large-scale feature selection in classification. ACM Trans Knowl Discov Data (TKDD) 13(5):50

    Google Scholar 

  • Xue Y, Cai X, Neri F (2022) A multi-objective evolutionary algorithm with interval based initialization and self-adaptive crossover operator for large-scale feature selection in classification. Appl Soft Comput 2:109420

    Article  Google Scholar 

  • Zainudin M, Sulaiman M, Mustapha N et al (2017) Feature selection optimization using hybrid relief-f with self-adaptive differential evolution. Int J Intell Eng Syst 10(3):21–29

    Google Scholar 

  • Zhang Y, Gong D, Hu Y et al (2015) Feature selection algorithm based on bare bones particle swarm optimization. Neurocomputing 148:150–157

    Article  Google Scholar 

  • Zhang Y, Dw G, Cheng J (2017) Multi-objective particle swarm optimization approach for cost-based feature selection in classification. IEEE/ACM Trans Comput Biol Bioinform (TCBB) 14(1):64–75

    Article  Google Scholar 

  • Zhou X, Wang X, Dougherty ER (2005) Gene selection using logistic regressions based on aic, bic and mdl criteria. New Math Nat Comput 1(01):129–145

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (61876089, 61876185, 61902281, 61395121), the Key Research and Development Program of Jiangsu Province (BE2020633), and the Priority Academic Program Development of Jiangsu Higher Education Institutions.

Author information

Authors and Affiliations

  1. School of Computer and Software, Nanjing University of Information Science and Technology, Nanjing, 210044, China

    Yu Xue, Xu Cai & Weiwei Jia

Authors
  1. Yu Xue
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xu Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Weiwei Jia
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yu Xue.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xue, Y., Cai, X. & Jia, W. Particle swarm optimization based on filter-based population initialization method for feature selection in classification. J Ambient Intell Human Comput 14, 7355–7366 (2023). https://doi.org/10.1007/s12652-022-04444-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12652-022-04444-1

Keywords

Search

Navigation