Skip to main content
SpringerLink
Log in

New mechanism for archive maintenance in PSO-based multi-objective feature selection

  • Focus
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

In classification problems, a large number of features are typically used to describe the problem’s instances. However, not all of these features are useful for classification. Feature selection is usually an important pre-processing step to overcome the problem of “curse of dimensionality”. Feature selection aims to choose a small number of features to achieve similar or better classification performance than using all features. This paper presents a particle swarm Optimization (PSO)-based multi-objective feature selection approach to evolving a set of non-dominated feature subsets which achieve high classification performance. The proposed algorithm uses local search techniques to improve a Pareto front and is compared with a pure multi-objective PSO algorithm, three well-known evolutionary multi-objective algorithms and a current state-of-the-art PSO-based multi-objective feature selection approach. Their performances are examined on 12 benchmark datasets. The experimental results show that in most cases, the proposed multi-objective algorithm generates better Pareto fronts than all other methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  • Asuncion A, Newman D (2007) Uci machine learning repository

  • Bhowan U, McCloskey D (2015) Genetic programming for feature selection and question-answer ranking in ibm watson. In: Genetic Programming. Springer, New York, pp 153–166

  • Bin W, Qinke P, Jing Z, Xiao C (2012) A binary particle swarm optimization algorithm inspired by multi-level organizational learning behavior. Eur J Oper Res 219(2):224–233

    Article  MathSciNet  MATH  Google Scholar 

  • Boubezoul A, Paris S (2012) Application of global optimization methods to model and feature selection. Pattern Recogn 45(10):3676–3686

    Article  MATH  Google Scholar 

  • Cervante L, Xue B, Zhang M, Shang L (2012) Binary particle swarm optimisation for feature selection: a filter based approach. In: 2012 IEEE Congress on Evolutionary Computation (CEC), pp 1–8

  • Chaaraoui AA, Flórez-Revuelta F (2013) Human action recognition optimization based on evolutionary feature subset selection. In: Proceedings of the 15th annual conference on Genetic and evolutionary computation (GECCO). ACM, pp 1229–1236

  • Chakraborty B (2002) Genetic algorithm with fuzzy fitness function for feature selection. In: Proceedings of the 2002 IEEE International Symposium on Industrial Electronics (ISIE), vol 1, pp 315–319

  • Chakraborty B, Chakraborty G (2013) Fuzzy consistency measure with particle swarm optimization for feature selection. In: 2013 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp 4311–4315

  • Chuang LY, Chang HW, Tu CJ, Yang CH (2008) Improved binary pso for feature selection using gene expression data. Comput Biol Chem 32(1):29–38

    Article  MATH  Google Scholar 

  • Dash M, Liu H (1997) Feature selection for classification. Intell Data Anal 1(3):131–156

    Article  Google Scholar 

  • Deb K, Agrawal S, Pratap A, Meyarivan T (2000) A fast elitist non-dominated sorting genetic algorithm for multi-objective optimization: Nsga-ii. Lect Notes Comput Sci 1917:849–858

    Article  Google Scholar 

  • Gheyas IA, Smith LS (2010) Feature subset selection in large dimensionality domains. Pattern Recogn 43(1):5–13

    Article  MATH  Google Scholar 

  • Guyon I, Elisseeff A (2003) An introduction to variable and feature selection. J Mach Learn Res 3:1157–1182

    MATH  Google Scholar 

  • Hamdani TM, Won JM, Alimi AM, Karray F (2007) Multi-objective feature selection with nsga ii. In: Adaptive and Natural Computing Algorithms. Springer, New York, pp 240–247

  • Hancer E, Xue B, Karaboga D, Zhang M (2015) A binary abc algorithm based on advanced similarity scheme for feature selection. Appl Soft Comput 36:334–348

    Article  Google Scholar 

  • Huang CL, Dun JF (2008) A distributed pso-svm hybrid system with feature selection and parameter optimization. Appl Soft Comput 8(4):1381–1391

    Article  Google Scholar 

  • Huang CL, Wang CJ (2006) A ga-based feature selection and parameters optimizationfor support vector machines. Expert Syst Appl 31(2):231–240

    Article  Google Scholar 

  • Kennedy J, Eberhart R et al (1995) Particle swarm optimization. Proc IEEE Int Conf Neural Netw 4:1942–1948

    Article  Google Scholar 

  • Knowles J, Corne D (1999) The pareto archived evolution strategy: A new baseline algorithm for pareto multiobjective optimisation. In: Congress on Evolutionary Computation, vol 1. IEEE

  • Lane MC, Xue B, Liu I, Zhang M (2013) Particle swarm optimisation and statistical clustering for feature selection. In: AI 2013: Advances in Artificial Intelligence. Springer, New York, pp 214–220

  • Lane MC, Xue B, Liu I, Zhang M (2014) Gaussian based particle swarm optimisation and statistical clustering for feature selection. In: Evolutionary computation in combinatorial optimisation. Springer, New York, pp 133–144

  • Lee S, Soak S, Oh S, Pedrycz W, Jeon M (2008) Modified binary particle swarm optimization. Prog Nat Sci 18(9):1161–1166

    Article  MathSciNet  Google Scholar 

  • Li X (2003) A non-dominated sorting particle swarm optimizer for multiobjective optimization. In: Proceedings of the 5th annual conference on Genetic and Evolutionary Computation (GECCO). Springer, New York, pp 37–48

  • Liang D, Tsai CF, Wu HT (2015) The effect of feature selection on financial distress prediction. Knowl-Based Syst 73:289–297

    Article  Google Scholar 

  • Lin F, Liang D, Yeh CC, Huang JC (2014) Novel feature selection methods to financial distress prediction. Expert Syst Appl 41(5):2472–2483

    Article  Google Scholar 

  • Lin SW, Ying KC, Chen SC, Lee ZJ (2008) Particle swarm optimization for parameter determination and feature selection of support vector machines. Expert Syst Appl 35(4):1817–1824

    Article  Google Scholar 

  • Liu Y, Wang G, Chen H, Dong H, Zhu X, Wang S (2011) An improved particle swarm optimization for feature selection. J Bionic Eng 8(2):191–200

    Article  Google Scholar 

  • Marill T, Green DM (1963) On the effectiveness of receptors in recognition systems. IEEE Trans Inf Theory 9(1):11–17

    Article  Google Scholar 

  • Mohemmed AW, Zhang M, Johnston M (2009) Particle swarm optimization based adaboost for face detection. In: IEEE Congress on Evolutionary Computation (CEC), pp 2494–2501

  • Neshatian K, Zhang M (2009a) Dimensionality reduction in face detection: a genetic programming approach. In: IEEE 24th International Conference on Image and Vision Computing New Zealand (IVCNZ’09), pp 391–396

  • Neshatian K, Zhang M (2009b) Genetic programming for feature subset ranking in binary classification problems. In: Genetic programming. Springer, New York, pp 121–132

  • Nguyen H, Xue B, Liu I, Zhang M (2014a) Filter based backward elimination in wrapper based pso for feature selection in classification. In: 2014 IEEE Congress on Evolutionary Computation (CEC), pp 3111–3118

  • Nguyen HB, Xue B, Liu I, Zhang M (2014b) Pso and statistical clustering for feature selection: a new representation. In: Simulated evolution and learning. Springer, New York, pp 569–581

  • Nguyen HB, Xue B, Liu I, Andreae P, Zhang M (2015) Gaussian transformation based representation in particle swarm optimisation for feature selection. In: Applications of evolutionary computation. Springer, New York, pp 541–553

  • Oreski S, Oreski G (2014) Genetic algorithm-based heuristic for feature selection in credit risk assessment. Expert Syst Appl 41(4):2052–2064

    Article  Google Scholar 

  • Pudil P, Novovičová J, Kittler J (1994) Floating search methods in feature selection. Pattern Recogn Lett 15(11):1119–1125

    Article  Google Scholar 

  • Seo JH, Lee YH, Kim YH (2014) Feature selection for very short-term heavy rainfall prediction using evolutionary computation. Adv Meteorol 2014:203545. doi:10.1155/2014/203545

  • Stearns SD (1976) On selecting features for pattern classifiers. In: Proceedings of the 3rd International Conference on Pattern Recognition (ICPR 1976), pp 71–75

  • Tran B, Xue B, Zhang M (2014) Improved pso for feature selection on high-dimensional datasets. In: Simulated evolution and learning. Springer, New York, pp 503–515

  • Unler A, Murat A (2010) A discrete particle swarm optimization method for feature selection in binary classification problems. Eur J Oper Res 206(3):528–539

    Article  MATH  Google Scholar 

  • Van Den Bergh F (2006) An analysis of particle swarm optimizers. Ph.D. thesis, University of Pretoria

  • Vieira SM, Mendonça LF, Farinha GJ, Sousa JM (2013) Modified binary pso for feature selection using svm applied to mortality prediction of septic patients. Appl Soft Comput 13(8):3494–3504

    Article  Google Scholar 

  • Wang L (2005) A hybrid genetic algorithm-neural network strategy for simulation optimization. Appl Math Comput 170(2):1329–1343

    MathSciNet  MATH  Google Scholar 

  • Wang X, Yang J, Teng X, Xia W, Jensen R (2007) Feature selection based on rough sets and particle swarm optimization. Pattern Recogn Lett 28(4):459–471

    Article  Google Scholar 

  • Whitney AW (1971) A direct method of nonparametric measurement selection. IEEE Trans Comput 100(9):1100–1103

    Article  MathSciNet  MATH  Google Scholar 

  • Xue B, Cervante L, Shang L, Browne WN, Zhang M (2012a) A multi-objective particle swarm optimisation for filter-based feature selection in classification problems. Connect Sci 2–3:91–116

    Article  Google Scholar 

  • Xue B, Zhang M, Browne W (2012b) New fitness functions in binary particle swarm optimisation for feature selection. In: 2012 IEEE Congress on Evolutionary Computation (CEC), pp 2145–2152

  • Xue B, Zhang M, Browne WN (2013) Particle swarm optimization for feature selection in classification: a multi-objective approach. IEEE Trans Cybern 43(6):1656–1671

    Article  Google Scholar 

  • Xue B, Zhang M, Browne WN (2014) Particle swarm optimisation for feature selection in classification: novel initialisation and updating mechanisms. Appl Soft Comput 18:261–276

    Article  Google Scholar 

  • Xue B, Zhang M, Browne W, Yao X (2015a) A survey on evolutionary computation approaches to feature selection. IEEE Trans Evol Comput (99):1–1. doi:10.1109/TEVC.2015.2504420

  • Xue B, Zhang M, Browne WN (2015b) A comprehensive comparison on evolutionary feature selection approaches to classification. Int J Comput Intell Appl 14(02):1550008

  • Yang CS, Chuang LY, Ke CH, Yang CH (2008) Boolean binary particle swarm optimization for feature selection. In: 2008 IEEE Congress on Evolutionary Computation (CEC), pp 2093–2098

  • Yuan H, Tseng SS, Gangshan W, Fuyan Z (1999) A two-phase feature selection method using both filter and wrapper. In: 1999 IEEE International Conference on Systems, Man, and Cybernetics (SMC), vol 2, pp 132–136

  • Zhang Y, Gong D, Hu Y, Zhang W (2015) Feature selection algorithm based on bare bones particle swarm optimization. Neurocomputing 148:150–157

    Article  Google Scholar 

  • Zhao H, Sinha AP, Ge W (2009) Effects of feature construction on classification performance: an empirical study in bank failure prediction. Expert Syst Appl 36(2):2633–2644

  • Zhu Z, Ong YS, Dash M (2007) Wrapper-filter feature selection algorithm using a memetic framework. IEEE Trans Syst Man Cybern 37(1):70–76

  • Zitzler E, Laumanns M, Thiele L (2001) Spea 2: improving the strength pareto evolutionary algorithm

Download references

Author information

Authors and Affiliations

  1. School of Engineering and Computer Science, Victoria University of Wellington, Wellington, New Zealand

    Hoai Bach Nguyen, Bing Xue, Peter Andreae & Mengjie Zhang

  2. School of Mathematics and Statistics, Victoria University of Wellington, PO Box 600, Wellington, 6140, New Zealand

    Ivy Liu

Authors
  1. Hoai Bach Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bing Xue
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ivy Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Peter Andreae
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mengjie Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hoai Bach Nguyen.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Communicated by B. Xue and A.G. Chen.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nguyen, H.B., Xue, B., Liu, I. et al. New mechanism for archive maintenance in PSO-based multi-objective feature selection. Soft Comput 20, 3927–3946 (2016). https://doi.org/10.1007/s00500-016-2128-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-016-2128-8

Keywords

Search

Navigation