Bing Xue
ABSTRACT
Feature selection (FS) is an important data preprocessing technique, which has two goals of minimising the classification error and minimising the number of features selected. Based on particle swarm optimisation (PSO), this paper proposes two multi-objective algorithms for selecting the Pareto front of non-dominated solutions (feature subsets) for classification. The first algorithm introduces the idea of non-dominated sorting based multi-objective genetic algorithm II into PSO for FS. In the second algorithm, multi-objective PSO uses the ideas of crowding, mutation and dominance to search for the Pareto front solutions. The two algorithms are compared with two single objective FS methods and a conventional FS method on nine datasets. Experimental results show that both proposed algorithms can automatically evolve a smaller number of features and achieve better classification performance than using all features and feature subsets obtained from the two single objective methods and the conventional method. Both the continuous and the binary versions of PSO are investigated in the two proposed algorithms and the results show that continuous version generally achieves better performance than the binary version. The second new algorithm outperforms the first algorithm in both continuous and binary versions.
- H. Almuallim and T. G. Dietterich. Learning boolean concepts in the presence of many irrelevant features. Artificial Intelligence, 69:279--305, 1994. Google Scholar
Digital Library
- G. Azevedo, G. Cavalcanti, and E. Filho. An approach to feature selection for keystroke dynamics systems based on pso and feature weighting. In IEEE Congress on Evolutionary Computation (CEC'07), pages 3577--3584, 2007.Google ScholarCross Ref
- B. Chakraborty. Genetic algorithm with fuzzy fitness function for feature selection. In IEEE International Symposium on Industrial Electronics (ISIE'02), volume 1, pages 315--319, 2002.Google ScholarCross Ref
- B. Chakraborty. Feature subset selection by particle swarm optimization with fuzzy fitness function. In 3rd International Conference on Intelligent System and Knowledge Engineering (ISKE'08), volume 1, pages 1038--1042, 2008.Google ScholarCross Ref
- M. Dash and H. Liu. Feature selection for classification. Intelligent Data Analysis, 1(4):131--156, 1997.Google ScholarCross Ref
- K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan. A fast and elitist multiobjective genetic algorithm: Nsga-ii. IEEE Transactions on Evolutionary Computation, 6(2):182--197, 2002. Google ScholarDigital Library
- A. Frank and A. Asuncion. UCI machine learning repository, 2010.Google Scholar
- M. Gutlein, E. Frank, M. Hall, and A. Karwath. Large-scale attribute selection using wrappers. In IEEE Symposium on Computational Intelligence and Data Mining (CIDM '09), pages 332--339, 2009.Google ScholarCross Ref
- C. L. Huang and J. F. Dun. A distributed pso-svm hybrid system with feature selection and parameter optimization. Application on Soft Computing, 8:1381--1391, 2008. Google ScholarDigital Library
- J. Kennedy and R. Eberhart. Particle swarm optimization. In IEEE International Conference on Neural Networks, volume 4, pages 1942--1948, 1995.Google ScholarCross Ref
- J. Kennedy and R. Eberhart. A discrete binary version of the particle swarm algorithm. In IEEE International Conference on Systems, Man, and Cybernetics, 1997. Computational Cybernetics and Simulation., volume 5, pages 4104--4108, 1997.Google ScholarCross Ref
- J. Kennedy and W. Spears. Matching algorithms to problems: an experimental test of the particle swarm and some genetic algorithms on the multimodal problem generator. In IEEE Congress on Evolutionary Computation (CEC'98), pages 78--83, 1998.Google ScholarCross Ref
- K. Kira and L. A. Rendell. A practical approach to feature selection. Assorted Conferences and Workshops, pages 249--256, 1992. Google ScholarDigital Library
- R. Kohavi and G. H. John. Wrappers for feature subset selection. Artificial Intelligence, 97:273--324, 1997. Google ScholarDigital Library
- X. Li. A non-dominated sorting particle swarm optimizer for multiobjective optimization. In GECCO, pages 37--48, 2003. Google ScholarDigital Library
- Y. Liu, G. Wang, H. Chen, and H. Dong. An improved particle swarm optimization for feature selection. Journal of Bionic Engineering, 8(2):191--200, 2011.Google ScholarCross Ref
- T. Marill and D. Green. On the effectiveness of receptors in recognition systems. IEEE Transactions on Information Theory, 9(1):11--17, 1963. Google ScholarDigital Library
- H. Ming. A rough set based hybrid method to feature selection. In International Symposium on Knowledge Acquisition and Modeling (KAM '08), pages 585--588, 2008. Google ScholarDigital Library
- A. Mohemmed, M. Zhang, and M. Johnston. Particle swarm optimization based adaboost for face detection. In IEEE Congress on Evolutionary Computation (CEC'09), pages 2494--2501, 2009. Google ScholarDigital Library
- K. Neshatian and M. Zhang. Dimensionality reduction in face detection: A genetic programming approach. In 24th International Conference Image and Vision Computing New Zealand (IVCNZ'09), pages 391--396, 2009.Google ScholarCross Ref
- Y. Shi and R. Eberhart. A modified particle swarm optimizer. In IEEE International Conference on Evolutionary Computation (CEC'98), pages 69--73, 1998.Google ScholarCross Ref
- M. R. Sierra and C. A. C. Coello. Improving pso-based multi-objective optimization using crowding, mutation and epsilon-dominance. In EMO, pages 505--519, 2005. Google ScholarDigital Library
- A. Unler and A. Murat. A discrete particle swarm optimization method for feature selection in binary classification problems. European Journal of Operational Research, 206(3):528--539, 2010.Google ScholarCross Ref
- F. Van Den Bergh. An analysis of particle swarm optimizers. PhD thesis, Pretoria, South Africa, 2002. Google ScholarDigital Library
- A. Whitney. A direct method of nonparametric measurement selection. IEEE Transactions on Computers, C-20(9):1100--1103, 1971. Google ScholarDigital Library
- C. S. Yang, L. Y. Chuang, C. H. Ke, and C. H. Yang. Boolean binary particle swarm optimization for feature selection. In IEEE Congress on Evolutionary Computation (CEC'08), pages 2093--2098, 2008.Google ScholarCross Ref
- H. Yuan, S. S. Tseng, and W. Gangshan. A two-phase feature selection method using both filter and wrapper. In IEEE International Conference on Systems, Man, and Cybernetics (SMC'99), volume 2, pages 132--136, 1999.Google Scholar
Index Terms
- Multi-objective particle swarm optimisation (PSO) for feature selection
Recommendations
A multi-objective particle swarm optimisation for filter-based feature selection in classification problems
Feature selection has the two main objectives of minimising the classification error rate and the number of features. Based on binary particle swarm optimisation BPSO, we develop two novel multi-objective feature selection frameworks for classification, ...
A particle swarm optimisation based multi-objective filter approach to feature selection for classification
PRICAI'12: Proceedings of the 12th Pacific Rim international conference on Trends in Artificial IntelligenceFeature selection (FS) has two main objectives of minimising the number of features and maximising the classification performance. Based on binary particle swarm optimisation (BPSO), we develop a multi-objective FS framework for classification, which is ...
Barebones particle swarm for multi-objective optimisation problems
Control parameters, inertia weight and acceleration coefficients influence strongly performance of multi-objective particle swarm optimisation (MOPSO) algorithms. To eliminate the need for tuning of these parameters for different optimisation problems, ...
Comments