Skip to main content
SpringerLink
Log in
Book cover

Asia-Pacific Conference on Simulated Evolution and Learning

SEAL 2014: Simulated Evolution and Learning pp 503–515Cite as

Improved PSO for Feature Selection on High-Dimensional Datasets

  • Conference paper

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 8886))

Abstract

Classification on high-dimensional (i.e. thousands of dimensions) data typically requires feature selection (FS) as a pre-processing step to reduce the dimensionality. However, FS is a challenging task even on datasets with hundreds of features. This paper proposes a new particle swarm optimisation (PSO) based FS approach to classification problems with thousands or tens of thousands of features. The proposed algorithm is examined and compared with three other PSO based methods on five high-dimensional problems of varying difficulty. The results show that the proposed algorithm can successfully select a much smaller number of features and significantly increase the classification accuracy over using all features. The proposed algorithm outperforms the other three PSO methods in terms of both the classification performance and the number of features. Meanwhile, the proposed algorithm is computationally more efficient than the other three PSO methods because it selects a smaller number of features and employs a new fitness evaluation strategy.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Ambroise, C., McLachlan, G.J.: Selection bias in gene extraction on the basis of microarray gene-expression data. Proceedings of the National Academy of Sciences 99(10), 6562–6566 (2002)

    Article  MATH  Google Scholar 

  2. Bala, J., Huang, J., Vafaie, H., Dejong, K., Wechsler, H.: Hybrid learning using genetic algorithms and decision trees for pattern classification. In: The 14th International Joint Conference on Artificial Intelligence, vol. 1

    Google Scholar 

  3. Chakraborty, B.: Genetic algorithm with fuzzy fitness function for feature selection. In: IEEE International Symposium on Industrial Electronics (ISIE 2002), vol. 1, pp. 315–319 (2002)

    Google Scholar 

  4. Chuang, L.Y., Chang, H.W., Tu, C.J., Yang, C.H.: Improved binary pso for feature selection using gene expression data. Computational Biology and Chemistry 32(1), 29–38 (2008)

    Article  MATH  Google Scholar 

  5. Dash, M., Liu, H.: Feature selection for classification. Intelligent Data Analysis 1, 131–156 (1997)

    Article  Google Scholar 

  6. Davis, R.A., Charlton, A.J., Oehlschlager, S., Wilson, J.C.: Novel feature selection method for genetic programming using metabolomic 1h NMR data. Chemometrics and Intelligent Laboratory Systems 81(1), 50–59 (2006)

    Article  Google Scholar 

  7. Dorigo, M., Di Caro, G.: Ant colony optimization: a new meta-heuristic. In: IEEE Congress on Evolutionary Computation, vol. 2, pp. 1470–1477 (1999)

    Google Scholar 

  8. Engelbrecht, A.P.: Computational intelligence: an introduction, 2nd edn. Wiley (2007)

    Google Scholar 

  9. Guyon, I., Elisseeff, A.: An introduction to variable and feature selection. J. Mach. Learn. Res. 3, 1157–1182 (2003)

    MATH  Google Scholar 

  10. Huang, J., Cai, Y., Xu, X.: A hybrid genetic algorithm for feature selection wrapper based on mutual information. Pattern Recognition Letters 28(13), 1825–1844 (2007)

    Article  Google Scholar 

  11. Jensen, R., Shen, Q.: Finding rough set reducts with ant colony optimization. In: Proceedings of the 2003 UK Workshop on Computational Intelligence, pp. 15–22 (2003)

    Google Scholar 

  12. Kanan, H.R., Faez, K.: An improved feature selection method based on ant colony optimization (ACO) evaluated on face recognition system. Applied Mathematics and Computation 205(2), 716–725 (2008), Special Issue on Advanced Intelligent Computing Theory and Methodology in Applied Mathematics and Computation

    Google Scholar 

  13. Kennedy, J., Eberhart, R.: Particle swarm optimization. In: IEEE International Conference on Neural Networks, vol. 4, pp. 1942–1948 (1995)

    Google Scholar 

  14. Kohavi, R., John, G.H.: Wrappers for feature subset selection. Artificial Intelligence 97(1-2), 273–324 (1997), relevance

    Article  MATH  Google Scholar 

  15. Lane, M.C., Xue, B., Liu, I., Zhang, M.: Gaussian based particle swarm optimisation and statistical clustering for feature selection. In: Blum, C., Ochoa, G. (eds.) EvoCOP 2014. LNCS, vol. 8600, pp. 133–144. Springer, Heidelberg (2014)

    Chapter  Google Scholar 

  16. Lanzi, P.L.: Fast feature selection with genetic algorithms: a filter approach. In: IEEE International Conference on Evolutionary Computation, pp. 537–540 (1997)

    Google Scholar 

  17. Marill, T., Green, D.M.: On the effectiveness of receptors in recognition systems. IEEE Transactions on Information Theory 9(1), 11–17 (1963)

    Article  Google Scholar 

  18. Ming, H.: A rough set based hybrid method to feature selection. In: International Symposium on Knowledge Acquisition and Modeling, KAM 2008, pp. 585–588 (December 2008)

    Google Scholar 

  19. Neshatian, K., Zhang, M.: Pareto front feature selection: Using genetic programming to explore feature space. In: The 11th Annual Conference on Genetic and Evolutionary Computation, GECCO 2009, pp. 1027–1034 (2009)

    Google Scholar 

  20. Oliveira, L., Sabourin, R., Bortolozzi, F., Suen, C.: Feature selection using multi-objective genetic algorithms for handwritten digit recognition. In: 16th International Conference on Pattern Recognition (ICPR 2002), vol. 1, pp. 568–571 (2002)

    Google Scholar 

  21. Pudil, P., Novovičová, J., Kittler, J.: Floating search methods in feature selection. Pattern Recogn. Lett. 15(11), 1119–1125 (1994)

    Article  Google Scholar 

  22. Shi, Y., Eberhart, R.: A modified particle swarm optimizer. In: IEEE International Conference on Evolutionary Computation (CEC 1998), pp. 69–73 (1998)

    Google Scholar 

  23. Shi, Y., Eberhart, R.: Empirical study of particle swarm optimization. In: IEEE Congress on Evolutionary Computation (CEC 1999), vol. 3, pp. 1945–1950 (1999)

    Google Scholar 

  24. Stearns, S.D.: On selecting features for pattern classifiers. In: Proceedings of the 3rd International Conference on Pattern Recognition (ICPR 1976), Coronado, CA, pp. 71–75 (1976)

    Google Scholar 

  25. Whitney, A.: A direct method of nonparametric measurement selection. IEEE Transactions on Computers C-20(9), 1100–1103 (1971)

    Article  MathSciNet  Google Scholar 

  26. Xue, B.: Particle Swarm Optimisation for Feature Selection in Classification. Ph.D. thesis, Victoria University of Wellington, Wellington, New Zealand (2014)

    Google Scholar 

  27. Xue, B., Cervante, L., Shang, L., Browne, W.N., Zhang, M.: A multi-objective particle swarm optimisation for filter based feature selection in classification problems. Connection Science 24(2-3), 91–116 (2012)

    Article  Google Scholar 

  28. Xue, B., Cervante, L., Shang, L., Browne, W.N., Zhang, M.: Binary PSO and rough set theory for feature selection: A multi-objective filter based approach. International Journal of Computational Intelligence and Applications 13(02), 1450009 (2014)

    Article  Google Scholar 

  29. Xue, B., Zhang, M., Browne, W.N.: Particle swarm optimization for feature selection in classification: A multi-objective approach. IEEE Transactions on Cybernetics 43(6), 1656–1671 (2013)

    Article  Google Scholar 

  30. Xue, B., Zhang, M., Browne, W.N.: Particle swarm optimisation for feature selection in classification: Novel initialisation and updating mechanisms. Applied Soft Computing 18, 261–276 (2014)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Victoria University of Wellington, PO Box 600, Wellington, 6140, New Zealand

    Binh Tran, Bing Xue & Mengjie Zhang

Authors
  1. Binh Tran
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bing Xue
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mengjie Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Otago University, Dunedin, New Zealand

    Grant Dick

  2. Victoria University of Welling, New Zealand

    Will N. Browne

  3. University of Otago, Dunedin, New Zealand

    Peter Whigham

  4. Unitec Institute of Technology, Victoria University of Wellington, New Zealand

    Mengjie Zhang

  5. Le Quy Don Technical University, Hanoi, Vietnam

    Lam Thu Bui

  6. Department of Computer Science and Intelligent Systems, Graduate School of Engineering, Osaka Prefecture University, 1-1 Gakuen-cho, Naka-ku, 599-8531, Sakai, Osaka, Japan

    Hisao Ishibuchi

  7. Department of Computing, University of Surrey, GU2 7XH, Guildford, Surrey, UK

    Yaochu Jin

  8. RMIT University, Melbourne, Australia

    Xiaodong Li

  9. Department of Electrical & Electronic Engineering, Xi’an Jiaotong-Liverpool University, Suzhou, China

    Yuhui Shi

  10. Indian Institute of Information Technology and Management, Gwalior, India

    Pramod Singh

  11. Department of Electrical and Computer Engineering, National University of Singapore, 4 Engineering Drive 3, 117576, Singapore, Singapore

    Kay Chen Tan

  12. USTC-Birmingham Joint Research Institute in Intelligent Computation and Its Applications (UBRI), School of Computer Science and Technology, University of Science and Technology of China, 230027, Hefei, China

    Ke Tang

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer International Publishing Switzerland

About this paper

Cite this paper

Tran, B., Xue, B., Zhang, M. (2014). Improved PSO for Feature Selection on High-Dimensional Datasets. In: Dick, G., et al. Simulated Evolution and Learning. SEAL 2014. Lecture Notes in Computer Science, vol 8886. Springer, Cham. https://doi.org/10.1007/978-3-319-13563-2_43

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-13563-2_43

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-13562-5

  • Online ISBN: 978-3-319-13563-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation