Skip to main content
SpringerLink
Log in

A comparison of several nearest neighbor classifier metrics using Tabu Search algorithm for the feature selection problem

  • Original Paper
  • Published:
Optimization Letters Aims and scope Submit manuscript

Abstract

The feature selection problem is an interesting and important topic which is relevant for a variety of database applications. This paper utilizes the Tabu Search metaheuristic algorithm to implement a feature subset selection procedure while the nearest neighbor classification method is used for the classification task. Tabu Search is a general metaheuristic procedure that is used in order to guide the search to obtain good solutions in complex solution spaces. Several metrics are used in the nearest neighbor classification method, such as the euclidean distance, the Standardized Euclidean distance, the Mahalanobis distance, the City block metric, the Cosine distance and the Correlation distance, in order to identify the most significant metric for the nearest neighbor classifier. The performance of the proposed algorithms is tested using various benchmark datasets from UCI Machine Learning Repository.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Aha D.W. and Bankert R.L. (1996). A comparative evaluation of sequential feature selection algorithms. In: Fisher, D. and Lenx, J.-H. (eds) Artificial Intelligence and Statistics, pp. Springer, New York

    Google Scholar 

  2. Cantu-Paz, E.: Feature Subset Selection, Class Separability, and Genetic Algorithms. Genetic and Evolutionary Computation Conference, pp. 959–970 (2004)

  3. Cantu-Paz, E., Newsam, S., Kamath, C.: Feature selection in scientific application. In: Proceedings of the 2004 ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 788–793 (2004)

  4. Duda R.O. and Hart P.E. (1973). Pattern Classification and Scene Analysis. Wiley, New York

    MATH  Google Scholar 

  5. Duda R.O., Hart P.E. and Stork D.G. (2001). Pattern Classification and Scene Analysis. Wiley, New York

    Google Scholar 

  6. Gendreau M. (2003). An introduction to Tabu Search. In: Glover, F. and Kochenberger, G.A. (eds) Handbook of Metaheuristics., pp 37–54. Kluwer, Dordrecht

    Chapter  Google Scholar 

  7. Giudici P. (2003). Applied Data Mining: Statistical Methods for Business and Industry. Wiley, Chichester

    MATH  Google Scholar 

  8. Glover F. (1989). Tabu Search I. ORSA J. Computing. 1(3): 190–206

    MATH  Google Scholar 

  9. Glover F. (1990). Tabu Search II. ORSA J. Computing. 2(1): 4–32

    MATH  Google Scholar 

  10. Glover, F., Laguna, M., Taillard, E., de Werra, D. (eds.): Tabu Search. J.C. Baltzer AG, Science Publishers, Basel, Switzerland (1993)

  11. Hastie T., Tibshirani R. and Friedman J. (2001). The Elements of Statistical Learning; Data mining, Inference, and Prediction. Springer Series in Statistics. Springer, New York

    Google Scholar 

  12. Jain A. and Zongker D. (1997). Feature selection: evaluation, application and small sample performance. IEEE Trans. Patt. Anal. Mach. Intell. 19: 153–158

    Article  Google Scholar 

  13. Kira, K., Rendell, L.: A practical approach to feature selection. In: Proceedings of the Ninth International Conference on Machine Learning, Aberdeen, Scotland, pp. 249–256 (1992)

  14. Kohavi R. and John G. (1997). Wrappers for feature subset selection. Artif. Intell. 97: 273–324

    Article  MATH  Google Scholar 

  15. Lopez F.G., Torres M.G., Batista B.M., Perez J.A.M. and Moreno-Vega J.M. (2006). Solving feature subset selection problem by a parallel scatter search.. Euro. J. Oper. Res. 169: 477–489

    Article  MATH  Google Scholar 

  16. Narendra P.M. and Fukunaga K. (1977). A branch and bound algorithm for feature subset selection. IEEE Trans. Comput. 26(9): 917–922

    Article  MATH  Google Scholar 

  17. Pudil P., Novovicova J. and Kittler J. (1994). Floating search methods in feature selection. Patt. Recognit. Lett. 15: 1119–1125

    Article  Google Scholar 

  18. Rokach L. and Maimon O. (2005). Clustering Methods. In: Maimon, O. and Rokach, L. (eds) Data Mining and Knowledge Discovery Handbook, pp 321–352. Springer, New York

    Chapter  Google Scholar 

  19. Siedlecki W. and Sklansky J. (1988). On automatic feature selection. Int. J. Patt. Recognit. Artif. Intell. 2(2): 197–220

    Article  Google Scholar 

  20. Siedlecki W. and Sklansky J. (1989). A note on genetic algorithms for large-scale feature selection. Patt. Recognit. Lett. 10: 335–347

    Article  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Industrial Systems Control Laboratory, Department of Production Engineering and Management, Technical University of Crete, 73100, Chania, Greece

    Magdalene Marinaki

  2. Decision Support Systems Laboratory, Department of Production Engineering and Management, Technical University of Crete, 73100, Chania, Greece

    Yannis Marinakis & Nikolaos Matsatsinis

  3. Financial Engineering Laboratory, Department of Production Engineering and Management, Technical University of Crete, 73100, Chania, Greece

    Michael Doumpos & Constantin Zopounidis

Authors
  1. Magdalene Marinaki
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yannis Marinakis
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Michael Doumpos
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nikolaos Matsatsinis
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Constantin Zopounidis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Constantin Zopounidis.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Marinaki, M., Marinakis, Y., Doumpos, M. et al. A comparison of several nearest neighbor classifier metrics using Tabu Search algorithm for the feature selection problem. Optimization Letters 2, 299–308 (2008). https://doi.org/10.1007/s11590-007-0057-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11590-007-0057-2

Keywords

Search

Navigation