Skip to main content
SpringerLink
Log in

A fast prototype reduction method based on template reduction and visualization-induced self-organizing map for nearest neighbor algorithm

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

The k nearest neighbor is a lazy learning algorithm that is inefficient in the classification phase because it needs to compare the query sample with all training samples. A template reduction method is recently proposed that uses only samples near the decision boundary for classification and removes those far from the decision boundary. However, when class distributions overlap, more border samples are retrained and it leads to inefficient performance in the classification phase. Because the number of reduced samples are limited, using an appropriate feature reduction method seems a logical choice to improve classification time. This paper proposes a new prototype reduction method for the k nearest neighbor algorithm, and it is based on template reduction and ViSOM. The potential property of ViSOM is displaying the topology of data on a two-dimensional feature map, it provides an intuitive way for users to observe and analyze data. An efficient classification framework is then presented, which combines the feature reduction method and the prototype selection algorithm. It needs a very small data size for classification while keeping recognition rate. In the experiments, both of synthetic and real datasets are used to evaluate the performance. Experimental results demonstrate that the proposed method obtains above 70 % speedup ratio and 90 % compression ratio while maintaining similar performance to kNN.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Similar content being viewed by others

References

  1. Hart PE, Stock DG, Duda RO (2001) Pattern classification, 2nd edn. Wiley, Hoboken

    MATH  Google Scholar 

  2. Mahmoud SA, Al-Khatib WG (2011) Recognition of Arabic (Indian) bank check digits using log-Gabor filters. Appl Intell 35(3):445–456

    Article  Google Scholar 

  3. Zhao L, Wang L, Xu Q (2012) Data stream classification with artificial endocrine system. Appl Intell 37(3):390–404

    Article  Google Scholar 

  4. Malek H, Ebadzadeh MM, Rahmati M (2012) Three new fuzzy neural networks learning algorithms based on clustering, training error and genetic algorithm. Appl Intell 37(3):280–289

    Article  Google Scholar 

  5. Chen Y, Garcia E, Gupta M, Rahimi A, Cazzanti L (2009) Similarity based classification: concepts and algorithms. J Mach Learn Res 10:747–776

    MathSciNet  MATH  Google Scholar 

  6. Domeniconi C, Jing P, Gunopulos D (2002) Locally adaptive metric nearest neighbor classification. IEEE Trans Pattern Anal Mach Intell 24(9):1281–1285

    Article  Google Scholar 

  7. Paredes R, Vidal E (2006) Leaning weighted metric to minimize nearest-neighbor classification error. IEEE Trans Pattern Anal Mach Intell 28(7):1100–1110

    Article  Google Scholar 

  8. Wang J, Neskovic P, Cooper L (2007) Improving nearest neighbor rule with a simple adaptive distance measure. Pattern Recognit Lett 28(2):207–213

    Article  Google Scholar 

  9. Li BY, Chen YW, Chen YQ (2008) The nearest neighbor algorithm of local probability centers. IEEE Trans Syst Man Cybern, Part B, Cybern 38(1):141–154

    Article  Google Scholar 

  10. Hart PE (1968) The condensed nearest neighbor rule. IEEE Trans Inf Theory 14(3):515–516

    Article  Google Scholar 

  11. Wilson DR, Martinez TR (2000) Reduction techniques for instance-based learning algorithms. Mach Learn 38(3):257–286

    Article  MATH  Google Scholar 

  12. Angiulli F (2007) Fast nearest neighbor condensation for large data sets classification. IEEE Trans Knowl Data Eng 19(11):1450–1464

    Article  Google Scholar 

  13. Pekalska E, Duin RPW, Paclık P (2006) Prototype selection for dissimilarity based classifiers. Pattern Recognit 39(2):189–208

    Article  MATH  Google Scholar 

  14. Wu Y, Ianakiev K, Govindaraju V (2002) Improved k-nearest neighbor classification. Pattern Recognit 35:2311–2318

    Article  MATH  Google Scholar 

  15. Chang CL (1974) Finding prototypes for nearest neighbor classifiers. IEEE Trans Comput C-23(11):1179–1184

    Article  Google Scholar 

  16. Veenman CJ, Reinders MJT (2005) The nearest subclass classifier: a compromise between the nearest mean and nearest neighbor classifier. IEEE Trans Pattern Anal Mach Intell 27(9):1417–1429

    Article  Google Scholar 

  17. Kohonen T (1990) The self-organizing map. Proc IEEE 78(9):1464–1480

    Article  Google Scholar 

  18. Lam W, Keung CK, Liu D (2002) Discovering useful concept prototypes for classification based on filtering and abstraction. IEEE Trans Pattern Anal Mach Intell 14(8):1075–1090

    Article  Google Scholar 

  19. Nanni L, Lumini A (2008) Particle swarm optimization for prototype reduction. Neurocomputing 72(4–6):1092–1097

    Google Scholar 

  20. Garcia S, Derrac J, Cano JR, Herrera F (2012) Prototype selection for nearest neighbor classification: taxonomy and empirical study. IEEE Trans Pattern Anal Mach Intell 34(3):417–435

    Article  Google Scholar 

  21. Triguero I, Derrac J, Garcıa S, Herrera F (2012) A taxonomy and experimental study on prototype generation for nearest neighbor classification. IEEE Trans Syst Man Cybern, Part C, Appl Rev 42(1):86–100

    Article  Google Scholar 

  22. Fayed HA, Atiya AF (2009) A novel template reduction approach for the k-nearest neighbor method. IEEE Trans Neural Netw 20(5):890–896

    Article  Google Scholar 

  23. Bohlooli A, Jamshidi K (2012) A GPS-free method for vehicle future movement directions prediction using SOM for VANET. Appl Intell 36(3):685–697

    Article  Google Scholar 

  24. Wu J, Li IJ (2010) A SOM-based dimensionality reduction method for KNN classifiers. In: International Conference on System Science and Engineering, pp 173–178

    Google Scholar 

  25. Yin H (2002) ViSOM-a novel method for multivariate data projection and structure visualization. IEEE Trans Neural Netw 13(1)

  26. Wilson DL (1972) Asymptotic properties of nearest neighbor rules using edited data. IEEE Trans Syst Man Cybern 2(3):408–421

    Article  MATH  Google Scholar 

  27. Tomek I (1976) An experiment with the edited nearest-neighbor rule. IEEE Trans Syst Man Cybern 6(6):448–452

    Article  MathSciNet  MATH  Google Scholar 

  28. Sanchez JS, Barandela R, Marques AI, Alejo R, Badenas J (2003) Analysis of new techniques to obtain quality training sets. Pattern Recognit Lett 24(7):1015–1022

    Article  Google Scholar 

  29. Aha DW, Kibler D, Albert MK (1991) Instance-based learning algorithms. Mach Learn 6(1):37–66

    Google Scholar 

  30. Ho SY, Liu CC, Liu S (2002) Design of an optimal nearest neighbor classifier using an intelligent genetic algorithm. Pattern Recognit Lett 23(13):1495–1503

    Article  MATH  Google Scholar 

  31. Garcia S, Cano JR, Herrera F (2008) A memetic algorithm for evolutionary prototype selection: a scaling up approach. Pattern Recognit 41(8):2693–2709

    Article  MATH  Google Scholar 

  32. Gates W (1972) The reduced nearest neighbor rule. IEEE Trans Inf Theory 18(3):431–433

    Article  Google Scholar 

  33. Devi FS, Murty MN (2002) An incremental prototype set building technique. Pattern Recognit 35(2):505–513

    Article  MATH  Google Scholar 

  34. Berglund E (2010) Improved PLSOM algorithm. Appl Intell 32(1):122–130

    Article  Google Scholar 

  35. Kamimura R (2011) Structural enhanced information and its application to improved visualization of self-organizing maps. Appl Intell 34(1):102–115

    Article  MathSciNet  Google Scholar 

  36. Theodoridis S, Koutroumbas K (2006) Pattern recognition, 3rd edn. Academic Press, San Diego

    MATH  Google Scholar 

  37. Blake C, Keogh E, Merz CJ (2009) UCI repository of machine learning databases. Department of Information and Computer Science, University of California. http://www.ics.uci.edu/~mlearn

  38. Kohonen T (1993) Things you haven’t heard about the self-organizing map. In: IEEE International Conference on Neural Networks, vol 3, pp 1147–pages 1156

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Dept. of Computer Science and Engineering, National Chung Hsing University, 250, Kuo Kuang Rd., 402, Taichung, Taiwan

    I-Jing Li, Jia-Chian Chen & Jiunn-Lin Wu

Authors
  1. I-Jing Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jia-Chian Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jiunn-Lin Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jiunn-Lin Wu.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Li, IJ., Chen, JC. & Wu, JL. A fast prototype reduction method based on template reduction and visualization-induced self-organizing map for nearest neighbor algorithm. Appl Intell 39, 564–582 (2013). https://doi.org/10.1007/s10489-013-0433-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-013-0433-9

Keywords

Search

Navigation