Skip to main content
SpringerLink
Log in

Incorporating neighbors’ distribution knowledge into support vector machines

  • Methodologies and Application
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

The prior knowledge plays an important role in increasing the performance of the support vector machines (SVMs). Traditional SVMs do not consider any prior knowledge of the training set. In this paper, the neighbors’ distribution knowledge is incorporated into SVMs. The neighbors’ distribution can be measured by the sum of the cosine value of the angle, which is between the difference between the sample and its corresponding neighbor, and the difference between the sample and the mean of corresponding neighbors. The neighbors’ distribution knowledge reflects the sample’s importance in the training processing. It can be explained as the relative margin or instance weight. In this paper, the neighbors’ distribution knowledge is regarded as the relative margin and incorporated into the framework of density-induced margin support vector machines whose relative margin is measured by relative density degree. The results of the experiments, performed on both artificial synthetic datasets and real-world benchmark datasets, demonstrate that SVMs performs better after incorporating neighbors’ distribution. Furthermore, experimental results also show that neighbors’ distribution are more suitable than relative density degree to represent the relative margin.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  • Alcala-Fdez J, Sanchez L, Garcia S, del Jesus MJ, Ventura S, Garrell JM, Otero CR, Bacardit J, Rivas VM, Fernández JC, Herrera F (2009) KEEL: a software tool to assess evolutionary algorithms for data mining problems. Soft Comput 13(3):307–318

    Article  Google Scholar 

  • Alcalá J, Fernández A, Luengo J, Derrac J, García S, Sánchez L, Herrera F (2010) Keel data-mining software tool: Data set repository, integration of algorithms and experimental analysis framework. J Multiple-Valued Logic Soft Comput 17(2–3):255–287

    Google Scholar 

  • Bertelli L, Yu T, Vu D, Gokturk, B (2011) Kernelized structural SVMlearning for supervised object segmentation. In: 2011 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, Milpitas, pp 2153–2160

  • Bottou L (2010) Large-scale machine learning with stochastic gradient descent. In: Proceedings of COMPSTAT’2010. Physica-Verlag HD, pp 177–186

  • Chang CC, Lin CJ (2011) LIBSVM: A library for support vector machines. ACM Trans Intell Syst Technol 2(3):27

    Article  Google Scholar 

  • Chapelle O, Schölkopf B (2001) Incorporating invariances in non-linear support vector machines. In: Advances in neural information processing systems. MIT Press, Vancouver, pp 609–616

  • Cristianini N, Shawe-Taylor J (2000) An introduction to support vector machines and other kernel-based learning methods. Cambridge University Press, Cambridge

    Book  MATH  Google Scholar 

  • Friedman JH, Bentley JL, Finkel RA (1977) An algorithm for finding best matches in logarithmic expected time. ACM Trans Math Softw 3(3):209–226

    Article  MATH  Google Scholar 

  • García S, Fernández A, Luengo J, Herrera F (2010) Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: experimental analysis of power. Inf Sci 180(10):2044–2064

    Article  Google Scholar 

  • Gumus E, Kilic N, Sertbas A, Ucan ON (2010) Evaluation of face recognition techniques using PCA, wavelets and SVM. Expert Syst Appl 37(9):6404–6408

    Article  Google Scholar 

  • Hsieh CJ, Chang KW, Lin CJ, Keerthi SS, Sundararajan S (2008) A dual coordinate descent method for large-scale linear SVM. In: Proceedings of the 25th international conference on machine learning. ACM, Helsinki, pp 408–415

  • Hua X, Ding S (2015) Weighted least squares projection twin support vector machines with local information. Neurocomputing 160:228–237

    Article  Google Scholar 

  • Hwang JP, Park S, Kim E (2011) A new weighted approach to imbalanced data classification problem via support vector machine with quadratic cost function. Expert Syst Appl 38(7):8580–8585

    Article  Google Scholar 

  • Karasuyama M, Harada N, Sugiyama M, Takeuchi I (2012) Multi-parametric solution-path algorithm for instance-weighted support vector machines. Mach Learn 88(3):297–330

    Article  MathSciNet  MATH  Google Scholar 

  • Khemchandani R, Chandra S (2007) Twin support vector machines for pattern classification. IEEE Trans Pattern Anal Mach Intell 29(5):905–910

    Article  MATH  Google Scholar 

  • Kruskal WH, Wallis WA (1952) Use of ranks in one-criterion variance analysis. J Am Stat Assoc 47(260):583–621

    Article  MATH  Google Scholar 

  • Lauer F, Bloch G (2008) Incorporating prior knowledge in support vector machines for classification: a review. Neurocomputing 71(7):1578–1594

    Article  Google Scholar 

  • Lee K, Kim DW, Lee D, Lee KH (2005) Improving support vector data description using local density degree. Pattern Recogn 38(10):1768–1771

    Article  MATH  Google Scholar 

  • Lichman M (2013) UCI machine learning repository. http://archive.ics.uci.edu/ml. University of California, School of Information and Computer Science, Irvine, CA

  • Mangasarian OL, Wild EW (2001) Proximal support vector machine classifiers. In: Proceedings KDD-2001: knowledge discovery and data mining. ACM, San Francisco

  • Mangasarian OL, Wild EW (2006) Multisurface proximal support vector machine classification via generalized eigenvalues. IEEE Trans Pattern Anal Mach Intell 28(1):69–74

    Article  Google Scholar 

  • Niu XX, Suen CY (2012) A novel hybrid CNN-SVM classifier for recognizing handwritten digits. Pattern Recogn 45(4):1318–1325

  • Palmieri F, Fiore U, Castiglione A, De Santis A (2013) On the detection of card-sharing traffic through wavelet analysis and support vector machines. Appl Soft Comput 13(1):615–627

    Article  Google Scholar 

  • Palmieri F, Fiore U, Castiglione A (2014) A distributed approach to network anomaly detection based on independent component analysis. Concurr Comput Pract Exp 26(5):1113–1129

    Article  Google Scholar 

  • Shao YH, Chen WJ, Zhang JJ, Wang Z, Deng NY (2014) An efficient weighted Lagrangian twin support vector machine for imbalanced data classification. Pattern Recogn 47(9):3158–3167

    Article  MATH  Google Scholar 

  • Tian J, Gu H, Liu W, Gao C (2011) Robust prediction of protein subcellular localization combining PCA and WSVMs. Comput Biol Med 41(8):648–652

    Article  Google Scholar 

  • Vapnik V (2013) The nature of statistical learning theory. Springer, Berlin

    MATH  Google Scholar 

  • Vapnik V, Vashist A (2009) A new learning paradigm: learning using privileged information. Neural Netw 22(5):544–557

    Article  MATH  Google Scholar 

  • Wang F, Zhang D (2013) A new locality-preserving canonical correlation analysis algorithm for multi-view dimensionality reduction. Neural Process Lett 37(2):135–146

    Article  Google Scholar 

  • Wang J, Shen HT, Song J, Ji J (2014) Hashing for similarity search: a survey. arXiv preprint arXiv:1408.2927

  • Wu X, Srihari R (2004) Incorporating prior knowledge with weighted margIn support vector machines. In: Proceedings of the tenth ACM SIGKDD international conference on knowledge discovery and data mining. ACM, Seattle, pp 326–333

  • Wu Y, Wei B, Liu H, Li T, Rayner S (2011) MiRPara: a SVM-based software tool for prediction of most probable microRNA coding regions in genome scale sequences. BMC Bioinform 12(1):107

    Article  Google Scholar 

  • Xiong T, Cherkassky V (2005) A combined SVM and LDA approach for classification. In: 2005 IEEE international joint conference on neural networks, 2005. IJCNN’05. Proceedings, vol 3. IEEE, Montreal, pp 1455–1459

  • Yang X, Song Q, Wang Y (2007) A weighted support vector machine for data classification. Int J Pattern Recogn Artif Intell 21(05):961–976

    Article  Google Scholar 

  • Yang X, Chen S, Chen B, Pan Z (2009) Proximal support vector machine using local information. Neurocomputing 73(1):357–365

    Article  Google Scholar 

  • Zafeiriou S, Tefas A, Pitas I (2007) Minimum class variance support vector machines. IEEE Trans Image Process 16(10):2551–2564

    Article  MathSciNet  Google Scholar 

  • Zhang L, Zhou WD (2011) Density-induced margin support vector machines. Pattern Recogn 44(7):1448–1460

    Article  MATH  Google Scholar 

  • Zhang H, Cao L, Gao S (2014) A locality correlation preserving support vector machine. Pattern Recogn 47(9):3168–3178

    Article  MATH  Google Scholar 

  • Zhu F, Ye N, Yu W, Xu S, Li G (2014a) Boundary detection and sample reduction for one-class support vector machines. Neurocomputing 123:166–173

    Article  Google Scholar 

  • Zhu F, Yang J, Ye N, Gao C, Li G, Yin T (2014b) Neighbors’ distribution property and sample reduction for support vector machines. Appl Soft Comput 16:201–209

    Article  Google Scholar 

Download references

Acknowledgments

The authors would like to thank the editor and the anonymous reviewers for their critical and constructive comments and suggestions. This work was partially supported by the National Science Fund for Distinguished Young Scholars under Grant Nos. 61125305, 61472187, 61233011 and 61373063, the Key Project of Chinese Ministry of Education under Grant No. 313030, the 973 Program (No. 2014CB349303), Fundamental Research Funds for the Central Universities (No. 30920140121005), Program for Changjiang Scholars and Innovative Research Team in University No. IRT13072, National Basic Research Program of China (973 Program) (2012CB114505), China National Funds for Distinguished Young Scientists (31125008).

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, People’s Republic of China

    Fa Zhu & Jian Yang

  2. Department of Geomatics Engineering, University of Calgary, Calgary, Canada

    Sheng Xu

  3. Department of Computer Science, University of Regina, Regina, Canada

    Cong Gao

  4. College of Information Science and Technology, Nanjing Forestry University, Nanjing, People’s Republic of China

    Ning Ye

  5. College of Forest Resources and Environment, Nanjing Forestry University, Nanjing, People’s Republic of China

    Tongming Yin

Authors
  1. Fa Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jian Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sheng Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Cong Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ning Ye
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Tongming Yin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fa Zhu.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest to this work.

Additional information

Communicated by V. Loia.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhu, F., Yang, J., Xu, S. et al. Incorporating neighbors’ distribution knowledge into support vector machines. Soft Comput 21, 6407–6420 (2017). https://doi.org/10.1007/s00500-016-2199-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-016-2199-6

Keywords

Search

Navigation