Skip to main content
SpringerLink
Log in

K-nearest neighbor-based weighted twin support vector regression

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Twin support vector regression (TSVR) finds ϵ-insensitive up- and down-bound functions by resolving a pair of smaller-sized quadratic programming problems (QPPs) rather than a single large one as in a classical SVR, which makes its computational speed greatly improved. However the local information among samples are not exploited in TSVR. To make full use of the knowledge of samples and improve the prediction accuracy, a K-nearest neighbor-based weighted TSVR (KNNWTSVR) is proposed in this paper, where the local information among samples are utilized. Specifically, a major weight is given to the training sample if it has more K-nearest neighbors. Otherwise a minor weight is given to it. Moreover, to further enhance the computational speed, successive overrelaxation approach is employed to resolve the QPPs. Experimental results on eight benchmark datasets and a real dataset demonstrate our weighted TSVR not only yields lower prediction error but also costs lower running time in comparison with other algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Notes

  1. http://archive.ics.uci.edu/ml/datasets.html.

References

  1. Cai D, He X, Zhang W, Han J (2007) Regularized locality preserving indexing via spectral regression. In: Proc 2007 ACM int. conf. on information and knowledge management

    Google Scholar 

  2. Cheng H, Tan P, Jin R (2010) Efficient algorithm for localized support vector machine. IEEE Trans Knowl Data Eng 22(4):537–549

    Article  Google Scholar 

  3. Cover T, Hart P (1967) Nearest neighbor pattern classification. IEEE Trans Inf Theory 13:21–27

    Article  MATH  Google Scholar 

  4. Diosan L, Rogozan A, Pecuchet J-P (2013) Improving classification performance of support vector machine by genetically optimising kernel shape and hyper-parameters. Appl Intell 36:280–294

    Article  Google Scholar 

  5. Fung G, Mangasarian O (2001) Proximal support vector machine classifiers. In: Seven international proceedings on knowledge discovery and data mining, pp 77–86

    Google Scholar 

  6. Fung G, Mangasarian O (2005) Multicategory proximal support vector machine classifiers. Mach Learn 59:77–97

    Article  MATH  Google Scholar 

  7. Ghorai S, Mukherjee A, Dutta P (2009) Nonparallel plane proximal classifier. Signal Process 89(4):510–522

    Article  MATH  Google Scholar 

  8. Jayadeva, Khemchandani R, Chandra S (2007) Twin support vector machines for pattern classification. IEEE Trans Pattern Anal Mach Intell 29(5):905–910

    Article  Google Scholar 

  9. Jayadeva, Khemchandani R, Chandra S (2007) Fuzzy multi-category proximal support vector classification via generalized eigenvalues. Soft Comput 11(7):679–685

    Article  Google Scholar 

  10. Khemchandani R, Jayadeva, Chandra S (2009) Optimal kernel selection in twin support vector machines. Optim Lett 3(1):77–88

    Article  MATH  MathSciNet  Google Scholar 

  11. Kumar M, Gopal M (2009) Least squares twin support vector machines for pattern classification. Expert Syst Appl 36(4):7535–7543

    Article  Google Scholar 

  12. Li I, Chen J, Wu J (2013) A fast prototype reduction method based on template reduction and visualization-induced self-organizing map for nearest neighbor algorithm. Appl Intell 39:564–582

    Article  Google Scholar 

  13. Mangasarian O, Musicant D (1999) Successive overrelaxation for support vector machines. IEEE Trans Neural Netw 10:1032–1037

    Article  Google Scholar 

  14. Matei O, Pop P, Vălean (2013) Optical character recognition in real environments using neural networks and k-nearest neighbor. Appl Intell 39:739–748

    Article  Google Scholar 

  15. Owusu E, Zhan Y, Mao Q (2013) An SVM-AdaBoost facial expression recognition system. Appl Intell. doi:10.1007/s10489-013-0478-9

    MATH  Google Scholar 

  16. Peng X (2010) TSVR: an efficient twin support vector machine for regression. Neural Netw 23(3):365–372

    Article  Google Scholar 

  17. Peng X (2010) A new twin support vector machine classifier and its geometric algorithms. Inf Sci 180(20):3863–3875

    Article  MATH  Google Scholar 

  18. Ripley B (1996) Pattern recognition and neural networks. Cambridge University Press, Cambridge

    MATH  Google Scholar 

  19. Shao Y, Wang Z, Chen W, Deng N (2013) Least squares twin parametric-margin support vector machine for classification. Appl Intell 39:451–464

    Article  Google Scholar 

  20. Vapnik V (1995) The nature of statistical learning theory. Springer, New York

    Book  MATH  Google Scholar 

  21. Wang C, You W (2013) Boosting-SVM: effective learning with reduced data dimension. Appl Intell 39:465–474

    Article  Google Scholar 

  22. Xu Y (2012) A rough margin-based linear ν support vector regression. Stat Probab Lett 82:528–534

    Article  MATH  Google Scholar 

  23. Xu Y, Guo R (2014) An improved v-twin support vector machine. Applied Intell. doi:10.1007/s10489-013-0500-2

  24. Xu Y, Wang L (2005) Fault diagnosis system based on rough set theory and support vector machine. Lect Notes Artif Intell 3614:981–988

    Google Scholar 

  25. Xu Y, Wang L (2012) A weighted twin support vector regression. Knowl-Based Syst 33:92–101

    Article  Google Scholar 

  26. Xu Y, Wang L, Zhong P (2012) A rough margin-based ν-twin support vector machine. Neural Comput Appl 21:1307–1317

    Article  Google Scholar 

  27. Xu Y, Lv X, Xi W (2012) A weighted multi-output support vector regression and its application. J Comput Inf Syst 8(9):3807–3814

    Google Scholar 

  28. Ye Q, Zhao C, Gao S, Zheng H (2012) Weighted twin support vector machines with local information and its application. Neural Netw 35:31–39

    Article  MATH  Google Scholar 

Download references

Acknowledgements

The authors gratefully acknowledge the helpful comments and suggestions of the reviewers, which have improved the presentation. This work was supported by the National Natural Science Foundation of China (No. 61153003, 11171346 and 11271367) and China Scholarship Fund (No. 201208110282).

Author information

Authors and Affiliations

  1. College of Science, China Agricultural University, Beijing, 100083, China

    Yitian Xu & Laisheng Wang

  2. Department of Computer Science and Engineering, Arizona State University, Tempe, AZ, 85287, USA

    Yitian Xu

Authors
  1. Yitian Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Laisheng Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yitian Xu.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Xu, Y., Wang, L. K-nearest neighbor-based weighted twin support vector regression. Appl Intell 41, 299–309 (2014). https://doi.org/10.1007/s10489-014-0518-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-014-0518-0

Keywords

Search

Navigation