Skip to main content
Springer Nature Link
Log in

Towards the online learning with Kernels in classification and regression

  • Original Paper
  • Published:
Evolving Systems Aims and scope Submit manuscript

Abstract

In this paper, optimization models and algorithms for online learning with Kernels (OLK) in classification and regression are proposed in a reproducing Kernel Hilbert Space (RKHS) by solving a constrained optimization model. The “Forgetting” factor in the model makes it possible that the memory requirement of the algorithm can be bounded as the learning process continues. The applications of the proposed OLK algorithms in classification and regression show their effectiveness in comparing with the state of art algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  • Dekel O, Shwartz SS, Singer Y (2008) The Forgetron: a Kernel-based Perceptron on a budget. SIAM J Comput 37:1342–1372

    Article  MATH  MathSciNet  Google Scholar 

  • Engel Y, Mannor S, Meir R (2004) The Kernel recursive least squares algorithm. IEEE Trans Singal Process 52:2275–22185

    Article  MathSciNet  Google Scholar 

  • Freund Y, Schapire RE (1999) Large margin classification using the perceptron algorithm. Mach Learn 37:277–296

    Article  MATH  Google Scholar 

  • Kecman V, Hadzic I (2000) Support vectors selection by linear programming, In: Proceedings of the international joint conference on neural networks, Como, Italy, pp 193–198

  • Kivinen J, Smola A, Williamson R (2001) Online learning with Kernels. NIPS, 785–792

  • Kivinen J, Smola A, Williamson R (2004) Online learning with Kernels. IEEE Trans Singal Process 52:2165–2176

    Article  MathSciNet  Google Scholar 

  • Li G, Wen C (2012) Identification of winner systems with clipped observations. IEEE Trans Signal Process 60:3845–3852

    Article  MathSciNet  Google Scholar 

  • Li G, Zhao G (2012) Online learning with Kernels in classification and regression. In: Proceedings of the 2012 IEEE conference on evolving and adaptive intelligent systems (EAIS), Madrid, Spain

  • Li G, Wen C, Huang BG, Chen Y (2011a) Error tolerance based support vector machine for regression. Neurocompting 74:771–782

    Article  Google Scholar 

  • Li G, Wen C, Zheng W, Chen Y (2011b) Identification of a class of nonlinear autoregressive models with exogenous inputs based on Kernel machines. IEEE Trans Signal Process 59:2146–2158

    Article  MathSciNet  Google Scholar 

  • Li G, Wen C, Li ZG, Zhang A, Feng Y, Mao KZ (2013) Model based online learning with Kernels. IEEE Trans Neural Netw Learn Syst 24:356–369

    Article  Google Scholar 

  • Nash SG, Sofer A (1996) Linear and nonlinear programming. McGraw-Hill Companies, USA

  • Orabona F, Keshet J, Caputo B (2011) Bounded Kernel-based online learning. J Mach Learn Res 10:2643–2666

    Google Scholar 

  • Scholkopf B, Smola A (2002) Learning with Kernels: support vector machines, regularization, optimization and beyond. MIT Press, Cambridge

  • Suykens JAK, Vandewalle J (1999) Least squares support vector machine classifiers. Neural Process Lett 9:293–300

    Article  MathSciNet  Google Scholar 

  • Vapnik V (1998) Statistical learning theroy. Wiley, New York

Download references

Author information

Authors and Affiliations

  1. School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore, 639798, Singapore

    Guoqi Li

  2. School of Aerospace Xi’an, Jiaotong University, Xi’an, Shaanxi, 710049, People’s Republic of China

    Guangshe Zhao

  3. Department of Computing Science, Institute of High Performance Computing, A*Star, Singapore, 138632, Singapore

    Feng Yang

Authors
  1. Guoqi Li
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Guangshe Zhao
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Feng Yang
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Feng Yang.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Li, G., Zhao, G. & Yang, F. Towards the online learning with Kernels in classification and regression. Evolving Systems 5, 11–19 (2014). https://doi.org/10.1007/s12530-013-9090-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12530-013-9090-9

Keywords