Skip to main content
SpringerLink
Log in

Online Bayes Point Machines

  • Conference paper
  • First Online:
Advances in Knowledge Discovery and Data Mining (PAKDD 2003)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 2637))

Included in the following conference series:

Abstract

We present a new and simple algorithm for learning large margin classifiers that works in a truly online manner. The algorithm generates a linear classifier by averaging the weights associated with several perceptron-like algorithms run in parallel in order to approximate the Bayes point. A random subsample of the incoming data stream is used to ensure diversity in the perceptron solutions. We experimentally study the algorithm’s performance on online and batch learning settings. The online experiments showed that our algorithm produces a low prediction error on the training sequence and tracks the presence of concept drift. On the batch problems its performance is comparable to the maximum margin algorithm which explicitly maximises the margin.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. C. Gentile, (2001) A new approximate maximal margin classification algorithm. Journal of Machine Learning Research, 2:213–242.

    Article  MathSciNet  Google Scholar 

  2. Y. Li. & P. Long, (2002) The relaxed online maximum margin algorithm. Machine Learning, 46(1–3):361–387.

    Article  MATH  Google Scholar 

  3. J. Kivinen, A. Smola, and R. C. Williamson, (2002) Online Learning with kernels. Advances in Neural Information Processing Systems 14, Cambridge, MA: MIT Press (pp. 785–793).

    Google Scholar 

  4. R.O. Duda & P.E. Hart & D.G. Stork, (2000) Pattern Classification And Scene Analysis 2nd Edition. John Wiley.

    Google Scholar 

  5. A.B.J. Novikoff, (1962) On convergence proofs on perceptrons. In Proceedings of the Symposium on Mathematical Theory of Automata, vol. XII, (pp. 615–622).

    Google Scholar 

  6. R. Herbrich, (2002) Learning Kernel Classifiers, Cambridge, MA: MIT Press.

    Google Scholar 

  7. R. Herbrich, T. Graepel & C. Campbell, (2001) Bayes Point Machines. Journal of Machine Learning Research, 1:245–279.

    Article  MATH  MathSciNet  Google Scholar 

  8. L. Breiman, (1996) Bagging predictors. Machine Learning, 24(2):120–140.

    MathSciNet  Google Scholar 

  9. R.E. Schapire, (1990) The strength of weak learnability. Machine Learning, 5:197–227.

    Google Scholar 

  10. S. Sonnenburg, (2002) New Methods for Splice Site Recognition. Master’s thesis, Humbold University.

    Google Scholar 

  11. W.J. Conover, (1980) Practical nonparametric statistics, 2nd Edition. John Wiley.

    Google Scholar 

  12. B. Widrow and M. E. Hoff, (1960) Adaptive switching circuits. 1960 IRE WESCON Convention Record, pt. 4, pp. 96–104.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Research School of Information Sciences and Engineering, The Australian National University, Canberra, ACT, 0200

    Edward Harrington, Jyrki Kivinen & Robert C. Williamson

  2. Microsoft Research, 7 J J Thomson Avenue, Cambridge, CB3 0FB, UK

    Ralf Herbrich

  3. Microsoft Research, 1 Microsoft Way, Redmond, WA, 98052, USA

    John Platt

Authors
  1. Edward Harrington
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ralf Herbrich
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jyrki Kivinen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. John Platt
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Robert C. Williamson
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Computer Science Department, Korea Advanced Institute of Science and Technology, 373-1 Koo-Sung Dong, Yoo-Sung Ku, Daejeon, 305-701, Korea

    Kyu-Young Whang

  2. Department of Statistics, Seoul National University, Sillimdong Kwanakgu, Seoul, 151-742, Korea

    Jongwoo Jeon

  3. School of Electrical Engineering and Computer Science, Seoul National University, Kwanak P.O. Box 34, Seoul, 151-742, Korea

    Kyuseok Shim

  4. Department of Computer Science and Engineering, University of Minnesota, 200 Union St SE, Minneapolis, MN, 55455, USA

    Jaideep Srivastava

Rights and permissions

Reprints and permissions

Copyright information

© 2003 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Harrington, E., Herbrich, R., Kivinen, J., Platt, J., Williamson, R.C. (2003). Online Bayes Point Machines. In: Whang, KY., Jeon, J., Shim, K., Srivastava, J. (eds) Advances in Knowledge Discovery and Data Mining. PAKDD 2003. Lecture Notes in Computer Science(), vol 2637. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-36175-8_24

Download citation

  • DOI: https://doi.org/10.1007/3-540-36175-8_24

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-04760-5

  • Online ISBN: 978-3-540-36175-6

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation