Skip to main content
SpringerLink
Log in

Online Extreme Entropy Machines for Streams Classification and Active Learning

  • Conference paper
  • First Online:
Book cover Proceedings of the 9th International Conference on Computer Recognition Systems CORES 2015

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 403))

Abstract

When dealing with large evolving datasets one needs machine learning models able to adapt to the growing number of information. In particular, stream classification is a research topic where classifiers need an ability to rapidly change their solutions and behave stably after many changes in training set structure. In this paper we show how recently proposed Extreme Entropy Machine can be trained in an online fashion supporting not only adding/removing points to/from the model but even changing the size of the internal representation on demand. In particular we show how one can build a well-conditioned covariance estimator in an online scenario. All these operations are guaranteed to converge to the optimal solutions given by their offline counterparts.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    \(A^2\) denotes element-wise squaring of A.

  2. 2.

    Up to numerical errors.

  3. 3.

    https://github.com/balajiln/mondrianforest.

  4. 4.

    \(\text {BAC} = \tfrac{1}{2}\left( \tfrac{\text {TP}}{\text {TP} + \text {FN}} + \tfrac{\text {TN}}{\text {TN} + \text {FP}} \right) \).

  5. 5.

    We use Moore–Penrose pseudoinverse solution.

References

  1. Bache, K., Lichman, M.: UCI machine learning repository. http://archive.ics.uci.edu/ml (2013)

  2. Bartocha, K., Podolak, I.T.: Classifier ensembles for virtual concept drift-the DEnboost algorithm. In: Corchado, E., Kurzyński, M., Woźniak, M. (eds.) Hybrid Artificial Intelligent Systems, pp. 164–171. Springer, Berlin (2011)

    Chapter  Google Scholar 

  3. Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  4. Chong, E.K., Zak, S.H.: An Introduction to Optimization, vol. 76. Wiley, New York (2013)

    MATH  Google Scholar 

  5. Czarnecki, W.M., Tabor, J.: Extreme entropy machines: robust information theoretic classification. Pattern Anal. Appl. (2015). doi:10.1007/s10044-015-0497-8

    Google Scholar 

  6. Czarnecki, W.M., Tabor, J.: Multithreshold entropy linear classifier: theory and applications. Expert Syst. Appl. 42, 5591–5606 (2015)

    Article  Google Scholar 

  7. Drineas, P., Mahoney, M.W.: On the nyström method for approximating a gram matrix for improved kernel-based learning. J. Mach. Learn. Res. 6, 2153–2175 (2005)

    MathSciNet  MATH  Google Scholar 

  8. Fernández-Delgado, M., Cernadas, E., Barro, S., Amorim, D.: Do we need hundreds of classifiers to solve real world classification problems? J. Mach. Learn. Res. 15(1), 3133–3181 (2014)

    MathSciNet  MATH  Google Scholar 

  9. Gaber, M.M., Zaslavsky, A., Krishnaswamy, S.: Mining data streams: a review. ACM Sigmod Rec. 34(2), 18–26 (2005)

    Article  MATH  Google Scholar 

  10. Huang, G.B., Zhu, Q.Y., Siew, C.K.: Extreme learning machine: a new learning scheme of feedforward neural networks. In: Proceedings. 2004 IEEE International Joint Conference on Neural Networks, vol. 2, pp. 985–990. IEEE (2004)

    Google Scholar 

  11. Kosina, P., Gama, J.: Very fast decision rules for classification in data streams. Data Min. Know. Discov. 29(1), 168–202 (2015)

    Article  MathSciNet  Google Scholar 

  12. Krawczyk, B., Stefanowski, J., Wozniak, M.: Data stream classification and big data analytics. Neurocomputing 150, 238–239 (2015)

    Article  Google Scholar 

  13. Lakshminarayanan, B., Roy, D.M., Teh, Y.W.: Mondrian forests: efficient online random forests. In: Advances in Neural Information Processing Systems, pp. 3140–3148 (2014)

    Google Scholar 

  14. Ledoit, O., Wolf, M.: A well-conditioned estimator for large-dimensional covariance matrices. J. Multivar. Anal. 88(2), 365–411 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  15. Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., et al.: Scikit-learn: machine learning in python. J. Mach. Learn. Res. 12, 2825–2830 (2011)

    MathSciNet  MATH  Google Scholar 

  16. Settles, B.: Active learning. Synth. Lect. Artif. Intel. Mach. Learn. 6(1), 1–114 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  17. Zong, W., Huang, G.B., Chen, Y.: Weighted extreme learning machine for imbalance learning. Neurocomputing 101, 229–242 (2013)

    Article  Google Scholar 

Download references

Acknowledgments

The work of the first author was partially founded by National Science Centre Poland grant no. 2013/09/N/ST6/03015, while the work of the second one by National Science Centre Poland grant no. 2014/13/B/ST6/01792.

Author information

Authors and Affiliations

  1. Faculty of Mathematics and Computer Science, Jagiellonian University, Krakow, Poland

    Wojciech Marian Czarnecki & Jacek Tabor

Authors
  1. Wojciech Marian Czarnecki
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jacek Tabor
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wojciech Marian Czarnecki .

Editor information

Editors and Affiliations

  1. Department of Systems, Wrocław University of Technology, Wroclaw, Poland

    Robert Burduk

  2. Department of Systems and Computer, Wrocław University of Technology, Wroclaw, Poland

    Konrad Jackowski

  3. Department of Systems and Computer, Wrocław University of Technology, Wroclaw, Poland

    Marek Kurzyński

  4. Dept. of Systems and Computer Networks, Wrocław University of Technology, Wroclaw, Poland

    Michał Woźniak

  5. Department of Systems, Wrocław University of Technology, Wroclaw, Poland

    Andrzej Żołnierek

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Czarnecki, W.M., Tabor, J. (2016). Online Extreme Entropy Machines for Streams Classification and Active Learning. In: Burduk, R., Jackowski, K., Kurzyński, M., Woźniak, M., Żołnierek, A. (eds) Proceedings of the 9th International Conference on Computer Recognition Systems CORES 2015. Advances in Intelligent Systems and Computing, vol 403. Springer, Cham. https://doi.org/10.1007/978-3-319-26227-7_35

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-26227-7_35

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-26225-3

  • Online ISBN: 978-3-319-26227-7

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation