Skip to main content
Springer Nature Link
Log in

Incremental Learning with Partial Instance Memory

  • Conference paper
  • First Online:
Foundations of Intelligent Systems (ISMIS 2002)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 2366))

Included in the following conference series:

Abstract

Agents that learn on-line with partial instance memory reserve some of the previously encountered examples for use in future training episodes. We extend our previous work by combining our method for selecting extreme examples with two incremental learning algorithms, aq11 and gem. Using these new systems, aq11-pm and gem-pm, and the task computer intrusion detection, we conducted a lesion study to analyze trade-offs in performance. Results showed that, although our partial-memory model decreased predictive accuracy by 2%, it also decreased memory requirements by 75%, learning time by 75%, and in some cases, concept complexity by 10%, an outcome consistent with earlier results using our partial-memory method and batch learning.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Maloof, M., Michalski, R.: Selecting examples for partial memory learning. Machine Learning 41 (2000) 27–52

    Article  Google Scholar 

  2. Michalski, R., Larson, J.: Incremental generation of VL1 hypotheses: The underlying methodology and the description of program AQ11. Technical Report UIUCDCS-F-83-905, Department of Computer Science, University of Illinois, Urbana (1983)

    Google Scholar 

  3. Reinke, R., Michalski, R.: Incremental learning of concept descriptions: A method and experimental results. In Hayes, J., Michie, D., Richards, J., eds.: Machine Intelligence 11. Clarendon Press, Oxford (1988) 263–288

    Google Scholar 

  4. Aha, D., Kibler, D., Albert, M.: Instance-based learning algorithms. Machine Learning 6 (1991) 37–66

    Google Scholar 

  5. Schlimmer, J., Granger, R.: Beyond incremental processing: Tracking concept drift. In: Proceedings of the Fifth National Conference on Artificial Intelligence, Menlo Park, CA, AAAI Press (1986) 502–507

    Google Scholar 

  6. Littlestone, N.: Redundant noisy attributes, attribute errors, and linear-threshold learning using Winnow. In: Proceedings of the Fourth Annual Workshop on Computational Learning Theory, San Francisco, CA, Morgan Kaufmann (1991) 147–156

    Google Scholar 

  7. Kibler, D., Aha, D.: Learning representative exemplars of concepts: An initial case study. In: Proceedings of the Fourth International Conference on Machine Learning, San Francisco, CA, Morgan Kaufmann (1987) 24–30

    Google Scholar 

  8. Widmer, G.: Tracking context changes through meta-learning. Machine Learning 27 (1997) 259–286

    Article  Google Scholar 

  9. Widmer, G., Kubat, M.: Learning in the presence of concept drift and hidden contexts. Machine Learning 23 (1996) 69–101

    Google Scholar 

  10. Maloof, M.: Progressive partial memory learning. PhD thesis, School of Information Technology and Engineering, George Mason University, Fairfax, VA (1996)

    Google Scholar 

  11. Michalski, R.: On the quasi-minimal solution of the general covering problem. In: Proceedings of the Fifth International Symposium on Information Processing. Volume A3. (1969) 125–128

    MathSciNet  Google Scholar 

  12. Clark, P., Niblett, T.: The CN2 induction algorithm. Machine Learning 3 (1989) 261–284

    Google Scholar 

  13. Cohen, W.: Fast effective rule induction. In: Proceedings of the Twelfth International Conference on Machine Learning, San Francisco, CA, Morgan Kaufmann (1995) 115–123

    Google Scholar 

  14. Michalski, R.: Pattern recognition as rule-guided inductive inference. IEEE Trans-actions on Pattern Analysis and Machine Intelligence 2 (1980) 349–361

    Article  MATH  Google Scholar 

  15. Michalski, R., Kaufman, K.: The AQ-19 system for machine learning and pattern discovery: A general description and user’s guide. Reports of the Machine Learning and Inference Laboratory MLI 01-4, Machine Learning and Inference Laboratory, George Mason University, Fairfax, VA (2001)

    Google Scholar 

  16. Michalski, R.: A theory and methodology of inductive learning. In Michalski, R., Carbonell, J., Mitchell, T., eds.: Machine Learning: An Artificial Intelligence Approach. Volume 1. Morgan Kaufmann, San Francisco, CA (1983) 83–134

    Google Scholar 

  17. Fisher, R.: The use of multiple measurements in taxonomic problems. Annals of Eugenics 7 (1936) 179–188

    Google Scholar 

  18. Blake, C., Merz, C.: UCI Repository of machine learning databases. [http://www.ics.uci.edu/~mlearn/mlrepository.html], Department of Information and Computer Sciences, University of California, Irvine (1998)

    Google Scholar 

  19. Bleha, S., Slivinsky, C., Hussien, B.: Computer-access security systems using keystroke dynamics. IEEE Transactions on Pattern Analysis and Machine Intelligence 12 (1990) 1217–1222

    Article  Google Scholar 

  20. Lane, T., Brodley, C.: Temporal sequence learning and data reduction for anomaly detection. ACM Transactions on Information and System Security 2 (1999) 295–331

    Article  Google Scholar 

  21. Lee, W., Stolfo, S., Mok, K.: Adaptive intrusion detection: A data mining approach. Artificial Intelligence Review 14 (2000) 533–567

    Article  MATH  Google Scholar 

  22. Maloof, M., Michalski, R.: A method for partial-memory incremental learning and its application to computer intrusion detection. In: Proceedings of the Seventh IEEE International Conference on Tools with Artificial Intelligence, Los Alamitos, CA, IEEE Press (1995) 392–397

    Chapter  Google Scholar 

  23. Davis, J.: CONVART: A program for constructive induction on time dependent data. Master’s thesis, Department of Computer Science,University of Illinois, Urbana (1981)

    Google Scholar 

  24. Bloedorn, E., Wnek, J., Michalski, R., Kaufman, K.: AQ17 — A multistrategy learning system: The method and user’s guide. Reports of the Machine Learning and Inference Laboratory MLI 93-12, Machine Learning and Inference Laboratory, George Mason University, Fairfax, VA (1993)

    Google Scholar 

  25. Kerber, R.: ChiMerge: Discretization of numeric attributes. In: Proceedings of the Tenth National Conference on Artificial Intelligence, Menlo Park, CA, AAAI Press (1992) 123–128

    Google Scholar 

  26. Baim, P.: A method for attribute selection in inductive learning systems. IEEE Transactions on Pattern Analysis and Machine Intelligence 10 (1988) 888–896

    Article  Google Scholar 

  27. Keppel, G., Saufley, W., Tokunaga, H.: Introduction to design and analysis. 2nd edn. W.H. Freeman, New York, NY (1992)

    Google Scholar 

  28. Maloof, M., Michsalski, R.: AQ-PM: A system for partial memory learning. In: Proceedings of the Eighth Workshop on Intelligent Information Systems, Warsaw, Poland, Polish Academy of Sciences (1999) 70–79

    Google Scholar 

  29. Winston, P.: Learning structural descriptions from examples. In Winston, P., ed.: Psychology of Computer Vision. MIT Press, Cambridge, MA (1975)

    Google Scholar 

  30. Thrun, S., et al.: The MONK’s problems: A performance comparison of different learning algorithms. Technical Report CMU-CS-91-197, School of Computer Science, Carnegie Mellon University, Pittsburg, PA (1991)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Georgetown University, Washington, DC, 20057, USA

    Marcus A. Maloof

  2. Machine Learning and Inference Laboratory, School of Computational Sciences, George Mason University, Fairfax, VA, 22030, USA

    Ryszard S. Michalski

  3. Institute of Computer Science, Polish Academy of Sciences, Warsaw

    Ryszard S. Michalski

Authors
  1. Marcus A. Maloof
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ryszard S. Michalski
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. UFR d’Informatique, Université Claude Bernard Lyon I, 8, boulevard Niels Bohr, 69622, Villeurbanne Cedex, France

    Mohand-Saïd Hacid

  2. Dept. of Computer Science College of IT, University of North Carolina, Charlotte, NC, 28223, USA

    Zbigniew W. Raś

  3. Băt. L. Equipe de Recherche en Ingénierie des Connaissances, Université Lumière Lyon 2, 5, avenue Pierre Mendes-France, 69676, Bron Cedex, France

    Djamel A. Zighed

  4. LRI, Université Paris Sud, Băt. 490, 91405, Orsay Cedex, France

    Yves Kodratoff

Rights and permissions

Reprints and permissions

Copyright information

© 2002 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Maloof, M.A., Michalski, R.S. (2002). Incremental Learning with Partial Instance Memory. In: Hacid, MS., Raś, Z.W., Zighed, D.A., Kodratoff, Y. (eds) Foundations of Intelligent Systems. ISMIS 2002. Lecture Notes in Computer Science(), vol 2366. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-48050-1_4

Download citation

  • DOI: https://doi.org/10.1007/3-540-48050-1_4

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-43785-7

  • Online ISBN: 978-3-540-48050-1

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics