Skip to main content
SpringerLink
Log in

Learning with Temporary Memory

  • Conference paper
Algorithmic Learning Theory (ALT 2008)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 5254))

Included in the following conference series:

Abstract

In the inductive inference framework of learning in the limit, a variation of the bounded example memory (Bem) language learning model is considered. Intuitively, the new model constrains the learner’s memory not only in how much data may be retained, but also in how long that data may be retained. More specifically, the model requires that, if a learner commits an example x to memory in some stage of the learning process, then there is some subsequent stage for which x no longer appears in the learner’s memory. This model is called temporary example memory (Tem) learning. In some sense, it captures the idea that memories fade.

Many interesting results concerning the Tem-learning model are presented. For example, there exists a class of languages that can be identified by memorizing k + 1 examples in the Tem sense, but that cannot be identified by memorizing k examples in the Bem sense. On the other hand, there exists a class of languages that can be identified by memorizing just 1 example in the Bem sense, but that cannot be identified by memorizing any number of examples in the Tem sense. (The proof of this latter result involves an infinitary self-reference argument.) Results are also presented concerning the special cases of: learning indexable classes of languages, and learning (arbitrary) classes of infinite languages.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Bakker, B.: Reinforcement learning with long short-term memory. Advances in Neural Inform. Processing Systems 14, 1475–1482 (2002)

    Google Scholar 

  2. Case, J.: Periodicity in generations of automata. Mathematical Systems Theory 8(1), 15–32 (1974)

    Article  MATH  MathSciNet  Google Scholar 

  3. Case, J.: Infinitary self-reference in learning theory. Journal of Experimental and Theoretical Artificial Intelligence 6(1), 3–16 (1994)

    Article  MATH  MathSciNet  Google Scholar 

  4. Carlucci, L., Case, J., Jain, S., Stephan, F.: Results on memory-limited U-shaped learning. Inform. Comput. 205(10), 1551–1573 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  5. Case, J., Jain, S., Lange, S., Zeugmann, T.: Incremental concept learning for bounded data mining. Inform. Comput. 152(1), 74–110 (1999)

    Article  MATH  MathSciNet  Google Scholar 

  6. Gold, E.M.: Language identification in the limit. Inform. Control 10(5), 447–474 (1967)

    Article  MATH  Google Scholar 

  7. Hopcroft, J.E., Motwani, R., Ullman, J.D.: Introduction to automata theory, languages, and computation, 2nd edn. Addison Wesley, Reading (2001)

    MATH  Google Scholar 

  8. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Computation 9(8), 1735–1780 (1997)

    Article  Google Scholar 

  9. Jain, S., Stephan, F.: Learning in Friedberg numberings. In: Hutter, M., Servedio, R.A., Takimoto, E. (eds.) ALT 2007. LNCS (LNAI), vol. 4754, pp. 79–93. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  10. Kinber, E., Stephan, F.: Language learning from texts: mind changes, limited memory, and monotonicity. Inform. Comput. 123(2), 224–241 (1995)

    Article  MATH  MathSciNet  Google Scholar 

  11. Lin, L.J., Mitchell, T.: Reinforcement learning with hidden states. In: Proc. of 2nd Intl. Conf. on Simulation of Adaptive Behavior, pp. 271–280 (1992)

    Google Scholar 

  12. Lange, S., Moelius, S.E., Zilles, S.: Learning with temporary memory (expanded version). Technical report, University of Delaware (2008), http://www.cis.udel.edu/~moelius/publications

  13. Lange, S., Zeugmann, T.: Incremental learning from positive data. Journal of Computer and System Sciences 53(1), 88–103 (1996)

    Article  MATH  MathSciNet  Google Scholar 

  14. Lange, S., Zeugmann, T., Zilles, S.: Learning indexed families of recursive languages from positive data: A survey. Theor. Comput. Sci. 397(1-3), 194–232 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  15. McCallum, R.A.: Learning to use selective attention and short-term memory in sequential tasks. In: Proc. of 4th Intl. Conf. on Simulation of Adaptive Behavior, pp. 315–324 (1996)

    Google Scholar 

  16. Mitchell, T.M.: Machine Learning. McGraw-Hill Higher Education, New York (1997)

    MATH  Google Scholar 

  17. Osherson, D., Stob, M., Weinstein, S.: Systems that Learn: An Introduction to Learning Theory for Cognitive and Computer Scientists, 1st edn. MIT Press, Cambridge (1986)

    Google Scholar 

  18. Rabaey, J.M., Chandrakasan, A., Nikolic, B.: Digital Integrated Circuits: A Design Perspective, 2nd edn. Prentice-Hall, Inc., Englewood Cliffs (2003)

    Google Scholar 

  19. Rogers, H.: Theory of Recursive Functions and Effective Computability. McGraw Hill, New York (1967) (Reprinted, MIT Press, 1987)

    MATH  Google Scholar 

  20. Wexler, K., Culicover, P.W.: Formal Principles of Language Acquisition. MIT Press, Cambridge (1980)

    Google Scholar 

  21. Wiehagen, R.: Limes-Erkennung rekursiver Funktionen durch spezielle Strategien. Elektron. Inform. Kybernetik 12(1/2), 93–99 (1976)

    MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Fachbereich Informatik, Hochschule Darmstadt,  

    Steffen Lange

  2. Department of Computer & Information Sciences, University of Delaware,  

    Samuel E. Moelius III

  3. Department of Computing Science, University of Alberta,  

    Sandra Zilles

Authors
  1. Steffen Lange
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Samuel E. Moelius III
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sandra Zilles
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Computer Science and Engineering, University of California, San Diego, USA

    Yoav Freund

  2. Department of Computer Science and Information Theory, Department of Computer Science and Budapest University of Technology and Economics, Stoczek u. 2, 1521, Budapest, Hungary

    László Györfi

  3. Department of Math., Stat. and Comp. Sci,, University of Illinois, 851 S. Morgan, IL 60607-7045, Chicago, USA

    György Turán

  4. Division of Computer Science, Hokkaido University, N-14, W-9, 060-0814, Sapporo, Japan

    Thomas Zeugmann

Rights and permissions

Reprints and permissions

Copyright information

© 2008 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Lange, S., Moelius, S.E., Zilles, S. (2008). Learning with Temporary Memory. In: Freund, Y., Györfi, L., Turán, G., Zeugmann, T. (eds) Algorithmic Learning Theory. ALT 2008. Lecture Notes in Computer Science(), vol 5254. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-87987-9_36

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-87987-9_36

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-87986-2

  • Online ISBN: 978-3-540-87987-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation