Skip to main content
SpringerLink
Log in

A guided tour across the boundaries of learning recursive languages

  • 1 Inductive Inference Theory
  • Chapter
  • First Online:
Algorithmic Learning for Knowledge-Based Systems

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 961))

Abstract

The present paper deals with the learnability of indexed families of uniformly recursive languages from positive data as well as from both, positive and negative data. We consider the influence of various monotonicity constraints to the learning process, and provide a thorough study concerning the influence of several parameters. In particular, we present examples pointing to typical problems and solutions in the field. Then we provide a unifying framework for learning. Furthermore, we survey results concerning learnability in dependence on the hypothesis space, and concerning order independence. Moreover, new results dealing with the efficiency of learning are provided. First, we investigate the power of iterative learning algorithms. The second measure of efficiency studied is the number of mind changes a learning algorithm is allowed to perform. In this setting we consider the problem whether or not the monotonicity constraints introduced do influence the efficiency of learning algorithms.

The paper mainly emphasis to provide a comprehensive summary of results recently obtained, and of proof techniques developed. Finally, throughout our guided tour we discuss the question of what a natural language learning algorithm might look like.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  • Adleman, L.M., and Blum, M. (1991), Inductive inference and unsolvability, Journal of Symbolic Logic56, 891–900.

    Google Scholar 

  • Angluin, D. (1980a), Finding patterns common to a set of strings, Journal of Computer and System Sciences, 21, 46–62.

    Google Scholar 

  • Angluin, D. (1980b), Inductive inference of formal languages from positive data, Information and Control, 45 (1980), 117–135.

    Google Scholar 

  • Angluin, D. (1992), Computational learning theory: Survey and selected bibliography, in “Proceedings 24th Annual ACM Workshop on Theory of Computing,” pp. 351–369, ACM Press.

    Google Scholar 

  • Angluin, D., and Smith, C.H. (1983), Inductive inference: theory and methods, Computing Surveys15, 237–269.

    Google Scholar 

  • Angluin, D., and Smith, C.H. (1987), Formal inductive inference, in “Encyclopedia of Artificial Intelligence” (St.C. Shapiro, Ed.), Vol. 1, pp. 409–418, Wiley-Interscience Publication, New York.

    Google Scholar 

  • Anthony, M. and Biggs, N. (1992), “Computational Learning Theory,” Cambridge University Press, Cambridge.

    Google Scholar 

  • Arikawa, S., Goto, S., Ohsuga, S., and Yokomori, T. (Eds.) (1990) “Proceedings 1st International Workshop on Algorithmic Learning Theory,” October 1990, Tokyo, Japanese Society for Artificial Intelligence.

    Google Scholar 

  • Arikawa, S., Maruoka, A., and Sato, T. (Eds.) (1991) “Proceedings 2nd International Workshop on Algorithmic Learning Theory,” October 1991, Tokyo, Japanese Society for Artificial Intelligence.

    Google Scholar 

  • Arikawa, S., Kuhara, S., Miyano, S., Mukouchi, Y., Shinohara, A. and Shinohara, T. (1992), A machine discovery from amino acid sequences by decision trees over regular patterns, in Proceedings International Conference on Fifth Generation Computer Systems, Vol. 2, pp. 618–625, Institute for New Generation Computer Technology (ICOT), Tokyo, Japan.

    Google Scholar 

  • Barzdin, Ya. M. (1974), Inductive inference of automata, functions and programs, in “Proceedings International Congress of Math.,” Vancouver, pp. 455–460.

    Google Scholar 

  • Barzdin, Ya.M., and Freivalds, R.V. (1972), On the prediction of general recursive functions, Sov. Math. Dokl.13, 1224–1228.

    Google Scholar 

  • Barzdin, Ya.M., and Freivalds, R.V. (1974), Прогнозированиye и преqdелQwный синтез еффективно перечислимыqh классов фунций, in “Теория АлгоритмОв и Программ,” Vol. 1 (Ya. M. Barzdin, ed.) Latvian State University, Riga, pp. 101–111.

    Google Scholar 

  • Barzdin, Ya.M., Kinber, E.B., and Podnieks, K.M. (1974), ОБ укорении синтеза и прогрозирования функций, in “Теориц Алгоритмов и Программ,” Vol. 1 (Ya.M. Barzdin, Ed.) Latvian State University, Riga, pp. 117–128.

    Google Scholar 

  • Berwick, R. (1985), “The Acquisition of Syntatic Knowledge,” MIT Press, Cambridge, Massachusetts.

    Google Scholar 

  • Blum, A., and Singh, M. (1990), Learning functions of k terms, in “Proceedings 3rd Workshop on Computational Learning Theory, July 1990, Rochester,” (M. Fulk and J. Case, Eds.), pp. 144–153, Morgan Kaufmann Publishers Inc., San Mateo.

    Google Scholar 

  • Brewka, G. (1991), “Nonmonotonic Reasoning: Logical Foundations of Commonsense,” Cambridge University Press, Cambridge.

    Google Scholar 

  • Case, J. (1988), The power of vacillation, in “Proceedings 1st Workshop on Computational Learning Theory, August 1988, Boston,” (D. Haussler and L. Pitt, Eds.), pp. 196–205, Morgan Kaufmann Publishers Inc., San Mateo.

    Google Scholar 

  • Case, J., and Lynes, C. (1982), Machine inductive inference and language identification, in “Proceedings Automata, Languages and Programming, Ninth Colloquium, Aarhus, Denmark,” (M. Nielsen and E.M. Schmidt, Eds.), Lecture Notes in Computer Science Vol. 140, pp. 107–115, Springer-Verlag, Berlin.

    Google Scholar 

  • Case, J., and Smith, C.H. (1983), Comparison of identification criteria for machine inductive inference, Theoretical Computer Science25, 193–220.

    Google Scholar 

  • Freivalds, R., Kinber, E.B. and Wiehagen, R. (1988), Probabilistic versus deterministic inductive inference in nonstandard numberings, Zeitschrift für Mathematische Logik und Grundlagen der Mathematik, 34 (1988), 531–539.

    Google Scholar 

  • Freivalds, R., Kinber, E.B. and Wiehagen, R. (1992), Convergently versus divergently incorrect hypotheses in inductive inference, GOSLER-Report 02/92, January 1992, FB Mathematik und Informatik, TH Leipzig.

    Google Scholar 

  • FulK, M. (1990), Prudence and other restrictions in formal language learning, Information and Computation, 85 1–11.

    Google Scholar 

  • Fulk, M., and Case, J. (Eds.) (1990), Proceedings of the 3rd Annual Workshop on Computational Learning Theory, July 1990, Rochester, Morgan Kaufmann Publishers Inc., San Mateo.

    Google Scholar 

  • Gasarch, W.I., and Velauthapillai, M. (1992), Asking questions versus verifiability, in “Proceedings 3rd International Workshop on Analogical and Inductive Inference,” October 1992, Dagstuhl, (K.P. Jantke, ed.) Lecture Notes in Artificial Intelligence Vol. 642, pp. 197–213, Springer-Verlag, Berlin.

    Google Scholar 

  • Gold, M.E. (1965), Limiting recursion, Journal of Symbolic Logic, 30 28–48.

    Google Scholar 

  • Gold, M.E. (1967), Language identification in the limit, Information and Control10, 447–474

    Google Scholar 

  • Haussler, D. (Ed.) (1992), Proceedings of the 5th Annual Workshop on Computational Learning Theory, July 1992, Pittsburgh, ACM Press, New York.

    Google Scholar 

  • Hopcroft, J.E., and Ullman, J.D. (1969), “Formal Languages and their Relation to Automata,” Addison-Wesley, Reading, Massachusetts.

    Google Scholar 

  • Ishizaka, H. (1989), Learning simple deterministic languages. in “Proceedings of the 2nd Annual Workshop on Computational Learning Theory, Santa Cruz, August 1989, (R. Rivest, D. Haussler and M.K. Warmuth, Eds.), pp. 162–174, Morgan Kaufmann Publishers Inc., San Mateo.

    Google Scholar 

  • Jain, S., and Sharma, A. (1989), Recursion theoretic characterizations of language learning, The University of Rochester, Dept. of Computer Science, TR 281.

    Google Scholar 

  • Jantke, K.P. (Ed.) (1989), “Proceedings 2nd International Workshop on Analogical and Inductive Inference, October 1989, Reinhardsbrunn Castle,” Lecture Notes in Artificial Intelligence Vol. 397.

    Google Scholar 

  • Jantke, K.P. (1991a), Monotonic and non-monotonic inductive inference, New Generation Computing8, 349–360.

    Google Scholar 

  • Jantke, K.P. (1991b), Monotonic and non-monotonic inductive inference of functions and patterns, in “Proceedings 1st International Workshop on Nonmonotonic and Inductive Logics, December 1990, Karlsruhe,” (J. Dix, K.P. Jantke and P.H. Schmitt, Eds.), Lecture Notes in Artificial Intelligence Vol. 543, pp. 161–177, Springer-Verlag, Berlin.

    Google Scholar 

  • Jantke, K.P. (Ed.) (1992), “Proceedings 3rd International Workshop on Analogical and Inductive Inference, October 1992, Dagstuhl Castle,” Lecture Notes in Artificial Intelligence Vol. 642.

    Google Scholar 

  • Kapur, S. (1992), Monotonic language learning, in “Proceedings 3rd Workshop on Algorithmic Learning Theory,” October 1992, Tokyo, (S. Doshita, K. Furukawa, K.P. Jantke and T. Nishida, Eds.), Lecture Notes in Artificial Intelligence Vol. 743, pp. 147–158, Springer-Verlag, Berlin.

    Google Scholar 

  • Kapur, S., and Bilardi, G. (1992), Language learning without overgeneralization, in “Proceedings 9th Annual Symposium on Theoretical Aspects of Computer Science, Cachan, France, February 13–15,” (A. Finkel and M. Jantzen, Eds.), Lecture Notes in Computer Science Vol. 577, pp. 245–256, Springer-Verlag, Berlin.

    Google Scholar 

  • Kearns, M., and Pitt, L. (1989), A polynomial-time algorithm for learning k-variable pattern languages from examples, in “Proceedings 1st Annual Workshop on Computational Learning Theory, August 1988, Boston,” (D. Haussler and L. Pitt, Eds.), pp. 196–205, Morgan Kaufmann Publishers Inc., San Mateo.

    Google Scholar 

  • Kinber, E.B. (1992), personal communication.

    Google Scholar 

  • Kodratoff, Y., and Michalski, R.S. (1990), “Machine Learning, An Artificial Intelligence Approach,” Vol. 3, Morgan Kaufmann Publishers Inc., San Mateo.

    Google Scholar 

  • Lange, S. (1994), The representation of recursive languages and its impact on the efficiency of learning, in “Proceedings 7th Annual ACM Conference on Computational Learning Theory, New Brunswick, July 1994,” (M. Warmuth, Ed.), pp. 256–267, ACM Press, New York.

    Google Scholar 

  • Lange, S., and Wiehagen, R. (1991), Polynomial-time inference of arbitrary pattern languages, New Generation Computing8, 361–370.

    Google Scholar 

  • Lange, S., and Zeugmann, T. (1992), Types of monotonic language learning and their characterization, in “Proceedings 5th Annual ACM Workshop on Computational Learning Theory, Pittsburgh, July 1992,” (D. Haussler, Ed.), pp. 377–390, ACM Press, New York.

    Google Scholar 

  • Lange, S., and Zeugmann, T. (1993a), Monotonic versus non-monotonic language learning, in “Proceedings 2nd International Workshop on Nonmonotonic and Inductive Logic, December 1991, Reinhardsbrunn,” (G. Brewka, K.P. Jantke and P.H. Schmitt, Eds.), Lecture Notes in Artificial Intelligence Vol. 659, pp. 254–269, Springer-Verlag, Berlin.

    Google Scholar 

  • Lange, S., and Zeugmann, T. (1993b), Learning recursive languages with bounded mind changes, International Journal of Foundations of Computer Science4, 157–178.

    Google Scholar 

  • Lange, S., and Zeugmann, T. (1993c), Language learning in dependence on the space of hypotheses, in “Proceedings 6th Annual ACM Conference on Computational Learning Theory,” Santa Cruz, July 1993, pp. 127–136, ACM Press, New York.

    Google Scholar 

  • Lange, S., and Zeugmann, T. (1993d), The learnability of recursive languages in dependence on the space of hypotheses, GOSLER-Report 20/93, July 1993, Fachbereich Mathematik und Informatik, TH Leipzig.

    Google Scholar 

  • Lange, S., and Zeugmann, T. (1993e), On the impact of order independence to the learnability of recursive languages, Research Report ISIS-RR-93-17E, Institute for Social Information Science, FUJITSU Laboratories Ltd, Numazu.

    Google Scholar 

  • Lange, S., and Zeugmann, T. (1994), Characterization of language learning on informant under various monotonicity constraints, Journal of Experimental and Theoretical Artificial Intelligence6, 73–94.

    Google Scholar 

  • Lange, S., Zeugmann, T., and Kapur, S (1992), Class preserving monotonic language learning, submitted to Theoretical Computer Science, and GOSLER-Report 14/92, FB Mathematik und Informatik, TH Leipzig.

    Google Scholar 

  • Machtey, M., and Young, P. (1978), “An Introduction to the General Theory of Algorithms,” North-Holland, New York.

    Google Scholar 

  • Michalski, R.S., Carbonell, J.G., and Mitchell, T.M. (1984), “Machine Learning, An Artificial Intelligence Approach,” Vol. 1, Springer-Verlag, Berlin.

    Google Scholar 

  • Michalski, R.S., Carbonell, J.G., and Mitchell, T.M. (1986), “Machine Learning, An Artificial Intelligence Approach,” Vol. 2, Morgan Kaufmann Publishers Inc., San Mateo.

    Google Scholar 

  • Mukouchi, Y. (1992), Inductive inference with bounded mind changes, in “Proceedings 3rd Workshop on Algorithmic Learning Theory,” October 1992, Tokyo, (S. Doshita, K. Furukawa, K.P. Jantke and T. Nishida, Eds.), Lecture Notes in Artificial Intelligence Vol. 743, pp. 125–134, Springer-Verlag, Berlin.

    Google Scholar 

  • Mukouchi, Y. (1994), Inductive inference of recursive concepts, Ph.D. Thesis, RI-FIS, Kyushu University 33, RIFIS-TR-CS-82, March 25th.

    Google Scholar 

  • Natarajan, B.K. (1991), “Machine Learning, A Theoretical Approach,” Morgan Kaufmann Publishers, Inc., San Mateo.

    Google Scholar 

  • Nix, R.P. (1983), Editing by examples, Yale University, Dept. Computer Science, Technical Report 280.

    Google Scholar 

  • Osherson, D., Stob, M., and Weinstein, S. (1986), “Systems that Learn, An Introduction to Learning Theory for Cognitive and Computer Scientists,” MIT-Press, Cambridge, Massachusetts.

    Google Scholar 

  • Pitt, L., and Valiant, L.G. (1988), Computational limitations on learning from examples, Journal of the ACM35, 965–984.

    Google Scholar 

  • Popper, K. (1968), “The Logic of Scientific Discovery,” Harper Torch Books.

    Google Scholar 

  • Rivest, R., Haussler, D., and Warmuth, M.K. (Eds.) (1989), Proceedings of the 2nd Annual Workshop on Computational Learning Theory, August 1989, Santa Cruz, Morgan Kaufmann Publishers Inc., San Mateo.

    Google Scholar 

  • Schäfer-Richter, G. (1984), Über Eingabeabhängigkeit und Komplexität von Inferenzstrategien, Rheinisch Westfälische Technische Hochschule Aachen, Dissertation.

    Google Scholar 

  • Shinohara, T. (1982), Polynomial time inference of extended regular pattern languages, in “Proceedings RIMS Symposia on Software Science and Engineering,” Kyoto, Lecture Notes in Computer Science 147, pp. 115–127, Springer-Verlag, Berlin.

    Google Scholar 

  • Solomonoff, R. (1964), A formal theory of inductive inference, Information and Control7, 1–22, 234–254.

    Google Scholar 

  • Trakhtenbrot, B.A., and Barzdin, Ya.M. (1970) “Конеч ые втоматы (Поведение и Синтез),” Наука, Москва, English translation: “Finite Automata-Behavior and Synthesis, Fundamental Studies in Computer Science 1,” North-Holland, Amsterdam, 1973.

    Google Scholar 

  • Wexler, K. (1992), The subset principle is an intensional principle, in “Knowledge and Language: Issues in Representation and Acquisition,” ((E. Reuland and W. Abraham, Eds.), Kluwer Academic Publishers.

    Google Scholar 

  • Wexler, K., and Culicover, P. (1980), “Formal Principles of Language Acquisition,” MIT Press, Cambridge, Massachusetts.

    Google Scholar 

  • Wiehagen, R. (1976), Limes-Erkennung rekursiver Funktionen durch spezielle Strategien, Journal of Information Processing and Cybernetics (EIK), 12, 93–99.

    Google Scholar 

  • Wiehagen, R. (1977), Identification of formal languages, in “Proceedings Mathematical Foundations of Computer Science, Tatranska Lomnica,” (J. Gruska, Ed.), Lecture Notes in Computer Science 53, pp. 571–579, Springer-Verlag, Berlin.

    Google Scholar 

  • Wiehagen, R. (1978), Characterization problems in the theory of inductive inference, in “Proceedings 5th Colloquium on Automata, Languages and Programming,” (G. Ausiello and C. Böhm, Eds.), Lecture Notes in Computer Science 62, pp. 494–508, Springer-Verlag, Berlin.

    Google Scholar 

  • Wiehagen, R. (1991), A thesis in inductive inference, in “Proceedings First International Workshop on Nonmonotonic and Inductive Logic,” (J. Dix, K.P. Jantke and P.H. Schmitt, Eds.), Lecture Notes in Artificial Intelligence 543, pp. 184–207, Springer-Verlag, Berlin.

    Google Scholar 

  • Wiehagen, R., Freivalds, R., and Kinber, E.B. (1984), On the power of probabilistic strategies in inductive inference, Theoretical Computer Science28, 111–133.

    Google Scholar 

  • Wiehagen, R., and Zeugmann, T. (1994), Learning and Consistency, this volume.

    Google Scholar 

  • Zeugmann, T., Lange, S., and Kapur, S. (199x), Characterizations of monotonic and dual monotonic language learning, Information and Computation, to appear.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Research Institute of Fundamental Information Science, Kyushu University 33, 812, Fukuoka, Japan

    Thomas Zeugmann

  2. FB Mathematik und Informatik, HTWK Leipzig, PF 66, 04251, Leipzig

    Steffen Lange

Authors
  1. Thomas Zeugmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Steffen Lange
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Klaus P. Jantke Steffen Lange

Rights and permissions

Reprints and permissions

Copyright information

© 1995 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Zeugmann, T., Lange, S. (1995). A guided tour across the boundaries of learning recursive languages. In: Jantke, K.P., Lange, S. (eds) Algorithmic Learning for Knowledge-Based Systems. Lecture Notes in Computer Science, vol 961. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-60217-8_12

Download citation

  • DOI: https://doi.org/10.1007/3-540-60217-8_12

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-60217-0

  • Online ISBN: 978-3-540-44737-5

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation