Abstract
In computational learning theory continuous efforts are made to formulate models of machine learning that are more realistic than previously available models. Two of the most popular models that have been recently proposed, Valiant’s PAC learning model and Angluin’s query learning model, can be thought of as refinements of preceding models such as Gold’s classic paradigm of identification in the limit, in which the question ofhow fast the learning can take place is emphasized. A considerable amount of results have been obtained within these two frameworks, resolving the learnability questions of many important classes of functions and languages. These two particular learning models are by no means comprehensive, and many important aspects of learning are not directly addressed in these models. Aiming towards more realistic theories of learning, many new models and extensions of existing learning models that attempt to formalize such aspects have been developed recently. In this paper, we will review some of these new extensions and models in computational learning theory, concentrating in particular on those proposed and studied by researchers at Theory NEC Laboratory RWCP, and their colleagues at other institutions.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Abe, N., “Feasible learnability of Formal Grammars and the Theory of Natural Language Acquisition,” inProceedings of COLING-88, August 1988.
Angluin, D. and Kharitonov, M., “When Wont’ Membership Queries Help?” inProc. of the 23rd Symposium on Theory of Computing, ACM Press, New York, NY, pp. 444–454, 1991.
Abe, N. and Mamitsuka, H., “A New Method for Predicting Protein Secondary Structures Based on Stochastic Tree Grammars,” inProceedings of the Eleventh International Conference on Machine Learning, 1994.
Abe, N. and Takeuchi, J., “The ‘Lob-Pass’ Problem and an On-Line Learning Model of Rational Choice,” inProceedings of the Sixth Annual ACM Workshop on Computational Learning Theory, Morgan Kaufmann, San Mateo, California, August 1993.
Abe, N. and Warmuth, M. K., “On the Computational Complexity of Approximating Probability Distributions by Probabilistic Automata,”Machine Learning, 9, 2/3, pp. 205–260, 1992.
Brown, M., Hughey, R., Krogh, A., Mian, I. S., Sjolander, K., and Haussler, D., “Using Dirichlet Mixture Priors to Derive Hidden Markov Models for Protein Families,” inProceedings of the First International Conference on Intelligent Systems for Molecular Biology, pp. 47–55, 1993.
Goldman, S., Kearns, M., and Schapire, R., “On the Sample Complexity of Weak Learning,” inProceedings of the 1990 Workshop on Computational Learning Theory, Morgan Kaufmann, San Mateo, California, August 1990.
Haussler, D., “Decision Theoretic Generalizations of the PAC Model for Neural Net and Other Learning Applications,”Information and Computation,100,1, September 1992.
Herrnstein, R., “Rational Choice Theory,”American Psychologist, 45, 3, pp. 356–367, 1990.
Kearns, M. and Seung, S., “Learning from a Population of Hypotheses,” inProceedings of the Sixth Annual ACM Workshop on Computational Learning Theory, Morgan Kaufmann, San Mateo, California, August 1993.
Kearns, M. and Schapire, R., “Efficient Distribution-Free Learning of Probabilistic Concepts,”Journal of Computer and System Sciences,48,3, June 1994. A special issue on 31st IEEE Conference on Foundations of Computer Science.
Laird, P. D., “Efficient Unsupervised Learning,” inProceedings of the 1988 Workshop on Computational Learning Theory, Morgan Kaufmann, San Mateo, California, August 1988.
Levinson, S. E., Rabiner, L. R., and Sondhi, M. M., “An Introduction to the Application of the Theory of Probabilistic Functions of a Markov Process to Automatic Speech Recognition,”The Bell System Technical Journal,62,4, April 1983.
Nakamura, A. and Abe, N., “Exact Learning of Linear Combinations of Monotone Terms from Function Value Queries,” inTheoretical Computer Science, 137, Elsevier, 1995.
Nakamura, A., Abe, N., and Takeuchi, J., “Efficient Distribution-Free Population Learning of Simple Concepts,” inProceedings of the Fifth International Workshop on Algorithmic Learning Theory, Springer-Verlag, October 1994.
Osherson, D., Stob, M., and Weinstein, S.,Systems that Learn: An Introduction for Cognitive and Computer Scientists, MIT Press, 1986.
Paz, A.,Introduction to Probabilistic Automata, Academic Press, 1971.
Pollard, D.,Convergence of Stochastic Processes, Springer-Verlag, 1984.
Rissanen, J., “Stochastic Complexity and Modeling,”The Annals of Statistics, 14, 3, pp. 1080–1100, 1986.
Rost, B and Sander, C., “Prediction of Protein Secondary Structure at Better Than 70% Accuracy,”J. Mol. Biol., 232, pp. 584–599, 1993.
Sakakibara, Y., Brown, M., Underwood, R. C., Mian, I. S., and Haussler, D., “Stochastic Context-Free Grammars for Modeling RNA,” inProceedings of the 27th Hawaii International Conference on System Sciences, volume V, pp. 284–293, 1994.
Schabes, Y., “Stochastic Lexicalized Tree Adjoining Grammars,” inProceedings of COLING-92, pp. 426–432, 1992.
Sloan, R. H., “Computational Learning Theory: New Models and Algorithms,”Ph.D thesis, MIT, 1989. Issued as MIT/LCS/TR-448.
Sander, C. and Schneider, R., “Database of Homology-Derived Structures and the Structural Meaning of Sequence Alignment,”Proteins: Struct. Funct. Genet., 9, pp. 56–68, 1991.
Vitter, J. and Lin, J., “Learning in Parallel,”Information Computing, pp. 179–202, 1992.
Vijay-Shanker, K. and Joshi, A. K., “Some Computational Properties of Tree Adjoining Grammars,” in23rd Meeting of A. C. L., 1985.
Yamanishi, K., “A Learning Criterion for Stochastic Rules,”Machine Learning, 9, 2/3, a special issue for COLT’90, 1992.
Yamanishi, K. and Konagaya, A., “Learning Stochastic Motifs from Genetic Sequences,” inthe Eighth International Workshop on Machine Learning, 1991.
Author information
Authors and Affiliations
Additional information
Real World Computing Partnership
Naoki Abe: He received the B. S. and M. S. degrees in Computer Science from Massachusetts Institute of Technology in 1984, and obtained the Ph. D. degree in Computer and Information Sciences from the University of Pennsylvania in 1989. He was a post doctoral researcher at the University of California, Santa Cruz from 1989 to 1990, and conducted research in computational learning theory. He joined NEC Corporation in 1990, where he is currently a principal researcher in the C & C Research Laboratories. His research interests include computational learning theory and its application to genetic information processing and natural language processing.
About this article
Cite this article
Abe, N. Towards realistic theories of learning. New Gener Comput 15, 3–25 (1997). https://doi.org/10.1007/BF03037558
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF03037558