Abstract
Gold showed in 1967 that not even regular grammars can be exactly identified from positive examples alone. Since it is known that children learn natural grammars almost exclusively from positives examples, Gold's result has been used as a theoretical support for Chomsky's theory of innate human linguistic abilities. In this paper new results are presented which show that within a Bayesian framework not only grammars, but also logic programs are learnable with arbitrarily low expected error from positive examples only. In addition, we show that the upper bound for expected error of a learner which maximises the Bayes' posterior probability when learning from positive examples is within a small additive term of one which does the same from a mixture of positive and negative examples. An Inductive Logic Programming implementation is described which avoids the pitfalls of greedy search by global optimisation of this function during the local construction of individual clauses of the hypothesis. Results of testing this implementation on artificially-generated data-sets are reported. These results are in agreement with the theoretical predictions.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
References
D. Angluin. Inference of reversible languages. Journal of the ACM, 29:741–765, 1982.
A.W. Biermann and J.A. Feldman. On the synthesis of finite-state machines from samples of their behaviour. IEEE Transactions on Computers, C(21):592–597, 1972.
W. Buntine. A Theory of Learning Classification Rules. PhD thesis, School of Computing Science, University of Technology, Sydney, 1990.
N. Chomsky. Knowledge of language: its nature, origin and use. Praeger, New York, 1986. First published 1965.
E.M. Gold. Language identification in the limit. Information and Control, 10:447–474, 1967.
D. Haussler, M Kearns, and R. Shapire. Bounds on the sample complexity of Bayesian learning using information theory and the VC dimension. In COLT91: Proceedings of the 4th Annual Workshop on Computational Learning Theory, pages 61–74, San Mateo, CA, 1991. Morgan Kauffmann.
D. Haussler, M Kearns, and R. Shapire. Bounds on the sample complexity of Bayesian learning using information theory and the VC dimension. Machine Learning Journal, 14(1):83–113, 1994.
R.J. Mooney and M.E. Califf. Induction of first-order decision lists: Results on learning the past tense of english verbs. Journal of Artificial Intelligence Research, 3:1–24, 1995.
S. Muggleton. Bayesian inductive logic programming. In M. Warmuth, editor, Proceedings of the Seventh Annual ACM Conference on Computational Learning Theory, pages 3–11, New York, 1994. ACM Press.
S. Muggleton. Inverse entailment and Progol. New Generation Computing, 13:245–286, 1995.
S. Muggleton. Stochastic logic programs. In L. De Raedt, editor, Advances in Inductive Logic Programming. IOS Press/Ohmsha, 1996.
S. Muggleton, M.E. Bain, J. Hayes-Michie, and D. Michie. An experimental comparison of human and machine learning formalisms. In Proceedings of the Sixth International Workshop on Machine Learning, Los Altos, CA, 1989. Kaufmann.
S. Muggleton and C.D. Page. A learnability model for universal representations. Technical Report PRG-TR-3-94, Oxford University Computing Laboratory, Oxford, 1994.
S. Pinker. Language learnability and language development. Harvard University Press, Cambridge, Mass., 1984.
G.D. Plotkin. A note on inductive generalisation. In B. Meltzer and D. Michie, editors, Machine Intelligence 5, pages 153–163. Edinburgh University Press, Edinburgh, 1969.
J.R. Quinlan and R.M. Cameron. Induction of logic programs: FOIL and related systems. New Generation Computing, 13:287–312, 1995.
L. De Raedt and M. Bruynooghe. A theory of clausal discovery. In Proceedings of the 13th International Joint Conference on Artificial Intelligence. Morgan Kaufmann, 1993.
T. Shinohara. Inductive inference of monotonic formal systems from positive data. In Proceedings of the first international workshop on algorithmic learning theory, Tokyo, 1990. Ohmsha.
L.G. Valiant. A theory of the learnable. Communications of the ACM, 27:1134–1142, 1984.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1997 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Muggleton, S. (1997). Learning from positive data. In: Muggleton, S. (eds) Inductive Logic Programming. ILP 1996. Lecture Notes in Computer Science, vol 1314. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-63494-0_65
Download citation
DOI: https://doi.org/10.1007/3-540-63494-0_65
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-63494-2
Online ISBN: 978-3-540-69583-7
eBook Packages: Springer Book Archive