Abstract
The problem of valid induction could be stated as follows: are we justified in accepting a given hypothesis on the basis of observations that frequently confirm it? The present paper argues that this question is relevant for the understanding of Machine Learning, but insufficient. Recent research in inductive reasoning has prompted another, more fundamental question: there is not just one given rule to be tested, there are a large number of possible rules, and many of these are somehow confirmed by the data — how are we to restrict the space of inductive hypotheses and choose effectively some rules that will probably perform well on future examples? We analyze if and how this problem is approached in standard accounts of induction and show the difficulties that are present. Finally, we suggest that the explanation-based learning approach and related methods of knowledge intensive induction could be, if not a solution, at least a tool for solving some of these problems.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bergadano, F., Giordana, A., and Saitta, L. (1991),Machine Learning: An Integrated Framework and Its Applications, Ellis Horwood Publishers, Chichester, UK.
Bergadano, F., Matwin, S., Michalski, R. S., and Zhang, J. (1992). ‘Learning Two-Tiered Descriptions of Flexible Concepts: the Poseidon System’,Machine Learning 8, pp. 5–43.
Bergadano, F. (1991), ‘The Problem of Induction and Machine Learning’, inProc. Int. Joint Conf. on Artificial Intelligence, Sydney, Australia, pp. 1073–1079.
Blumer, A., Ehrenfeucht, A., Haussler, D., and Warmuth, M. (1987), ‘Occam's Razor’,Information Processing Letters 24, pp. 377–380.
Buntine, W. (1989), ‘A Critique of the Valiant Model’, inProc. of the IJCAI, Detroit, MI, pp. 837–842.
Carnap, R. (1950),Logical Foundations of Probability, The University of Chicago Press.
Carnap, R. (1952),The Continuum of Inductive Methods, The University of Chicago Press.
Carnap, R. (1956), ‘Meaning Postulates’, inMeaning and Necessity, The University of Chicago Press.
Devroye, L. (1988), ‘Automatic Pattern Recognition: A Study of the Probability of Error’,IEEE Trans. on PAMI 10(4), pp. 530–543.
Edwards, W., Lindman, H., and Savage, L. J. (1963), ‘Inference for Psychological Research’,Psychological Review 70, pp. 193–242.
Haussler, D. (1988), ‘Quantifying Inductive Bias — AI Learning Algorithms and Valiant's Learning Framework’,Artificial Intelligence 36, pp. 177–221.
Hoeffding, W. (1963), ‘Probability Inequalities for Sums of Bounded Random Variables’,Journal of the American Statistical Association 58, pp. 13–30.
Howson, C. and Urbach, P. (1989),Scientific Reasoning — the Bayesian Approach, Open Court, Illinois.
Kearns, M., Li, M., Pitt, L., and Valiant, L. (1987), ‘On the Learnability of Boolean Formulae’, inProc. of the Fourth Machine Learning Workshop, Irvine, CA, pp. 285–295.
Kyburg, H. (1974),The Logical Foundations of Statistical Inference, D. Reidel Publishing Company.
Lindley, D. V. (1990), ‘The Present Position in Bayesian Statistics’,Statistical Science 5(1), pp. 44–65.
Mitchell, T., Keller, R. M., and Kedar-Cabelli, S. (1986), ‘Explanation-Based Generalization: a Unifying View’,Machine Learning 1, pp. 47–80.
Pearl, J. (1979), ‘Capacity and Error Estimates for Boolean Classifiers with Limited Complexity’,IEEE Trans. on PAMI 1(4), pp. 350–355.
Russell, S. (1988), ‘Tree-Structured Bias’, inProc. AAAI Conference, pp. 641–645.
Russell, S. (1991) ‘Inductive Learning by Machines’,Philosophical Studies 64(1), pp. 37–64.
Saitta, L. and Bergadano, F. (1992), ‘Error Probability and Valiant's Learning Framework’,IEEE Trans. on Pattern Analysis and Machine Intelligence, to appear.
Uspensky, J. V. (1974),Introduction to Mathematical Probability, McGraw Hill, New York.
Valiant, L. G. (1984a), ‘A Theory of the Learnable’,Communications of the ACM 27(11), pp. 1134–1142.
Valiant, L. G. (1984b), ‘Deductive Learning’,Phil. Trans. Royal Society of London 312, pp. 441–446.
Vapnik, V. (1982),Estimation of Dependencies Based on Empirical Data, Springer Verlag, New York.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Bergadano, F. Machine Learning and the foundations of inductive inference. Mind Mach 3, 31–51 (1993). https://doi.org/10.1007/BF00974304
Issue Date:
DOI: https://doi.org/10.1007/BF00974304