Abstract
In view of a great proliferation of machine learning methods and paradigms, there is a need for a general conceptual framework that would explain their interrelationships and provide a basis for their integration into multistrategy learning systems. This article presents initial results on theInferential Theory of Learning that aims at developing such a framework, with the primary emphasis on explaining logical capabilities of learning systems, i.e., theircompetence. The theory views learning as a goal-oriented process of modifying the learner's knowledge by exploring the learner's experience. Such a process is described as a search through aknowledge space, conducted by applying knowledge transformation operators, calledknowledge transmutations. Transmutations can be performed using any type of inference—deduction, induction, or analogy. Several fundamental pairs of transmutations are presented in a novel and very general way. These include generalization and specialization, explanation and prediction, abstraction and concretion, and similization and dissimilization. Generalization and specialization transmutations change thereference set of a description (the set of entities being described). Explanations and predictions derive additional knowledge about the reference set (explanatory or predictive). Abstractions and concretions change the level of detail in describing a reference set. Similizations and dissimilizations hypothesize knowledge about a reference set based on its similarity or dissimilarity with another reference set. The theory provides a basis formultistrategy task-adaptive learning (MTL), which is outlined and illustrated by an example. MTL dynamically adapts strategies to thelearning task, defined by the input information, the learner's background knowledge, and the learning goal. It aims at synergistically integrating a wide range of inferential learning strategies, such as empirical and constructive inductive generalization, deductive generalization, abductive derivation, abstraction, similization, and others.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
Adler, M.J., & Gorman, W. (Eds.). (1987). The great ideas: A synopicon of great books of the western world.Encyclopedia Britannica, Vol. 1, Ch. 39 (Induction), pp 565–571.
Aristotle (1987). Posterior analytics. In R.M. Hutchins (Ed.),The works of Aristotle, Vol. 1.Encyclopedia Britannica. Inc.
Bacon, F. (1620).Novum Organum. In R.M. Hutchins (Ed.),Great Books of the Western World, Vol. 30.Encyclopedia Britannica, 1987.
Bergadano, F., Matwin, S., Michalski, R.S., & Zhang, J. (1992). Learning two-tiered descriptions of flexible concepts: The POSEIDON system.Machine Learning, 8, 5–43. (Originally published inMachine Learning and Inference Reports, No. ML1-3, Center for Artificial Intelligence, George Mason University, Fairfax, VA, September 1990).
Birnbaum, L., & Collins, G. (1991).Proceedings of the 8th International Conference on Machine Learning. Chicago.
Bloedorn, E., & Michalski, R.S. (1991). Data-driven constructive induction.Proceedings of the Tools for Artificial Intelligence Conference. San Jose, CA.
Cohen, L.J. (1970).The implications of induction. London:
Collins, A., & Michalski, R.S. (1989). The logic of plausible-reasoning: A core theory,Cognitive Science, 13 1–49.
Console, L., Theseider, D., & Torasso, P. (1991). On the relationship between abduction and deduction.Journal of Logic and Computation, 1 (5).
Danyluk, A.P. (1987). The use of explanations for similarity-based learning.Proceedings of IJCAI-87 (pp. 274–276). Milan, Italy.
Danyluk, A.P. (1989). Recent results in the use of context for learning new rules. Technical Report No. TR-98-066. Philips Laboratories.
Danyluk, A.p. (1993). Gemini: An integration of analytical and empirical learning. In R.S. Michalski & G. Tecuci (Eds.)Machine learning: A multistrategy approach. Vol.IV.
De Raedt, L., & Bruynooghe, M. (1993). Interactive theory revision. In R.S. Michalski & G. Tecuci (Eds.),Machine learning: A multistrategy approach. Vol.IV. Morgan Kaufmann.
Dietterich, T.G., & Flann, N.S. (1988). An inductive approach to solving the imperfect theory problem.Proceedings of 1988 Symposium on Explanation-Based Learning, (pp. 42–46). Stanford University, 1988.
Dietterich, T.G. (1986). Learning at the knowledge level.Machine Learning, 1 (3), 287–316. (Reprinted in J.W. Shavlik & T.G. Dietterich (Eds.), (1990).Readings in machine learning. San Mateo, CA: Morgan Kaufmann.)
Fulk, M., & Case, J. (1990).Proceedings of the 3rd Annual Workshop on Computational Learning Theory. University of Rochester, NY, August 6–8.
Giordana, A., Saitta, L., & Roverso, D. (1991). Abstracting concepts with inverse resolution.Proceedings of the 8th International Workshop on Machine Learning (pp. 142–146). Evanston, IL:
Goldberg, D.E. (1989).Genetic algorithms in search, optimization, and machine learning. Reading, MA: Addison-Wesley.
Goodman, L.A., & Kruskal, W.H. (1979).Measures of association for cross classifications. New York: Springer-Verlag.
Grosof, B.N., & Russell (1989). Declarative bias for structural domains.Proceedings of the Sixth International Workshop on Machine Learning, Cornell University, Ithaca, NY: Morgan Kaufmann.
Hieb, M., & Michalski, R.S. (1993). A knowledge representation system based on dynamically interlaced hierarchies: Basic ideas and examples.Reports of Machine Learning and Inference Laboratory MLI 93-5, Fairfax, VA: Center for Artificial Intelligence, George Mason University.
Hunter, L. (1990). Planning to learn.Proceedings of the Twelfth Annual Conference of the Cognitive Science Society. (pp. 26–34). Hillsdale, NJ: Lawrence Erlbaum Associates.
Kodratoff, Y., & Michalski, R.S. (Eds.) (1990).Machine learning: An artificial intelligence approach (Vol. 3). Morgan Kaufmann.
Kodratoff, Y., & Tecuci, G. (1987). DISCIPLE-1: Interactive apprentice system in weak theory fields.Proceedings of IJCAI-87 (pp. 271–273). Milan, Italy.
Laird, J.E. (Ed.). (1988).Proceedings of the Fifth International Conference on Machine Learning, University of Michigan, Ann Arbor, June 12–14.
Laird, J.E., Rosenbloom, P.S., & Newell, A. (1986). Chunking in SOAR: the anatomy of a general learning mechanism.Machine Learning, 1(1), 11–46.
Lebowitz, M. (1986). Integrated learning: Controlling explanation.Cognitive Science, 10 (2), 219–240.
Michalski, R.S. (1983). Theory and methodology of inductive learning. In R.S. Michalski, J.G. Carbonell, and T.M. Mitchell (Eds.),Machine learning: An artificial intelligence approach. Tioga.
Michalski, R.S. (1990). Learning flexible concepts: Fundamental ideas and a method based on two-tiered representation. In Y. Kodratoff and R.S. Michalski (Eds.),Machine learning: An artificial intelligence approach, vol. III. Morgan Kaufmann.
Michalski, R.S. (1990a). Toward unified theory of learning: Multistrategy task-adaptive learning.Reports of Machine Learning and Inference Laboratory MLI-90-1. (Subsequently published in B.G. Buchanan and D.C. Wilkins (Eds.),Readings in knowledge acquisition and learning. Morgan Kaufmann.
Michalski, R.S. (1991b). A methodological framework for multistrategy task-adaptive learning.Proceedings of the Fifth International Symposium on Methodologies for intelligent Systems. Knoxville, TN: Elsevier.
Michalski, R.S. (1991). Toward unified theory of learning: An outline of basic ideas. In M. De Glas & D. Gabbay (Eds.),Proceedings of the First World Conference on the Fundamentals of Artificial Intelligence, Paris, France, July 1–5.
Michalski, R.S., & Kodratoff, Y. (1990). Research in machine learning: Recent progress, classification of methods and future directions. In Y. Kodratoff and R.S. Michalski (Eds.),Machine learning: An artificial intelligence approach (Vol. 3). Morgan Kaufmann.
Minton, S. (1988). Quantitative results concerning the utility of explanation-based learning.Proceedings of AAAI-88 (pp. 564–569). Saint Paul, MN.
Minton, S., Carbonell, J.G., Etzioni, O., Knoblock, C.A., & Kuokka, D.R. (1987). Acquiring effective search control rules: Explanation-based learning in the PRODIGY system.Proceedings of the 4th International Machine Learning Workshop (pp. 122–133). University of California, Irvine.
Mitchell, T.M., Keller, T., & Kedar-Cabelli, S. (1986). Explanation-based generalization: A unifying view.Machine learning, 1 (1), 47–80.
Mooney, R.J., & Ourston, D. (1993). A multistrategy approach to theory refinement. In R.S. Michalski & G. Tecuci (Eds.),Machine learning: A multistrategy approach, Vol.IV. Morgan Kaufmann.
Newell, (1981). The Knowledge Level.Al Magazine, 2 1–20.
Muggleton, S. (1988). A strategy for constructing new predicates in first-order logic.Proceedings of EWSL-88 (pp. 123–130). Glasgow, Scotland.
Pazzani, M.J. (1988). Integrating explanation-based and empirical learning methods in OCCAM.Proceedings of EWSL-88 (pp. 147–166). Glasgow, Scotland.
Pierce, C.S. (1965). Elements of logic. In Ch. Hartshorne & P. Weiss (Eds.),Collected papers of Charles Sanders Peirce (1839–1914). Cambridge, MA: The Belknap Press, Harvard University Press.
Pearl, J. (1988).Probabilistic reasoning in intelligent systems: Networks of plausible inference. Morgan Kaufmann.
Piatetsky-Shapiro, G. (1992). Probabilistic data dependencies. In I.M. Zytkow (Ed.),Proceedings of the ML92 Workshop on Machine Discovery, Aberdeen, Scotland, July 4.
Plaisted, D. (1981). Theorem proving with abstraction.Artificial Intelligence, 16 47–108.
Polya, G. (1968).Mathematics and plausible reasoning, Vol. I and II. Princeton, NJ: Princeton University Press.
Poole, D. (1989). Explanation and prediction: An architecture for default and abductive reasoning.Computational Intelligence, 5 97–110.
Popper, K.R. (1972).Objective knowledge: An evolutionary approach. Oxford: Clarendon Press.
Porter, B.W., & Mooney, R.J. (Eds.). (1990).Proceedings of the 7th International Machine Learning Conference, Austin, TX.
Ram, A. (1991). A theory of questions and question asking.The Journal of the Learning Sciences, 1(3&4), 273–318.
Ram, A., & Hunter, L. (in press). The use of explicit goals for knowledge to guide inference and learning.Applied Intelligence. (Also available as Technical Report GIT-CC-92/04, College of Computing, Georgia Institute of Technology, Atlanta, Georgia, January, 1992.)
Rivest, R., Haussler D., & Warmuth, M. (1989).Proceedings of the Second Annual Workshop on Computational Learning Theory, University of Santa Cruz, July 31-August 2.
Russell, S. (1989).The use of knowledge in analogy and induction. San Mateo, CA: Morgan Kaufmann.
Schafer, D. (Ed.). (1989).Proceedings of the 3rd International Conference on Genertic Algorithms. Fairfax, VA: George Mason University.
Schultz, T.R., & Kestenbaum, N.R. (1985). Causal reasoning in children. In G.J. Whitehurst (Ed.),Annals of Child development, vol. 2, pp. 195–249. JAI Press.
Segre, A.M. (Ed.). (1989).Proceedings of the Sixth International Workshop on Machine Learning, Cornell University. Ithaca, NY: Morgan Kaufmann.
Sleeman, D., & Edwards, P. (1992).Proceedings of the Ninth International Workshop. Great Britain: Aberdeen, Morgan Kaufmann.
Stepp, R.S., & Michalski, R.S. (1993). How to structure structured objects.Proceedings of the International Machine Learning Workshop. (pp. 156–160). Urbana, IL: University of Illinois Allerton House.
Tecuci, G., & Michalski, R.S. (1991a). A method for multistrategy task-adaptive learning based on plausible justifications. In L. Birnbaum & G. Collins (Eds.).Machine Learning: Proceedings of the eighth international workshop. San Matteo, CA: Morgan Kaufmann.
Tecuci, G., & Michalski, R.S. (1991b). Input ‘understanding’ as a basis for multistrategy task-adaptive learning. In Z. Ras & M. Zemankova (Eds.),Proceedings of the 6th International Symposium on Methodologies for Intelligent Systems. Lecture Notes on Artificial Intelligence. Berlin: Springer Verlag.
Tecuci, G. (in press). Plausible justification trees: A framework for deep and dynamic integration of learning strategies.Machine Learning Journal (Special Issue on Multistrategy Learning).
Touretzky, D., Hinton, G., & Sejnowski, T. (Eds.). (1988).Proceedings of the 1988 Connectionist Models Summer School, Carnegie Mellon University, June 17–26.
Utgoff, P. (1986). Shift of bias for inductive concept learning. In R.S. Michalski, J.G. Carbonell, & T.M. Mitchell (Eds.),Machine learning: An artificial intelligence approach (Vol. 2). Morgan Kaufmann.
Warmuth, M., & Valiant, L. (Eds.). (1991).Proceedings of the 4th Annual Workshop on Computational Learning Theory, Santa Cruz, CA: Morgan Kaufmann.
Whewell, W. (1857).History of the inductive sciences, 3rd edition. London.
Whitehall, B.L. (1990). Knowledge-based learning: Integration and deductive and inductive learning for knowledge base completion. Ph.D. Thesis, Computer Science Department, University of Illinois.
Whitehall, B.L., & Lu, S. C-Y. Theory Completion using Knowledge-Based Learning, In R.S. Michalski & G. Tecuci (Eds.),Machine learning: A multistrategy approach, Vol. IV. Morgan Kaufmann.
Wilkins, D.C., Clancey, W.J., & Buchanan, B.G. (1986).An overview of the odysseus learning apprentice. New York: Kluwer Academic.
Wnek, J., & Michalski, R.S. (1991a). Hypothesis-driven constructive induction in AQ17: A method and experiments. In K. Morik, F. Bergadano, and W. Buntine (Eds.), InProceedings of the IJCAI-91 Workshop on Evaluating and Changing Representation in Machine Learning, (pp. 13–22). Sydney, Australia, August 24–30.
Wnek, J., & Michalski, R.S. (1991b). An experimental comparison of symbolic and subsymbolic learning paradigms: Phase I-Learning logic-style concepts. In R.S. Michalski & G. Tecuci (Eds.),Proceedings of the First International Workshop on Multistrategy Learning. GMU Center for Artificial Intelligence, Harpers Ferry, Nov. 7–9.
Wnek, J., & Michalski, R.S. (1993). Comparing Symbolic and Subsymbolic Learning: Three Studies. In R.S. Michalski & G. Tecuci (Eds.),Machine learning: A multistrategy approach, Vol. IV. Los Altos, CA: Morgan Kaufmann.
Zadrozny, W. (1990). The logic of abduction (preliminary report).First International Workshop on Principles of Diagnosis, Stanford, CA.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Michalski, R.S. Inferential theory of learning as a conceptual basis for multistrategy learning. Mach Learn 11, 111–151 (1993). https://doi.org/10.1007/BF00993074
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.1007/BF00993074