Abstract
The quality of theories produced with the help of machine learning algorithms is usually measured in terms of accuracy and coverage. This paper reopens the issue of understandability of induced theories, which, while prominent in the early days of ML, seems to have fallen from favor in the sequel. This issue is especially relevant in the broader context of using ML as an aide in design and maintenance of knowledge bases for knowledge based systems. The guiding question is: beyond accuracy, what constitutes a good theory? An attempt at surveying relevant work in the fields of linguistics and cognitive psychology is made. The sympathetic reader will find this somewhat motivates the author's personal intuitions about the quality of a theory, hinging on understandability. These intuitions, in turn, point toward some simple criteria that may help in measuring quality. By way of consolation for those who do not share the author's intuitions, the criteria proposed here are objective in the sense that the measurements they provide may be evaluated from a number of contrary perspectives. Some empirical results are given in the context of theory restructuring: redundancy elimination and introduction of new intermediate concepts.
This is a preview of subscription content, log in via an institution.
Preview
Unable to display preview. Download preview PDF.
References
L. W. Barsalou. Ad hoc categories. Memory & Cognition, 11(3):211–227, 1983.
Bernd Ulrich Biere. Verständlich-Machen. Max Niemeyer Verlag, Tübingen, 1989.
M. Bock. Wort-, Satz-, Textverarbeitung. Kohlhammer, Stuttgart, 1978.
Maurice Bruynooghe. Adding redundancy to obtain more reliable and readable Prolog programs. In Michel Van Caneghem (ed.), Proceedings of the First International Logic Programming Conference, pp. 129–133, Marseille, France, 1982. ADDP-GIA.
W. Buntine. Generalized Subsumption and its Applications to Induction and Redundancy. Artificial Intelligence, 36:149–176, 1988.
Jaime G. Carbonell. Paradigms for Machine Learning. Artificial Intelligence, 40:1–9, 1989.
Werner Emde. An Inference Engine for Representing Multiple Theories. In K. Monk (ed.), Knowledge Representation and Organization in Machine Learning, pp. 148–176. Springer, New York, Berlin, Tokyo, 1989. Also: KIT-Report Nr. 64, TU Berlin, 1988.
Werner Emde. Modellbildung, Wissensrevision und Wissensrepräsentation im Maschinellen Lernen. Informatik-Fachberichte 281. Springer Verlag, Berlin, New York, 1991. PhD thesis.
Werner Emde. Inductive Learning from very few Classified Examples. In Proc. 7th European Conference on Machine Learning (ECML-94), 1994. also available as GMD Tech. Report.
Li-Min Fu and Brace Buchanan. Learning Intermediate Concepts in Constructing a Hierarchical Knowledge Base. In Proc. 9th International Joint Conference on Artificial Intelligence, pp. 659–666, San Mateo, CA, 1985. Morgan Kaufman.
M. A. Gernsbacher. Language Comprehension as Structure Building. Lawrence Erlbaum, Hillsdale NJ, 1990.
Hans Hörmann. To Mean — To Understand: Problems of Psychological Semantics. Springer-Verlag, 1981.
Jörg-Uwe Kietz and Stefan Wrobel. Controlling the Complexity of Learning in Logic through Syntactic and Task-Oriented Models. In Stephen Muggleton (ed.), Proc. 1st Int. Workshop on ILP, pp. 107–126, Viana de Castelo, Portugal, 1991. Also in S.Muggleton (ed.), Inductive Logic Programming, Academic Press, 1992.
W. Kintsch. The Representation of Meaning in Memory. Erlbaum, Potomac, MD, 1974.
W. Kintsch and J Keenan. Reading rate and retention as a function of the number of propositions in the base structure of sentences. Cognitive Psychology, 5:257–274, 1973.
Yves Kodratoff. Guest Editor's Introduction (The Comprehensibility Manifesto). AI Communications, 7(2):83–85, 1994.
Franz von Kutschera. Elementare Logik. Wien, 1967.
Franz von Kutschera. Wissenschaftstheorie, volume II. Wilhelm Fink Verlag, München, 1972.
A. M. Lesgold. Pronominalization: a Device for Unifying Sentences in Memory. Journal of Verbal Learning and Verbal Behavior, 11:316–323, 1972.
G. Mandler. Organisation and Memory. In K. W. Spence and J. T. Spence (eds.), The Psychology of Learning and Motivation, volume 2, pp. 189–196. Academic Press, New York, 1967.
Ryszard S. Michalski. A Theory and Methodology of Inductive Learning. In Machine Learning — An Artificial Intelligence Approach, volume I, pp. 83–134. Morgan Kaufman, San Mateo, CA, 1983.
Donald Michie. The superarticulacy phenomenon in the context of software manufacture. Proceedings of the Royal Society of London, A 405:185–212, 1986.
K. Morik, S. Wrobel, Jörg-Uwe Kietz, and W. Emde. Knowledge Acquisition and Machine Learning. Academic Press, London, 1993.
Stephen Muggleton. Structuring Knowledge by Asking Questions. In Ivan Bratko and Nada Lavrač(eds.), Progress in Machine Learning—Proc. Second European Working Session on Learning (EWSL), Wilmslow, UK, 1987. Sigma Press.
Stephen Muggleton and Wray Buntine. Machine Invention of First-Order Predicates by Inverting Resolution. In Proc. Fifth Intern. Conf. on Machine Learning, San Mateo, CA, 1988. Morgan Kaufman.
Stephen Muggleton and Luc deRaedt. Inductive Logic Programming: Theory and Methods. Journal of Logic Programming, 19/20:629–680, 1994.
Stephen Muggleton and Cao Feng. Efficient Induction of Logic Programs. In Proc. First Conf. on Algorithmic Learning Theory, Tokyo, 1990. Ohmsha Publishers.
Claire Nedellec (ed.). Proc. IJCAI Workshop on Machine Learning and Comprehensibility, Claire.Nedellec@lri.fr, 1995.
G. Piatetsky-Shapiro and W. Frawley. Knowledge discovery in databases. The MIT press, 1991. (editors).
Gordon D. Plotkin. A note on inductive generalization. In B. Meltzer and D. Michie (eds.), Machine Intelligence, volume 5, chapter 8, pp. 153–163. American Elsevier, 1970.
Karl Popper. Die beiden Grundprobleme der Erkenntnistheorie: aufgrund von Ms. aus d. Jahren 1930–1933, volume 18 of Die Einheit der Gesellschaftswissenschaften. Mohr, Tübingen, 1933. edited by Troels Eggers Hansen, appeared 1979.
J. Ross Quinlan. Learning Logical Definitions from Relations. Machine Learning, 5(3):239–266, 1990.
A. J. Sanford and S. C. Garrod. Understanding Written Language. Wiley and Sons, Chichester, 1981.
P. Smolensky. Connectionist AI, Symbolic AI, and the Brain. Artificial Intelligence Review, 1:95–109, 1987.
E. Sommer, K. Morik, J.M. Andre, and M. Uszynski. What On-line Learning Can Do for Knowledge Acquisition. Knowledge Acquisition, 6:435–460, 1994.
E. Sommer, Werner Emde, Jörg-Uwe Kietz, and Stefan Wrobel. Mobal 42 User Guide ((always) Draft). Arbeitspapiere der gmd, GMD, 1996. Available via WWW http://nathan.gmd.de/projects/ml/lit/mlpublist.html.
E. Sommer. Learning Relations without Closing the World. In Proc. of the European Conference on Machine Learning (ECML-94), Berlin, 1994. Springer-Verlag.
E. Sommer. Restructuring in Horn clause knowledge bases. Technical report, ESPRIT Project ILP (6020), 1994. ILP Deliverable GMD 2.1.
E. Sommer. Rulebase Stratification: an Approach to theory restructuring. In Proc. 4th Intl. Workshop on Inductive Logic Programming (ILP-94), 1994. Available via WWW http://nathan.gmd.de/projects/ml/lit/mlpublist.html.
E. Sommer. An Approach to Quantifying the Quality of Induced Theories. In Claire Nedellec (ed.), Proc. IJCAI Workshop on Machine Learning and Comprehensibility, 1995. Available via WWW http: //nathan.gmd.de/projects/ml/lit/mlpublist. html.
E. Sommer. Fender: An approach to theory restructuring. In Stefan Wrobel and Nada Lavrac (eds.), Proc. of the European Conference on Machine Learning (ECML-95), volume 912 of Lecture Notes in Artificial Intelligence, Berlin, 1995. Springer-Verlag.
E. Sommer. Mobal's theory restructuring tool RRT. Technical report, ESPRIT Project ILP (6020), 1995. ILP Deliverable GMD 2.2.
E. Sommer. Theory Restructuring. NN, 1996. (submitted).
Stefan Wrobel. Concept Formation and Knowledge Revision. Kluwer Academic Publishers, Dordrecht, Netherlands, 1994.
L. S. Wygotski. Denken und Sprechen. Conditio humana. S. Fischer, 1964. (First published in Russian 1934, english translation “Thought and language” 1962 by MIT Press).
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1996 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Sommer, E. (1996). An approach to measuring theory quality. In: Shadbolt, N., O'Hara, K., Schreiber, G. (eds) Advances in Knowledge Acquisition. EKAW 1996. Lecture Notes in Computer Science, vol 1076. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-61273-4_13
Download citation
DOI: https://doi.org/10.1007/3-540-61273-4_13
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-61273-5
Online ISBN: 978-3-540-68391-9
eBook Packages: Springer Book Archive