Abstract
Online learning algorithms such as Winnow have received much attention in Machine Learning. Their performance degrades only logarithmically with the input dimension, making them useful in large spaces such as relational theories. However, online first-order learners are intrinsically limited by a computational barrier: even in the finite, function-free case, the number of possible features grows exponentially with the number of first-order atoms generated from the vocabulary. To circumvent this issue, we exploit the paradigm of closure-based learning which allows the learner to focus on the features that lie in the closure space generated from the examples which have lead to a mistake. Based on this idea, we develop an online algorithm for learning theories formed by disjunctions of existentially quantified conjunctions of atoms. In this setting, we show that the number of mistakes depends only logarithmically on the number of features. Furthermore, the computational cost is essentially bounded by the size of the closure lattice.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Abiteboul, S., Hull, R., Vianu, V.: Foundations of Databases. Addison-Wesley, Reading (1995)
Birkhoff, G.: Lattice Theory, 3rd edn. American Mathematical Society, Providence (1967)
Blum, A.: Learning boolean functions in an infinite attribute space. Machine Learning 9(4), 373–386 (1992)
Blum, A.: On-line algorithms in machine learning. In: Fiat, A. (ed.) Dagstuhl Seminar 1996. LNCS, vol. 1442, pp. 306–325. Springer, Heidelberg (1998)
Carpineto, C., Romano, G., d’Amado, P.: Inferring dependencies from relations: a conceptual clustering approach. Computational Intelligence 15(4), 415–441 (1999)
Chawla, D., Li, L., Scott, S.: Efficiently approximating weighted sums with exponentially many terms. In: Proceedings of the 14th Annual Conference on Computational Learning Theory, pp. 82–98 (2001)
Cumby, C.M., Roth, D.: Relational representations that facilitate learning. In: Principles of Knowledge Representation and Reasoning: Proceedings of the 7th International Conference, pp. 425–434 (2000)
Ganter, B., Wille, R.: Formal Concept Analysis: Mathematical Foundations. Springer, Heidelberg (1997)
Godin, R., Missaoui, R., Alaoui, H.: Incremental concept formation algorithms based on Galois lattices. Computational Intelligence 11, 246–267 (1995)
Golding, A.R., Roth, D.: A Winnow-based approach to context-sensitive spelling correction. Machine Learning 34, 107–130 (1999)
Goldman, S.A., Kwek, S., Scott, S.D.: Agnostic learning of geometric patterns. Journal of Computer and System Sciences 62(1), 123–151 (2001)
Haussler, D.: Learning conjunctive concepts in structural domains. Machine Learning 4, 7–40 (1989)
Khardon, R.: Learning horn expressions with LogAn-H. In: Proceedings of the 17th International Conference on Machine Learning, pp. 471–478 (2000)
Khardon, R., Roth, D., Servedio, R.A.: Efficiency versus convergence of Boolean kernels for on-line learning algorithms. Advances in Neural Information Processing Systems 14, 423–430 (2001)
Kuznetsov, S.: On computing the size of a lattice and related decision problems. Order 18(4), 313–321 (2001)
Littlestone, N.: Learning quickly when irrelevant attributes abound: A new linearthreshold algorithm. Machine Learning 2(4), 285–318 (1988)
Maass, W., Warmuth, M.K.: Efficient learning with virtual threshold gates. Information and Computation 141(1), 66–83 (1998)
Mitchell, T.M.: Generalization as search. Artificial Intelligence 18(2), 203–226 (1982)
Roth, D., Yang, M.-H., Ahuja, N.: Learning to recognize three-dimensional objects. Neural Computation 14(5), 1071–1103 (2002)
Roth, D., Yih, W.: Relational learning via propositional algorithms: An information extraction case study. In: Proceedings of the 17th International Joint Conference on Artificial Intelligence, pp. 1257–1263 (2001)
Valiant, L.G.: Projection learning. Machine Learning 37(2), 115–130 (1999)
Valiant, L.G.: Robust logics. Artificial Intelligence 117(2), 231–253 (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Koriche, F. (2005). Online Closure-Based Learning of Relational Theories. In: Kramer, S., Pfahringer, B. (eds) Inductive Logic Programming. ILP 2005. Lecture Notes in Computer Science(), vol 3625. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11536314_11
Download citation
DOI: https://doi.org/10.1007/11536314_11
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-28177-1
Online ISBN: 978-3-540-31851-4
eBook Packages: Computer ScienceComputer Science (R0)