Abstract
Features are the main rule building blocks for rule learning algorithms. They can be simple tests for attribute values or complex logical terms representing available domain knowledge. In contrast to common practice in classification rule learning, we argue that the separation of feature construction and rule construction processes has a theoretical and practical justification. Explicit usage of features enables a unifying framework of both propositional and relational rule learning and we present and analyze procedures for feature construction in both types of domains. It is demonstrated that the presented procedure for constructing a set of simple features has the property that the resulting feature set enables the construction of complete and consistent rules whenever possible, and that the set does not include obviously irrelevant features. It is also shown that feature relevancy may improve the effectiveness of rule learning. It this work, we illustrate the relevancy concept in the coverage space, and show that the transformation from the attribute to the feature space enables a novel, theoretically justified way of handling unknown attribute values. The same approach enables that the estimated imprecision of continuous attributes can be taken into account, resulting in the construction of features that are robust to attribute imprecision.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Bergadano, F., Matwin, S., Michalski, R.S., Zhang, J.: Learning two-tiered descriptions of flexible concepts: The POSEIDON system. Machine Learning 8, 5–43 (1992)
Bruha, I., Franek, F.: Comparison of various routines for unknown attribute value processing: The covering paradigm. International Journal of Pattern Recognition and Artificial Intelligence 10(8), 939–955 (1996)
Clark, P., Niblett, T.: The CN2 induction algorithm. Machine Learning 3(4), 261–283 (1989)
Cohen, W.W.: Fast effective rule induction. In: Prieditis, A., Russell, S. (eds.) Proceedings of the 12th International Conference on Machine Learning (ML 1995), Lake Tahoe, CA, pp. 115–123. Morgan Kaufmann, San Francisco (1995)
Fayyad, U.M., Irani, K.B.: On the handling of continuous-valued attributes in decision tree generation. Machine Learning 8(2), 87–102 (1992)
Flach, P., Lachiche, N.: 1BC: A first-order bayesian classifier. In: Džeroski, S., Flach, P.A. (eds.) ILP 1999. LNCS (LNAI), vol. 1634, pp. 92–103. Springer, Heidelberg (1999)
Fürnkranz, J.: Separate-and-conquer rule learning. Artificial Intelligence Review 13(1), 3–54 (1999)
Fürnkranz, J., Flach, P.A.: ROC ’n’ rule learning – Towards a better understanding of covering algorithms. Machine Learning 58(1), 39–77 (2005)
Gamberger, D., Lavrač, N.: Expert-guided subgroup discovery: Methodology and application. Journal of Artificial Intelligence Research 17, 501–527 (2002)
Golub, T., Slonim, D., Tamayo, P., Huard, C., Gaaseenbeek, M., Mesirov, J., Coller, H., Loh, M., Downing, J., Caligiuri, M., Bloomfield, C., Lander, E.: Molecular classification of cancer: Class discovery and class prediction by gene expression monitoring. Science 286, 531–537 (1999)
Idestam-Almquist, P.: Generalization of clauses under implication. Journal of Artificial Intelligence Research 3, 467–489 (1995)
Idestam-Almquist, P.: Generalization of clauses. PhD thesis, Stockholm University, Department of Computer and Systems Sciences (1993)
Kaufman, K.A., Michalski, R.S.: An adjustable rule learner for pattern discovery using the AQ methodology. Journal of Intelligent Information Systems 14, 199–216 (2000)
Larson, J.B., Michalski, R.S.: Inductive inference of VL decision rules. SIGART Newsletter 63, 38–44 (1977)
Lavrač, N., Džeroski, S.: Inductive Logic Programming: Techniques and Applications. Ellis Horwood (1994)
Lavrač, N., Flach, P.: An extended transformation approach to inductive logic programming. ACM Transactions on Computational Logic 2(4), 458–494 (2001)
Lavrač, N., Džeroski, S., Grobelnik, M.: Learning nonrecursive definitions of relations with LINUS. In: Kodratoff, Y. (ed.) EWSL 1991. LNCS, vol. 482, pp. 265–281. Springer, Heidelberg (1991)
Lavrač, N., Gamberger, D., Jovanoski, V.: A sudy of relevance for learning in deductive databases. Journal of Logic Programming 40(2/3), 215–249 (1999)
Matheus, C.J.: A constructive induction framework. In: Proceedings of the 6th International Workshop on Machine Learning, pp. 474–475 (1989)
Mehra, P., Rendell, L., Wah, B.: Principled constructive induction. In: Proceedings of the 11th International Joint Conference on Artificial Intelligence (IJCAI 1989), pp. 651–656. Morgan Kaufmann, San Francisco (1989)
Michalski, R.S.: On the quasi-minimal solution of the covering problem. In: Proceedings of the 5th International Symposium on Information Processing (FCIP 1969), Bled, Yugoslavia. Switching Circuits, vol. A3, pp. 125–128 (1969)
Michalski, R.S.: Pattern recognition and rule-guided inference. IEEE Transactions on Pattern Analysis and Machine Intelligence 2, 349–361 (1980)
Michalski, R.S.: A theory and methodology of inductive learning. Artificial Intelligence 20(2), 111–162 (1983)
Michalski, R.S.: AQVAL/1 — computer implementation of a variable-valued logic system VL1 and examples of its application to pattern recognition. In: Proceedings of the 1st International Joint Conference on Pattern Recognition, pp. 3–17 (1973a)
Michalski, R.S.: Discovering classification rules using variable-valued logic system vl1. In: Proceedings of the 3rd International Joint Conference on Artificial Intelligence (IJCAI 1973), Stanford, CA, pp. 162–172 (1973b)
Michalski, R.S., Larson, J.B.: Selection of most representative training examples and incremental generation of vl1 hypotheses: the underlying methodology and the description of programs ESEL and AQ11. Technical Report 78-867, Department of Computer Science, University of Illinois at Urbana-Champaign (1978)
Michalski, R.S., Mozetič, I., Hong, J., Lavrač, N.: The multi-purpose incremental learning system AQ15 and its testing application to three medical domains. In: Proceedings of the 5th National Conference on Artificial Intelligence (AAAI 1986), Philadelphia, PA, pp. 1041–1045 (1986)
Provost, F., Fawcett, T.: Robust classification for imprecise environments. Machine Learning 42, 203–231 (2001)
Quinlan, J.R.: Induction of decision trees. Machine Learning 1, 81–106 (1986)
Quinlan, J.R.: Learning logical definitions from relations. Machine Learning 5, 239–266 (1990)
Wojtusiak, J., Michalski, R.S., Kaufman, K.A., Pietrzykowski, J.: The AQ21 natural induction program for pattern discovery: Initial version and its novel features. In: Proceedings of The 18th IEEE International Conference on Tools with Artificial Intelligence, Cincinnati, OH (2007)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Lavrač, N., Fürnkranz, J., Gamberger, D. (2010). Explicit Feature Construction and Manipulation for Covering Rule Learning Algorithms. In: Koronacki, J., Raś, Z.W., Wierzchoń, S.T., Kacprzyk, J. (eds) Advances in Machine Learning I. Studies in Computational Intelligence, vol 262. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-05177-7_6
Download citation
DOI: https://doi.org/10.1007/978-3-642-05177-7_6
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-05176-0
Online ISBN: 978-3-642-05177-7
eBook Packages: EngineeringEngineering (R0)