Layerwise Learning of Mixed Conjunctive and Disjunctive Rule Sets

Abstract

Conventional rule learning algorithms learn a description of the positive class in disjunctive normal form (DNF). Alternatively, there are also a few learners who can formulate their model in conjunctive normal form (CNF) instead. While it is clear that both representations are equally expressive, there are domains where DNF learners perform better and others where CNF learners perform better. Thus, an algorithm that can dynamically make use of the best of both worlds is certainly desirable. In this paper, we propose the algorithm Cord that can learn general logical functions by training alternating layers of conjunctive and disjunctive rule sets, using any conventional rule learner. In each layer, the conjunctions/disjunctions trained in the previous layer are used as input features for learning a CNF/DNF expression that forms the next layer. In our experiments on real-world benchmark data, Cord outperformed both state-of-the-art CNF and DNF learners, where the best final performance was typically achieved using a high number of intermediate, general concepts in early layers that were refined in later layers, underlining the importance of more flexible and deeper concept representations.