SDL, a stochastic algorithm for learning decision lists with limited complexity

Abstract

This paper deals with learning decision lists from examples. In real world problems, data are often noisy and imperfectly described. It is commonly acknowledged that in such cases, consistent but inevitably complex classification procedures usually cause overfitting: results are perfect on the learning set but worse on new examples. Therefore, one searches for less complex procedures which are almost consistent or, in other words, for a good compromise between complexity and goodness-of-fit. But such a requirement generally involves NP-completeness. In a way, CN2 provides a greedy approach to the problem. In this paper, we propose to search the solution space more extensively, using a stochastic procedure, an association of simulated annealing (SA) and simple tabu search (TS) in two distinct phases. In the first phase, we use SA to diversify the search. In the second phase, TS intensifies the search. We compare CART, CN2, and our method using natural and artificial domains.