Learning rules from incomplete training examples by rough sets

Abstract

Machine learning can extract desired knowledge from existing training examples and ease the development bottleneck in building expert systems. Most learning approaches derive rules from complete data sets. If some attribute values are unknown in a data set, it is called incomplete. Learning from incomplete data sets is usually more difficult than learning from complete data sets. In the past, the rough-set theory was widely used in dealing with data classification problems. In this paper, we deal with the problem of producing a set of certain and possible rules from incomplete data sets based on rough sets. A new learning algorithm is proposed, which can simultaneously derive rules from incomplete data sets and estimate the missing values in the learning process. Unknown values are first assumed to be any possible values and are gradually refined according to the incomplete lower and upper approximations derived from the given training examples. The examples and the approximations then interact on each other to derive certain and possible rules and to estimate appropriate unknown values. The rules derived can then serve as knowledge concerning the incomplete data set.

Introduction

Expert systems have been widely used in domains where mathematical models cannot be easily built, human experts are not available or the cost of querying an expert is high. Although a wide variety of expert systems have been built, knowledge acquisition remains a development bottleneck. Usually, a knowledge engineer is needed to establish a dialog with a human expert and to encode the knowledge elicited into a knowledge base to produce an expert system. The process is however very time-consuming (Buchanan and Shortliffe, 1984, Giarratano and Riley, 1989). Shortening the development time is then the most important factor for the success of an expert system.

Recently, machine-learning techniques have been developed to ease the knowledge-acquisition bottleneck. Among proposed approaches, deriving rules from training examples is the most common (Hong et al, 2001, Hong et al., 2000, Kodratoff and Michalski, 1983, Michalski et al., 1983, Michalski et al., 1984, Tsumoto, 1998). Given a set of examples, a learning program tries to induce rules that describe each class.

Most learning approaches derive rules from complete data sets. If some attribute values are unknown in a data set, it is called incomplete. Learning from incomplete data sets is usually more difficult than learning from complete data sets. Designing a sophisticated learning algorithm able to deal with incomplete data sets presents a challenge to workers in this research field. In the past, several methods were proposed to handle the problem of incomplete data sets (Chmielewski et al., 1993, Slowinski and Stefanowski, 1989, Slowinski and Stefanowski, 1994). For example, incomplete data sets may first be transformed into complete data sets (such as by similarity measure) before learning programs begin (Chmielewski et al., 1993), objects with unknown values may be directly removed from data sets (Chmielewski et al., 1993), or unknown objects may be processed in a particular way (Kryszkiewicz, 1998, Liang and Xu, 2000).

The rough-set theory was proposed by Pawlak in 1982 (Pawlak, 1982, Pawlak, 1996) and has been used in reasoning and knowledge acquisition for expert systems (Grzymala-Busse, 1988, Orlowska, 1994). It uses the concept of equivalence classes as its basic principle. Several applications and extensions of the rough-set theory have been proposed. Examples are Orlowska's (1994) reasoning with incomplete information, Germano and Alexandre's (1996) knowledge-base reduction, Lingras and Yao's (1998) data mining, Zhong, Dong, Ohsuga, and Lin's (1998) rule discovery. Because of the success of the rough-set theory in knowledge acquisition, many researchers in the database and machine-learning fields are very interested in this new research topic since it offers opportunities to discover useful information in training examples.

In this paper, we deal with the problem of producing a set of certain and possible rules from incomplete data sets in a different way. We propose a new learning approach based on rough sets, which can simultaneously derive rules from incomplete data sets and estimate the missing values in the learning process. Unknown values are first assumed to be any possible values and are gradually refined according to the incomplete lower and upper approximations derived from the given training examples. The examples and the approximations then interact on each other to derive certain and possible rules and to estimate appropriate unknown values.

The remainder of this paper is organized as follows. The rough-set theory is briefly reviewed in Section 2. Kryszkiewicz's approach for managing incomplete data sets is described in Section 3. The definitions used in this paper are described in Section 4. A novel learning algorithm based on the rough-set theory to simultaneously induce rules and estimate unknown values from incomplete data sets is proposed in Section 5. An example is given to illustrate the proposed algorithm in Section 6. Conclusion and future work are finally given in Section 7.