Learning rules from incomplete training examples by rough sets
Introduction
Expert systems have been widely used in domains where mathematical models cannot be easily built, human experts are not available or the cost of querying an expert is high. Although a wide variety of expert systems have been built, knowledge acquisition remains a development bottleneck. Usually, a knowledge engineer is needed to establish a dialog with a human expert and to encode the knowledge elicited into a knowledge base to produce an expert system. The process is however very time-consuming (Buchanan and Shortliffe, 1984, Giarratano and Riley, 1989). Shortening the development time is then the most important factor for the success of an expert system.
Recently, machine-learning techniques have been developed to ease the knowledge-acquisition bottleneck. Among proposed approaches, deriving rules from training examples is the most common (Hong et al, 2001, Hong et al., 2000, Kodratoff and Michalski, 1983, Michalski et al., 1983, Michalski et al., 1984, Tsumoto, 1998). Given a set of examples, a learning program tries to induce rules that describe each class.
Most learning approaches derive rules from complete data sets. If some attribute values are unknown in a data set, it is called incomplete. Learning from incomplete data sets is usually more difficult than learning from complete data sets. Designing a sophisticated learning algorithm able to deal with incomplete data sets presents a challenge to workers in this research field. In the past, several methods were proposed to handle the problem of incomplete data sets (Chmielewski et al., 1993, Slowinski and Stefanowski, 1989, Slowinski and Stefanowski, 1994). For example, incomplete data sets may first be transformed into complete data sets (such as by similarity measure) before learning programs begin (Chmielewski et al., 1993), objects with unknown values may be directly removed from data sets (Chmielewski et al., 1993), or unknown objects may be processed in a particular way (Kryszkiewicz, 1998, Liang and Xu, 2000).
The rough-set theory was proposed by Pawlak in 1982 (Pawlak, 1982, Pawlak, 1996) and has been used in reasoning and knowledge acquisition for expert systems (Grzymala-Busse, 1988, Orlowska, 1994). It uses the concept of equivalence classes as its basic principle. Several applications and extensions of the rough-set theory have been proposed. Examples are Orlowska's (1994) reasoning with incomplete information, Germano and Alexandre's (1996) knowledge-base reduction, Lingras and Yao's (1998) data mining, Zhong, Dong, Ohsuga, and Lin's (1998) rule discovery. Because of the success of the rough-set theory in knowledge acquisition, many researchers in the database and machine-learning fields are very interested in this new research topic since it offers opportunities to discover useful information in training examples.
In this paper, we deal with the problem of producing a set of certain and possible rules from incomplete data sets in a different way. We propose a new learning approach based on rough sets, which can simultaneously derive rules from incomplete data sets and estimate the missing values in the learning process. Unknown values are first assumed to be any possible values and are gradually refined according to the incomplete lower and upper approximations derived from the given training examples. The examples and the approximations then interact on each other to derive certain and possible rules and to estimate appropriate unknown values.
The remainder of this paper is organized as follows. The rough-set theory is briefly reviewed in Section 2. Kryszkiewicz's approach for managing incomplete data sets is described in Section 3. The definitions used in this paper are described in Section 4. A novel learning algorithm based on the rough-set theory to simultaneously induce rules and estimate unknown values from incomplete data sets is proposed in Section 5. An example is given to illustrate the proposed algorithm in Section 6. Conclusion and future work are finally given in Section 7.
Section snippets
Review of the rough-set theory
The rough-set theory, proposed by Pawlak in 1982 (Pawlak, 1982, Pawlak, 1996), can serve as a new mathematical tool for dealing with data classification problems. It adopts the concept of equivalence classes to partition training instances according to some criteria. Two kinds of partitions are formed in the mining process: lower approximations and upper approximations, from which certain and possible rules can easily be derived.
Formally, let U be a set of training examples (objects), A be a
Incomplete data sets
Data sets can be roughly classified into two classes: complete and incomplete data sets. All the objects in a complete data set have known attribute values. If at least one object in a data set has a missing value, the data set is incomplete. Table 2 shows an example of an incomplete data set.
In Table 2, the symbol ‘∗’ denotes an unknown attribute value. Thus, the SP values of Obj(5) and Obj(9) are unknown. Similarly, the DP value of Obj(7) is unknown. The data set is thus incomplete.
Learning
Definitions
Since an incomplete data set contains unknown attribute values, the original equivalence relation in rough sets must be modified to manage it. As before, an equivalence class is formed for each attribute value or for each value combination of attributes. In this paper, each object is represented as a tuple (obj, symbol), where the symbol may be certain (c) or uncertain (u). If an object obj(i) has a certain value vj(i) for attribute Aj, then (obj(i), c) is put in the equivalence class for vj(i)
A rough-set-based approach to simultaneously estimate missing values and derive rules
In this section, a new learning algorithm based on rough sets is proposed, which can simultaneously estimate the missing values and derive certain and possible rules from incomplete data sets. As mentioned before, each object is represented as a tuple (obj, symbol), where the symbol may be certain (c) or uncertain (u). If the object has a missing value of an attribute, it is first put into each incomplete equivalence class from that attribute.
The algorithm then calculates incomplete lower
An example
The incomplete data set in Table 2 (in Section 3) is used to demonstrate how the proposed algorithm can simultaneously estimate missing values and derive certain and possible rules. There are seven objects and two attributes SP and DP with some missing values in the data set. Three classes for BP are to be classified. The proposed learning algorithm processes this incomplete data set as follows.
Step 1. Since three classes exist in the incomplete data set, three partitions are formed as follows:
Conclusion and future work
In this paper, we have proposed a new learning approach to derive rules from incomplete data sets based on the rough-set theory. The proposed approach is different from others in that it can derive rules and estimate the missing values at the same time. The incomplete lower and upper approximations have been defined for managing uncertain objects in incomplete data sets. The interaction between data and approximations helps derive certain and possible rules from incomplete data sets and
References (21)
Rough set approach to incomplete information systems
Information Science
(1998)- et al.
Rough classification in incomplete information systems
Mathematical and Computer Modelling
(1989) Extraction of experts' decision rules from clinical databases using rough set model
Intelligent Data Analysis
(1998)Variable precision rough set model
Journal of Computer and System Sciences
(1993)- et al.
Rule-based expert system: the MYCIN experiments of the Standford heuristic programming projects
(1984) - et al.
The rule induction system LERS—a version for personal computers
Foundations of computing and decision sciences
(1993) - Germano, L. T., & Alexandre, P. (1996). Knowledge-base reduction based on rough-set techniques. The Canadian Conference...
- et al.
Expert systems—principles and programming
(1989) Knowledge acquisition under uncertainty: A rough set approach
Journal of Intelligent Robotic Systems
(1988)- et al.
Knowledge acquisition from quantitative data using the rough-set theory
Intelligent Data Analysis
(2000)
Cited by (94)
Dynamic information fusion in multi-source incomplete interval-valued information system with variation of information sources and attributes
2022, Information SciencesCitation Excerpt :Regarding knowledge acquisition, Li et al. [11] proposed a novel interval set model to induce classification rules from incomplete data. Hong et al. [12] established a new rules-deriving algorithm for incomplete data by estimating the missing value in the process of rule deriving. Leung et al. [13] efficiently acquired knowledge for incomplete information systems by defining a set of simpler discernibility functions.
A novel approach to predictive analysis using attribute-oriented rough fuzzy sets
2020, Expert Systems with ApplicationsCitation Excerpt :Dubois and Prade (Dubois & Prade, 1990) used an equivalence relation of the universe to introduce the upper and lower approximations of fuzzy sets (FSs) in an equivalent approximation space and proposed an extended concept called rough fuzzy sets (RFSs). As an efficient mathematical model for addressing inexact or uncertain knowledge, RST has been applied in diverse fields, such as medical diagnosis (Pattaraintakorn & Cercone, 2008), machine learning (Hong, Lin, Lin, & Wang, 2008; Hong, Tseng, & Wang, 2002), decision making (Xiong, Su, & Li, 2012, 2012, 2011, 2003, 2011), pattern recognition (Wang & Wang, 2009), case-based reasoning (Huang & Tseng, 2004), and DM (Lingras & Yao, 1998, 2009). Yu, Cai, and Li (2018) combined the advantages of TOPSIS and λ-rough sets to predict the highest potential of objects information systems (ISs).
A characterization of novel rough fuzzy sets of information systems and their application in decision making
2019, Expert Systems with ApplicationsCitation Excerpt :This method is particularly useful when dealing with uncertain and vague knowledge in information systems. Numerous applications of rough set methods have been presented in research related to decision-making (Salamó & López-Sánchez, 2011; Son, Kim, Kim, Park, & Kim, 2012; Swiniarski & Skowron, 2003; Tian, Zeng, & Keane, 2011; Xiong, Su, & Li, 2012), medical diagnosis (Pattaraintakorn & Cercone, 2008), case-based reasoning (Huang & Tseng, 2004), machine learning (Hong, Lin, Lin, & Wang, 2008; Hong, Tseng, & Wang, 2002), pattern recognition (Wang & Wang, 2009), and data mining (Lingras & Yao, 1998; Yamaguchi, 2009). The construction of a pair of lower and upper approximation operators is essential in rough set theory.
A λ-rough set model and its applications with TOPSIS method to decision making
2019, Knowledge-Based SystemsConsiderations on the principle of rule induction by STRIM and its relationship to the conventional Rough Sets methods
2018, Applied Soft Computing JournalCitation Excerpt :The conclusions reached in [30] were: However, the conventional methods for missing attribute values [27,28,30] are still to propose the principle for handling such datasets and illustrate the principle by using a small sample-size dataset of at most twelve. There are few examples of applying them to real-world datasets and confirmed to be useful as far as we know.
Uncertainty learning of rough set-based prediction under a holistic framework
2018, Information Sciences