Attribute reduction methods in fuzzy rough set theory: An overview, comparative experiments, and new directions

Highlights

• Attribute reduction methods based on fuzzy rough set theory are comprehensively reviewed.

• All methods are summarized through six different aspects.

• The experimental results clarify that the three types of reduction methods are effective.

• Statistical hypothesis test verified that the performance of these algorithms is different.

Abstract

Fuzzy rough set theory is a powerful tool to deal with uncertainty information, which has been successfully applied to the fields of attribute reduction, rule extraction, classification tree induction, etc. In order to comprehensively investigate attribute reduction methods in fuzzy rough set theory, this paper first briefly reviews the related concepts of fuzzy rough set theory. Then, all methods are summarized through six different aspects including data sources, preprocessing methods, fuzzy similarity metrics, fuzzy operations, reduction rules, and evaluation methods. Among them, reduction rules are reviewed in three categories, i.e., fuzzy dependency-based, fuzzy uncertainty measure-based, and fuzzy discernibility matrix-based. These three types of reduction rules are compared and analyzed through experiments. The experimental results clarify that these three reduction rules can retain fewer attributes and improve or maintain the classification accuracy of a classifier. Moreover, the statistical hypothesis test is conducted to evaluate the statistical difference of these methods. The results show that these algorithms are statistically significantly different. Finally, some new research directions are discussed.

Introduction

In recent years, rough set theory has received extensive attention in many fields such as data mining, pattern recognition, and machine learning. As a mathematical framework that can handle uncertainty data, the classical rough set (CRS) model has been successfully applied to attribute reduction (feature selection), rule extraction, and uncertainty reasoning [1], [2], [3], [4], [5], [6], [7]. Equivalence relations is used in CRS, which is only suitable for categorical (nominal) attributes. In fact, categorical and numerical attributes (i.e., mixed attribute) often coexist in real-life databases, such as attributes that exist in medical analysis and fault diagnosis [8]. One of the feasible methods to deal with numerical attributes is to discretize the numerical attributes. However, discretization is usually an important reason for information loss. In response to the above-mentioned problems, Dubois and Prade proposed Fuzzy Rough Set (FRS) model [9], [10], which provides an effective way to overcome the problem of discretization in real-value data and can be directly applied to numerical or mixed features.

The existing researches on FRS theory can be roughly divided into two topics: the construction of approximate space and the application of methods. On the one hand, a series of extensions on FRSs [11], [12], [13], [14], [15] are investigated according to the different requirements or application scenarios. On the other hand, FRSs have been successfully applied to many fields, such as attribute reduction [16], [17], [18], [19], rule extraction [20], classification tree induction [21], medical analysis [22], etc. Among them, attribute reduction is one of hot research topics in FRS theory, which is to delete irrelevant or unimportant attributes while keeping the ability to classification unchanged. The attributes given by attribute reduction may further used to construct learning models with better generalization ability and lower computation consumption. According to whether there is decision (label) information, the existing attribute reduction methods can be divided into supervised [23], semi-supervised [24], [25] and unsupervised [26], [27]. In addition, according to different search strategies, attribute reduction methods can also be divided into three types: filter-based method [28], wrapper-based method [29], and embedded-based method [30]. In recent years, two other search strategies have also been studied, namely hybrid method [31] and ensemble [25], [32] method. However, the review literature of attribute reduction methods based on FRS theory is still rare. Recently, in [33], authors reviewed the feature selection based on FRS theory, and they divided it into two types of methods: fuzzy dependency-based and fuzzy discernibility matrix-based method. In fact, the feature selection method based on FRSs also includes fuzzy uncertainty measure-based methods. What is more, comparative experiments on the related attribute reduction methods are not reported. Therefore, it is necessary to further discuss attribute reduction methods based on FRS theory.

In view of the above discussion, this paper focuses on a comprehensive overview on attribute reduction methods based on FRS theory before January 1, 2021. First, the FRS theory is reviewed. Then, in order to clarify the application of FRS theory in attribute reduction, all methods are summarized from six aspects including data sources, preprocessing methods, fuzzy similarity metrics, fuzzy operations, reduction rules, and evaluation methods. Among them, attribute reduction rules are divided into three categories: (1) fuzzy dependency-based, (2) fuzzy uncertainty measure-based, and (3) fuzzy discernibility matrix-based. Finally, these three types of attribute reduction rules are compared and analyzed through experiments. The experimental results clarify that these three types of methods can retain fewer attributes and improve or maintain the accuracy of a classifier. Besides, statistical hypothesis test is conducted to further evaluate the statistical difference of these methods. The results show that these algorithms are statistically significantly different. Further, some research directions in the future are discussed. The research framework of attribute reduction method based on FRS theory is shown in Fig. 1.

The rest of this paper is organized as follows. In Section 2, we briefly introduce the preliminaries about FRS theory. In Section 3, attribute reduction methods based on FRSs theory are summarized from six aspects. Discussions on these methods are carried out in Section 4. The analyses of the comparative experiments are presented in Section 5. Some future research directions are discussed in Section 6. Finally, Section 7 summarizes the paper.