A novel method to attribute reduction based on weighted neighborhood probabilistic rough sets

https://doi.org/10.1016/j.ijar.2022.01.010Get rights and content

Abstract

Attribute reduction is an important application of rough set theory. Most existing rough set models do not consider the weight information of attributes in information systems. In this paper, we first study the weight of each attribute in information systems by using data binning method and information entropy theory. Then, we propose a new rough set model of weighted neighborhood probabilistic rough sets (WNPRSs) and investigate its basic properties. Meanwhile, the dependency degree formula of an attribute relative to an attribute subset is defined based on WNPRSs. Subsequently, we design a novel attribute reduction method by using WNPRSs and the corresponding algorithm is also given. Finally, to evaluate the performance of the proposed algorithm, we conduct a data experiment and compare it with other existing attribute reduction algorithms on eight public datasets. Experimental result demonstrates that the proposed attribute reduction algorithm is effective and performs better than some of the existing algorithms.

Introduction

Nowadays, with the rapid expansion of information technology, more and more attributes are obtained in production practice. Some attributes may be irrelevant or redundant to the classification task and need to be removed before further data processing [36]. Attribute reduction or feature selection is an important method to reduce the number of attributes. The purpose of attribute reduction is to find an optimal attribute subset to predict the sample categories, thereby promoting data visualization and data understanding. Many attribute reduction methods have been proposed [20], [26]. In recent years, rough set theory [29] has become an influential attribute reduction tool.

The notion of rough sets was introduced by Pawlak [29] in 1982. With decades of development, rough set theory has become a powerful tool to deal with inaccurate, vague and incomplete problems in data analysis. In the rough sets model, we can cope with data without any prior information, like probability distributions or probability values. The classical rough set (RST) theory requires a strict equivalence relation, so it can only mine knowledge in an information system with categorical (nominal) attributes [46]. In practical, databases often contain both categorical attributes and numeric attributes (i.e., mixed attributes) [17]. In order to meet diverse requirements, RST model has been developed to a variety of models, such as fuzzy rough set models [8], [35], neighborhood rough set (NRS) models [15], [16], covering-based rough set models [44], [49], and dominance-based rough set models [6], [7].

In an information system with mixed attributes, the neighborhood relation is easy to compute, so some scholars utilize NRS to implement attribute reduction a hybrid information system [19]. For example, Hu et al. [16] used the NRS model for attribute reduction in information systems with mixed attributes. Wang et al. [36] utilize the relationship between fuzzy neighborhood and fuzzy decision to establish a novel model of fuzzy neighborhood rough set (FNRS) for feature subset selection. Recently, considering that the importance of different attributes in an information system may be different, Hu et al. [19] proposed a model of weighed NRS and established a new attribute reduction method.

On the other hand, the RST model depicts an objective concept via a pair of lower and upper approximate sets. Since these two approximate sets are calculated based on a strict inclusion relation, the RST model is very sensitive to the noisy data [21]. To solve this problem, some scholars extended the RST model and proposed a probabilistic rough set (PRS) model [11], [23], [30], [41]. Over the last two decades, PRS models, such as the 0.5-probabilistic rough set (0.5-PRS) model [39], the variable precision rough set (VPRS) model [50], the decision-theoretic rough set (DTRS) model [43], the Bayesian rough set (BRS) model [33] and the game-theoretic rough set model [12], have been introduced to deal with decision-making problems by allowing a certain degree of error. In these models, two approximate sets can be computed by a pair of thresholds α and β with 0β<α1. In theory, Hu [13] systematically studied most kinds of PRS models and introduced the three-way decision space theory. The application field of PRS is very broad, such as data mining [48], Bayesian risk decision making [25], face recognition [22], rule discovery [28], attribute reduction. [2], [24], [34] and so on. In recent years, scholars have proposed many attribute reduction methods based on different criteria in PRS models [24], [32]. These attribute reduction methods can be broadly classified into several categories: region-based, cost-based, uncertainty measure-based, and entropy-based attribute reduction methods.

For region-based attribute reduction methods, Yao and Zhao [45] firstly proposed the positive region preservation-based attribute reduction method. For cost-based reduction methods, Min et al. [27] put forward a test-cost-sensitive reduction method to maintain the minimum test cost of attributes subset. For uncertainty measure-based attribute reduction methods, by analyzing the non-monotonicity problem of uncertainty measures in PRS, Wang et al. [37] proposed three basic uncertainty measures with monotonicity, and established a new attribute reduction method based on these monotonic uncertainty measures. In order to develop heuristic algorithms to obtain region preservation reduction method, Chen et al. [4] proposed three types of monotonic measure by using the conditional entropy [31]. Gao et al. [9] introduced the maximum conditional entropy into the attribute reduction method based on VPRS model.

In addition, in the application and research of PRS models, it is very important to define equivalence class (or similarity class). For categorical datasets, the equivalence class of a sample is usually defined as: [x]R=aR{y|Ia(x)=Ia(y)}. However most datasets have both categorical and numerical attributes. It is difficult to find two objects with equal values under a same numerical attributes, in this case, [x]R may only contain itself. Therefore, Hu proposed the concepts of δ-neighborhood rough set [14] and k-nearest-neighbor rough set [18], which can solve this problem to a certain extent. However, the relationship between conditional attributes and decision attribute is sometimes nonlinear or even discontinuous. When Ia(x) changes a lot at some points, it may not have a great impact on the class of samples, while small changes at other points may change the class of samples. At this time, no matter how to select δ, it is difficult to solve this problem. The defects of k-nearest-neighbor rough set are obvious. The k samples nearest to sample x may not be enough to describe x, because these k samples are not really close to x. Data binning [5] is a data preprocessing technology, which groups multiple consecutive values into fewer ‘bins’. That is, continuous variables are discretized. The common methods of data binning are equal-width binning [10], equal-depth binning [5], minimum entropy data binning [3] and so on. It is a competitive solution to this problem mentioned above. Because data binning aims to make the sample difference of the same group small and the different groups large, this paper utilizes data binning method to preprocess the data in an information system. Then, considering each attribute plays a different role in the decision-making processe, we calculate the weight of each attribute based on information entropy theory. Based on this, we propose a new rough set model of WNPRSs and design a novel attribute reduction method.

The structure of this paper is arranged as follows. In Section 2, we briefly review several preliminary definitions and concepts. In Section 3, we propose a concept of weighted neighborhood probabilistic rough set (WNPRS) and develop a new method to attribute reduction based on WNPRS. In Section 4, we verify the feasibility of the proposed attribute reduction algorithm by data experiment. In the last section, we conclude our paper and propose the future work.

Section snippets

Preliminaries

This part is composed of four subsections to review various preliminaries regarding probabilistic rough sets, neighborhood rough sets, information acquisition and data binning.

Attribute reduction method based on weighted neighborhood probabilistic rough sets (WNPRSs)

In this section, we introduce a novel attribute reduction method based on weighted neighborhood probabilistic rough sets (WNPRSs). In the following, we analyze the shortcomings of classic rough sets (RSTs) and neighborhood rough sets (NRSs), which motives us to propose a new rough set model (named as WNPRSs) and develop a new method of attribute reduction in information systems.

Data experiment

In this section, we verify the feasibility and validity of the proposed attribute reduction method on large datasets. Eight datasets derived from UCI (https://archive.ics.uci.edu/ml/index.php) are used in the experimental analysis. Detailed information is shown in Table 4. All algorithms are executed in MATLAB 2016a, and run in hardware environment with Inter(R) Core(TM) i7-4790 CPU @3.60 GHz 3.60 GHz, with 16 GB RAM. In addition, because norms are compatible, we measure the distance uniformly

Conclusions

In this paper, based on data binning method and information entropy theory, we propose a new rough set model of weighted neighborhood probabilistic rough sets (WNPRSs) and investigate its basic properties. Then, we design a novel approach to attribute reduction by using WNPRSs. Data experiment shows that this method is a more competitive one. The classification accuracy of most datasets is improved. In the future, we plan to design a novel attribute reduction method by combining WNPRSs with

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgements

The authors would like to express their sincere thanks to the Editors and anonymous reviewers for their most valuable comments and suggestions in improving this paper greatly. The work described in this paper was supported by grants from the National Natural Science Foundation of China (Grant nos. 11971365 and 11571010).

References (50)

  • Q.H. Hu et al.

    Neighborhood classifiers

    Expert Syst. Appl.

    (2008)
  • M. Hu et al.

    A novel approach to attribute reduction based on weighted neighborhood rough sets

    Knowl.-Based Syst.

    (2021)
  • R. Jensen et al.

    Fuzzy–rough attribute reduction with application to web categorization

    Fuzzy Sets Syst.

    (2004)
  • W.W. Li et al.

    Multi-objective attribute reduction in three-way decision-theoretic rough set model

    Int. J. Approx. Reason.

    (2019)
  • H.X. Li et al.

    Sequential three-way decision and granulation for cost-sensitive face recognition

    Knowl.-Based Syst.

    (2016)
  • F. Min et al.

    Feature selection with test cost constraint

    Int. J. Approx. Reason.

    (2014)
  • F. Min et al.

    Test-cost-sensitive attribute reduction

    Inf. Sci.

    (2011)
  • Z. Pawlak et al.

    Rough sets: probabilistic versus deterministic approach

    Int. J. Man-Mach. Stud.

    (1988)
  • D. Ślȩzak et al.

    Attribute reduction in the Bayesian version of variable precision rough set model

    Electron. Notes Theor. Comput. Sci.

    (2003)
  • D. Ślȩzak et al.

    The investigation of the Bayesian rough set model

    Int. J. Approx. Reason.

    (2005)
  • M.L. Suo et al.

    Single-parameter decision-theoretic rough set

    Inf. Sci.

    (2020)
  • C.Z. Wang et al.

    Feature subset selection based on fuzzy neighborhood rough sets

    Knowl.-Based Syst.

    (2016)
  • G.Y. Wang et al.

    Monotonic uncertainty measures for attribute reduction in probabilistic rough set model

    Int. J. Approx. Reason.

    (2015)
  • J. Wang et al.

    A novel hybrid feature selection method considering feature interaction in neighborhood rough set

    Knowl.-Based Syst.

    (2021)
  • S.K.M. Wong et al.

    Comparison of the probabilistic approximate classification and the fuzzy set model

    Fuzzy Sets Syst.

    (1987)
  • Cited by (28)

    • Topological reduction approaches for relation decision systems

      2023, International Journal of Approximate Reasoning
    • A novel incremental attribute reduction by using quantitative dominance-based neighborhood self-information

      2023, Knowledge-Based Systems
      Citation Excerpt :

      Because of its advantages, RST has often been considered as the theoretical basis of attribute reduction. In recent decades, lots of attribute reduction methods that rely on different rough set models have been explored to solve different problems in practical applications [12–18]. Until now, fuzzy rough sets [19] and neighborhood rough sets [20] are relatively widely used in attribute reduction.

    View all citing articles on Scopus
    View full text