A new rule reduction and training method for extended belief rule base based on DBSCAN algorithm

https://doi.org/10.1016/j.ijar.2019.12.016Get rights and content

Abstract

Rule reduction is one of the focuses of numerous researches on belief-rule-based system, in some cases, too many redundant rules may be a concern to the rule-based system. Though rule reduction methods have been widely used in the belief-rule-based system, extended belief-rule-based system, which is an expansion of belief-rule-based system, still lacks methods to reduce and train rules in the extended belief rule base (EBRB). To this end, this paper proposes an EBRB reduction and training method. Based on the density-based spatial clustering applications with noise (DBSCAN) algorithm, a new EBRB reduction method is proposed, where all the rules in the EBRB will be visited and rules within the distance of the fusion threshold will be fused. Moreover, the EBRB training method using parameter learning, which uses a set of training data to train the parameters of EBRB, is also proposed to improve the accuracy of the EBRB system. Two case studies of regression and classification are used to illustrate the feasibility and efficiency of the proposed EBRB reduction and training method. Comparison results show that the proposed method can effectively downsize the EBRB and increase the accuracy of EBRB system.

Introduction

Among various knowledge representation schemes, rule-based system has been widely recognized as one of the most commonly used frameworks due to its advantages of expressing various types of knowledge under the same framework [1]. Constructed by human knowledge in the forms of IF-THEN rules, it has become one of the fastest-growing methods in the field of decision support system and artificial intelligence [2], [3]. Normally, simple IF-THEN rules such as “IF failure rate is high, THEN safety estimate is poor” are used to construct the rule base, which is an essential part of the rule-based system, and it has attracted numerous studies on its optimization [4], [5], [6]. Among those researches, rule base reduction has always been a focus as excessive rules could decrease the accuracy and increase the complexity of the rule-based system [7], [8], [9]. Moreover, as both the antecedent and the consequent terms of the IF-THEN rule are believed to be 100% certain, while such strict knowledge representation scheme may be inefficient in expressing information with uncertainty, many new rule-based systems have been developed to enhance the ability to deal with both qualitative and quantitative information under uncertainty.

In this regard, Yang et al. [10] proposed the belief-rule-base methodology using the evidential reasoning (RIMER) approach in 2006, which is developed on the basis of Dempster-Shafer theory of evidence [11], [12], [13], Bayesian probability theory [14], fuzzy set theory [15], [16] and traditional IF-THEN rule-based system [1], [17]. This methodology has its advantages in reflecting information under conditions of uncertainty, such as fuzziness, ignorance, and incompleteness, and has been widely used in consumer preference prediction [18], [19], [20], medical diagnosis [21], [22], [23], risk analysis [24], [25], [26], [27], and many other fields. Compared with belief functions [13], [17], the belief rule-based system is more suitable for problems involving prior knowledge, such as regression and classification problems. By grouping belief rules into one belief rule base (BRB), the belief rule-based system can effectively address different decision problems, however, as BRB is an essential part of the belief rule-based system and has a significant impact on the reasoning results, its optimization method has attracted extensive studies [28], [29], [30]. Moreover, belief rule base reduction has also become a focus of researchers, and many methods have been applied to provide promising belief rule base reduction methods [31], [32], [33], [34]. Furthermore, parameter learning method combined with belief rule base reduction has been studied by some researchers to optimize the belief rule base and reduce its scale at the same time [35]. Recently, based on the RIMER approach, Liu et al. [36] proposed a more general belief rule-based system by introducing belief structures into the antecedent attributes of each rule, called the extended belief rule base (EBRB), to better deal with uncertainty in the antecedent attributes. With belief degrees embedded in the entire antecedent attributes of each rule, EBRB can more effectively deal with different kinds of uncertainty and present more accurate results, and has been used in fields such as health estimation [37], sensor configuration optimization [38], activity recognition [39] and some other fields [40], [41].

However, several challenges still need to be addressed in the EBRB system. The first is to reduce the size of EBRB system and avoid inconsistency, and the second challenge is to enhance its representation power, which is essential for rule-based systems.

For the first challenge, some researchers approach it by adjusting the number of activated rules in EBRB system, which could reduce the inference time [42], [43], [44]. Calzada et al. [43] proposed a dynamic rule activation (DRA) method for EBRB system, where the activated rules are selected dynamically to search for a balance between the incompleteness and inconsistency. Lin et al. [44] proposed the rule activation method based on VP-tree and MVP-tree, where only partial rules will be retrieved and visited while calculating each rule's activation weight. Yang et al. [45] proposed the consistency analysis-based rule activation (CABRA) method to redefine suitable activated rules by using the set of consistent rules as an activation framework, which enables the EBRB system to overcome problems of inconsistency and incompleteness. Yang et al. [41] constructed the activation region of extended belief rules and revised the calculation formula of activation weights to redefine activated rules, and proposed new activation rule determination and weight calculation procedures. However, these rule activation methods cannot effectively reduce the size of EBRB, and information could be lost as only limited rules are activated for a given input. As for EBRB reduction, there have been few studies, Yang et al. [46] proposed EBRB rule reduction method based on data envelopment analysis (DEA), where DEA is used to evaluate the efficiency of each rule and remove inefficient rules from EBRB to reduce its size. As explained in Yang et al. [46], EBRB reduction is more important and effective than rule activation to solve the first challenge since: (1) the size of EBRB could be overlarge without EBRB reduction as input-output data pair can be easily transferred into belief rules in EBRB; (2) activating all the rules in EBRB for any input data is the exact reason for inconsistency, and it could get worse when the size of EBRB is increasing; (3) simply activate certain rules for each input data could lead to the loss of information as several related rules could be ignored.

For the second challenge, one of the most common ways is to increase the number of rules in the rule base, which may also increase its time cost. Recently, rule base training has been used as an efficient way to improve the reasoning performance of belief rule-based system. Yang et al. [47] proposed a new activation weight calculation and parameter optimization method based on sensitivity analysis, by using the new parameter optimization method, it enhanced the accuracy of the EBRB system. Though EBRB uses training data for rule generation, the training process is still necessary for increasing its representation power and enhancing its performance.

In order to address the abovementioned challenges, density-based spatial clustering of applications with noise (DBSCAN) algorithm and parameter learning are applied for EBRB reduction and training in this paper. For the first challenge, a new EBRB reduction method is proposed, where similar belief rules would be searched and fused using the DBSCAN algorithm based on the Euclidean distance between different rules. For the second challenge, the parameters of EBRB, namely, belief degrees of antecedent attributes, consequent belief degrees, attribute weights and rule weights are trained using an extended parameter learning method to enhance its accuracy. For an oversize EBRB, EBRB reduction method will be firstly used to prune the belief rules, and EBRB training method will be applied to train the parameter of the newly constructed EBRB. The main advantage of the proposed method is that it can effectively reduce the size of EBRB while maintaining relatively good accuracy. To illustrate the effectiveness of the proposed method, two cases, including function approximation and classification, are applied. Two main aspects, namely, accuracy and the number of rules, are used to compare with other EBRB systems.

The rest of this paper is organized as follows. Section 2 briefly reviews the basics of evidential reasoning (ER) approach and EBRB system. Section 3 proposes the EBRB training method and the EBRB reduction method. Section 4 provides two case studies to demonstrate the efficiency of the proposed EBRB reduction and training method, and the paper is concluded in Section 5.

Section snippets

Evidential reasoning approach

To capture incomplete, fuzzy and ignorant information, Yang and Singh [48] proposed the evidential reasoning approach based on the D-S theory of evidence. It uses belief structure to represent an assessment of an attribute ei as follows [49]:S(ei)={(H1,β1,i),,(Hn,βn,i),,(HN,βN,i),n=1,,N} where Hn is the nth evaluation grade, βn,i denotes a degree of belief, βn,i0, n=1Nβn,i1. The above belief structure reads that the attribute ei is assessed to Hn with a degree of belief βn,i. An

EBRB training method using parameter learning

Yang et al. [29] proposed optimal learning model for training BRBs, where rule weight θk, attribute weight δi and consequent belief degree βnk are trained simultaneously using a set of input-output data pairs to enhance BRB performance. However, as for the EBRB system, because belief degrees are embedded in all the antecedent terms of each rule as well, the training parameters should be expanded to antecedent attribute belief degree αi,jk, consequent belief degree βnk, rule weight θk and

Case studies

In order to illustrate the performance and effectiveness of the proposed EBRB reduction and training method, two cases of regression and classification problems are studied.

Conclusion

A new EBRB reduction and training method is proposed in this paper to solve the problem that too many belief rules of the oversize EBRB could increase the calculation time cost and affect the reasoning performance. The main conclusions of this paper can be summarized as follows:

  • (1)

    The reasoning performance of EBRB systems with different number of belief rules under the same number of training data are examined. This paper reveals the influence the number of belief rules has on the reasoning

Declaration of Competing Interest

We wish to confirm that there are no known conflicts of interest associated with this publication and there has been no significant financial support for this work that could have influenced its outcome.

Acknowledgement

The authors thanks the two anonymous reviewers for their useful comments. This work was supported by the National Natural Science Foundation of China (Nos. 61573283, 61903305), the Key Laboratory Open Foundation of Data Link Technology (No. CLDL-20182113) and the Research Funds for Interdisciplinary Subject, NWPU.

References (61)

  • G. Li et al.

    An optimal safety assessment model for complex systems considering correlation and redundancy

    Int. J. Approx. Reason.

    (2019)
  • D.-L. Xu et al.

    Inference and learning methodology of belief-rule-based expert system for pipeline leak detection

    Expert Syst. Appl.

    (2007)
  • L. Chang et al.

    Belief rule based expert system for classification problems with new rule activation and weight calculation procedures

    Inf. Sci.

    (2016)
  • L. Chang et al.

    Structure learning for belief rule base expert system: a comparative study

    Knowl.-Based Syst.

    (2013)
  • L. Chang et al.

    Indirect disjunctive belief rule base modeling using limited conjunctive rules: two possible means

    Int. J. Approx. Reason.

    (2019)
  • Y.-M. Wang et al.

    Dynamic rule adjustment approach for optimizing belief rule-base expert system

    Knowl.-Based Syst.

    (2016)
  • J. Liu et al.

    A novel belief rule base representation, generation and its inference methodology

    Knowl.-Based Syst.

    (2013)
  • M. Espinilla et al.

    Optimizing the configuration of an heterogeneous architecture of sensors for activity recognition, using the extended belief rule-based inference methodology

    Microprocess. Microsyst.

    (2017)
  • L.-H. Yang et al.

    Extended belief-rule-based system with new activation rule determination and weight calculation for classification problems

    Appl. Soft Comput.

    (2018)
  • L.-H. Yang et al.

    A consistency analysis-based rule activation method for extended belief-rule-based systems

    Inf. Sci.

    (2018)
  • L.-H. Yang et al.

    A data envelopment analysis (DEA)-based method for rule reduction in extended belief-rule-based systems

    Knowl.-Based Syst.

    (2017)
  • J.-B. Yang et al.

    The evidential reasoning approach for MADA under both probabilistic and fuzzy uncertainties

    Eur. J. Oper. Res.

    (2006)
  • A.-L. Jousselme et al.

    Distances in evidence theory: comprehensive survey and generalizations

    Int. J. Approx. Reason.

    (2012)
  • J. Derrac et al.

    Evolutionary fuzzy k-nearest neighbors algorithm using interval-valued fuzzy sets

    Inf. Sci.

    (2016)
  • G. David et al.

    Spectralcat: categorical spectral clustering of numerical and nominal data

    Pattern Recognit.

    (2012)
  • J. Wu et al.

    Self-adaptive attribute weighting for Naive Bayes classification

    Expert Syst. Appl.

    (2015)
  • L. Jiao et al.

    Belief rule-based classification system: extension of FRBCS in belief functions framework

    Inf. Sci.

    (2015)
  • Y.-H. Shao et al.

    Weighted linear loss twin support vector machine for large-scale classification

    Knowl.-Based Syst.

    (2015)
  • S. Datta et al.

    Near-bayesian support vector machines for imbalanced data classification with equal or unequal misclassification costs

    Neural Netw.

    (2015)
  • M. Fallahnezhad et al.

    A hybrid higher order neural classifier for handling classification problems

    Expert Syst. Appl.

    (2011)
  • Cited by (36)

    • BRN: A belief rule network model for the health evaluation of complex systems

      2023, Expert Systems with Applications
      Citation Excerpt :

      It limits the application of the BRB method in complex system health evaluation. To solve the above problem, some modified BRB models and learning algorithms have been proposed (Chen, Zhou, Zhang, Hu, & Cao, 2022; Zhang, Gao, Yang, & Bi, 2020). These methods can be summarized with respect to the following three aspects:

    View all citing articles on Scopus
    View full text