Rare-PEARs: A new multi objective evolutionary algorithm to mine rare and non-redundant quantitative association rules

Abstract

Since finding quantitative association rules (QARs) is an NP-hard problem, evolutionary methods are suitable solutions for discovery QARs. Nevertheless, most of the previous evolutionary methods to discover association rules only consider frequent dependency among items in datasets. They do not pay specific attention to interestingness and non-redundancy as two critical objectives. In this paper, the proposed algorithm (Rare-PEARs) gives a chance to each rule with different length and appearance (antecedent and consequent parts of rules) to be created. Therefore, various interesting, rare or interesting and rare rules can be found. Some of these rules might be uninteresting (those that contain frequent item sets). However, we try to avoid them by Rare-PEARs. To accomplish this goal, our method decomposes the process of association rule mining into N − 1 sub-problems (N is the number of attributes, and each sub-problem is handled by an independent sub-process during Rare-PEARs execution). Each sub-process starts individually with a different initial population. It then explores the search space of its corresponding sub-problem to find rules with semi-optimal intervals for each of the attributes. This process is done by a new definition of Non-Dominated concept. Rare-PEARs uses this definition to find semi-optimal intervals for attributes during the execution of each sub-process. Finally, Rare-PEARs collects QARs from sub-processes and determines the ultimate Non-Dominated rules based on the interestingness and reliability measures. Rare-PEARs tries to maximize three objectives: interestingness, accuracy and reliability while providing vast coverage on the input dataset. We compared Rare-PEARs with ten algorithms (multi-objective, mono-objective and classical algorithms of association rule mining) over several real-world datasets. The results demonstrate high efficiency of Rare-PEARs.

Introduction

The problem of association rule mining was presented in 1993 and 1994 [2], [1], and after that it has been used as analysis technique by researchers. An Association rule is an expression in the form of X ⇒ Y. In this expression, X and Y are sets of items or attributes-value pairs. The left (X) and right hands (Y) of this expression are antecedent and consequent, respectively. Each of them contains a number of attributes, and these attributes cannot be null. Attributes of association rules can be placed in two domains: the discrete and the continuous domain [4]. Furthermore, association rules in the unsupervised domain can be classified into two groups: categorical association rules versus quantitative association rules [10], [51] and frequent association rules versus infrequent/rare rules [50]. There is also another type of association rules, called class association rules. They are rules with only one attribute in the consequent. Class association rules are used by associative classifiers (Fig. 1). Rules of this category are frequent and strong (The strong rules are rules with high value of confidence) [49].

Apriori algorithm [3] was an outstanding job in identifying association rules, but gradually its drawbacks became obvious. In theory, Apriori algorithm can guarantee results with high accuracy; nevertheless, its runtime will be noticeable in large datasets. To fix this drawback, a number of improved algorithms have been proposed. FP-growth [7] and Partition [8] are two superior examples of these algorithms. They significantly improve the performance, but applying them in some cases is impossible [9]. Population-based algorithms are the recent generation of association rule mining methods [52], [53]. However, they face two major challenges for finding QARs: Determining the appropriate ranges for quantitative attributes and finding interesting rules. We tried to solve these problems in Rare-PEARs.

Various criteria (support, confidence, lift, etc.) are used by the different QARs algorithms. Most of the non-population based algorithms have a common property; they use support, confidence or both of them as the main criteria to determine the quality of rules [1], [2], [3], [14]. Ant colony [11] and PSO [12], [13] based algorithms are some examples of these type of algorithms. Generally, in these algorithms, the user specifies a minimal acceptable value for support. Rules are generated if their support value is more than that threshold. However, support as the sole criterion is not enough. It cannot show the degree of reliability for a rule, and hence, confidence is often used as a measure to calculate the amount of reliability. There are some other measures as well. Lift [37] and CF [38] (certainly factor) are two of the most widely used. They will be described in next section.

By growing the number of criteria, Multi Objective Evolutionary Algorithms (MOEAs) have been introduced. MOEAs are interesting methods for mining QARs. However, rapid convergence damages the efficiency of MOEAs. Some researches, such as [10] solve this problem by restarting. They start again when the difference between two consecutive populations is less than α percent. Sufficient exploration needs to choose α percent carefully. If this value is high, current generations may not have the chance to produce elite chromosomes. Otherwise, it will lead to fast convergence. Finding the optimal interval of each rule’s attribute is another challenge in QARs. We solved these two problems by our proposed method (Rare-PEARs). We split mining QARs problem into N − 1 sub-problems (for each sub-problem a sub-process is considered to find rules with a specific size) and propose new definition of Non-Dominated rules. We called this new definition of Non-Dominated rules as Non-Dominated II (this definition is explained in the last part of this section). Rare-PEARs bring variation in the population through the product item sets with different size for each sub-problem.

Association rules on continuous attributes are called quantitative association rules (QARs). QARs are represented as x ⇒ y, in which x and y are item sets and x ∩ y = ∅. QARs include additional data (intervals of attributes) so they are shown as x[a, b] ⇒ y[c, d]. In this expression, a, b, c, d are real values and also x ∩ y = ∅. QARs are simultaneously produced by N − 1 sub-processes in Rare-PEARs. The aim of each sub-process is finding rules with specific size (these sizes are 2 to N in sub-problems, respectively). This significantly reduces the runtime (we will describe it in Sections 3 Rare-PEARs: a new random permutation based evolutionary algorithm to mine quantitative association rule, 4 Experimental result). Finding semi-optimal intervals for attributes of rules with similar form is the first objective in Rare PEARs. Note that similar form rules are rules with the same antecedent and consequent parts, but their interval values for the attribute are different. An example would be: B[0.1, 0.2] → A[0.2, 0.4]C[−0.4, −0.2], B[0.15, 0.2] → A[0.1, 0.25]C[−0.1, 0.1] and B[0.05, 0.2] → A[−0.2, 0.1]C[−0.3, −0.1]. These are three rules with similar form. Finally, Non-Dominated rules are found between found rules in step one. In Section 3.1, we depict Non-Dominated II in Rare-PEARs.

The goal of this research is to discover rare and/or interesting association rules (the rules with different form will be explained more in Section 3.1). We have developed our approach with three motivations:

• Ability of handling datasets with quantitative values. Previous researches often work on binary or discretized values [5], [13], [58], [59], [60].

• Produce initial population by the aid of evolutionary random permutation for all of sub-problems. Each sub-process starts with individuals with a specific length. This strategy considers an equal survival chance for individuals with different length. However, previous researches [10], [15], [19], [21], [24], [31], [32] generate their initial population randomly (rules with different lengths), and it is possible that rules with a specific length do not appear in the initial population. Hence, rare or interesting rules with bigger length may not be produced by the evolutionary operators. Consequently, fewer or no rules with bigger lengths participate in the finding Non-Dominated rules competition.

• Determining suitable range has a direct effect on discovering interesting and/or rare rules. We accomplish this in our approach by introducing new definition of Non-Dominated rules (Contrary to previous researches [10], [14], [15], [21], [24], [25], [32]). In this new definition, we applied multi-objective concept to find the most interesting state of a rule.

This paper is arranged as follows. In Section 2, related works, including previous studies of association rule mining has been surveyed. This section focuses on evolutionary algorithms. At the end of this section time complexity of association rule mining is studied. In Section 3, we describe our methodology (Rare-PEARs) in detail. This section includes chromosome representation, population initialization and evolutionary operators. Section 4 shows a detailed comparison and presents a comprehensive analysis of the results. Finally, Section 5 concludes the paper.