Mining Correlated Patterns with Multiple Minimum All-Confidence Thresholds

Abstract

Correlated patterns are an important class of regularities that exist in a database. The all-confidence measure has been widely used to discover the patterns in real-world applications. This paper theoretically analyzes the all-confidence measure, and shows that, although the measure satisfies the null-invariant property, mining correlated patterns involving both frequent and rare items with a single minimum all-confidence (minAllConf) threshold value causes the “rare item problem” if the items’ frequencies in a database vary widely. The problem involves either finding very short length correlated patterns involving rare items at a high minAllConf threshold, or generating a huge number of patterns at a low minAllConf threshold. The cause for the problem is that the single minAllConf threshold was not sufficient to capture the items’ frequencies in a database effectively. The paper also introduces an alternative model of correlated patterns using the concept of multiple minAllConf thresholds. The proposed model facilitates the user to specify a different minAllConf threshold for each pattern to reflect the varied frequencies of items within it. Experiment results show that the proposed model is very effective.