A State-of-the-Art Review of Knowledge Discovery in Multiple Databases

1.1 Multi-relational Tables

Multi-relational (MR) data mining approaches look for patterns that involve multiple tables (relations) from a relational database [22], [29]. It includes MR association rule discovery, MR decision trees, MR distance-based methods, etc. Domingos [26] has presented a survey on MR data mining, and held discussions on several areas such as WWW, counter-terrorism, virtual marketing, social networks, computational biology, and ubiquitous computing that include relational data mining. Spyropoulou et al. [55] have introduced a new syntax for MR patterns in MR data, and the RMiner algorithm to mining them efficiently.

1.2 Time-based Databases

Time-stamped data are a natural domain for data analyses. Most of the data generated are related to time either directly or indirectly; hence, the data analyses using time-stamped data has become an important area of study. Another important domain is health-care data where most health-care data include a time stamp specifying the temporal representation of patient information. Nigrin and Kohane [49] demonstrated how their previously described data retrieval application, DXtractor, can be used as a database querying application with expressive power close to that of temporal databases and temporal query languages. They used standard SQL and existing time-stamp-based repositories only. Adhikari [12] studied various problems related to multiple time-stamped data.

1.3 Distributed Datasets

Distributed data mining (DDM) algorithms deal with mining multiple databases distributed over different geographical regions. In the last few years, researchers have started addressing problems where the databases are stored at different places that cannot be moved to a central storage area for a variety of reasons. The DDM environment often comes with different distributed sources of computation. Ubiquitous computing [30], sensor networks [65], grid computing [59], and privacy-sensitive multiparty data [35] present examples where centralization of data is either not possible or at least not always desirable.

1.4 Spatial Databases

Miller and Han [42] provided a scenario for discovering knowledge in spatial databases. Lazarevic and Obradovic [39] have applied techniques to discover knowledge using centralized and distributed learning from spatial heterogeneous databases. The centralized algorithm consists of a spatial clustering followed by local regression aimed at learning the relationships between the driving attributes and the target variable inside each region identified through clustering. In this distributed learning, similar regions in multiple databases are first discovered by applying a spatial clustering algorithm independently on all the sites, then identifying the corresponding clusters on the participating sites.

1.5 Multiple Stream Data

Sometimes, data streams are collected from multiple sensors. An important characteristic of stream data is that it never ends. Knowledge discovery in multiple data streams is an interesting problem for research [60]. Krempl et al. [36] identified challenges in covering the full cycle of knowledge discovery for data stream mining research, such as protecting data privacy, dealing with legacy systems, handling incomplete and delayed information, analysis of complex data, and evaluation of stream mining algorithms.

As many domains have already emerged, knowledge discovery in multiple related databases becomes an important area of data mining research. With the availability of multiple related databases, and due to the constraints as mentioned above, knowledge discovery in multiple databases is gaining more importance.

The rest of the paper is organized as follows. Section 2 provides various data preparation techniques often required in a multi-database environment. Then, in Section 3, we present a few techniques for mining multiple related databases. Over time, many patterns are reported in a multi-database environment, and they are discussed in Section 4. Some related works are discussed in Section 5, and we conclude the article in Section 6.

2.1 Preparation of Data Warehouses

Each branch database needs to be preprocessed to make multi-databases suitable for data mining. It could well be that all the data sources are not in the same format. Sometimes, data need to be converted from one type to another. The data need to be processed before any mining task takes place. A few important steps for preparing the data at a branch are aggregation, sampling, dimensionality reduction, feature subset selection, feature creation, discretization, and variable transformation [57]. Relevant data are required to be retained for the purpose of mining. Also, the definitions of data are required to be the same at every data source. The preparation of data warehouse at every branch of the organization could be a significant task [7], [52], [60].

2.2 Temporal Aggregation

In historical databases, temporal aggregation is a process in which the timeline is partitioned and the values of various attributes in the database are accumulated over these partitions [27]. A typical example of temporal aggregation is the monthly accumulation of salary payment. Due to the large variety of temporal data and their distribution over the timeline, efficient algorithms to perform temporal grouping are required. Moon et al. [48] proposed several methods for large-scale temporal aggregation. In this context, the choice of time granularity is an important issue, as the characteristics of temporal patterns are heavily dependent on this parameter. Let us consider an online shop that acquires monthly reports from their web hosts. The web hosts deliver the activity reports at regular intervals. Here, time granularity refers to a month instead of a year. Therefore, for this application, month-wise time-stamped data could be accumulated to form smaller databases. Similarly, for stock market applications, weekly data accumulation may be preferred over monthly data.

2.3 Partitioning the Database

Partitioning a large database becomes an important task in some cases. Consider an organization possessing data over 50 consecutive years. The organization might be interested in mining the knowledge for various activities such as finding the items whose supports are stable over the time [15], and extracting yearly periodic patterns [14]. In order to extract such knowledge, one could divide the given database into a number of yearly databases. Also, for the purpose of mining change [20], one may require partitioning a given database. Thus, a single data source generates multiple databases. In a prediction problem, one requires analyzing the past events that would originate from the previous databases. Given such a problem, it might be strategically necessary to divide a large database into smaller databases. While dividing a large database, one requires selecting a certain time period. Selection of time period is an important decision, and it is dependent on the problem. For many problems, one could divide the database into yearly databases [13], [14], [15]. One needs to consider the time granularity as 1 year for certain problems because a season re-appears on a yearly basis and the customers’ purchase patterns might vary from season to season.

2.4 Database Thinning

Database thinning refers to the process of discarding the items (in the transactions) that are not relevant to a given context. These items could be treated as outliers. The thinning process makes the size of transactions shorter; thus, the mining process becomes easier. In order to estimate the association between the select items, or, to find the patterns of select items in multiple databases, we may wish to discard other items in the local databases. One could apply database thinning to each local database [3].

2.5 Ordering of Databases

In the context of mining multiple large databases, the order of mining each local database seems to be an important issue. Mining multiple large databases could be thought of as a two-step process: mining each local database using a single database mining technique (SDMT) by applying an ordering method or a model, then synthesizing the patterns (derived from that mining) using an algorithm. By applying the above strategy, we proposed the pipelined feedback technique (PFT) [11] for mining multiple large databases.

2.6 Selection of Databases

An approximate form of knowledge resulting from a large number of databases would be adequate for many decision support applications. In this sense, the selection of databases might be important in many decision support applications as it reduces the cost of searching for necessary information. Their selection could be based on the inherent knowledge residing in the databases. For that purpose, we need to mine each local database. Then, we process the local patterns in different databases for the purpose of selecting the relevant databases. Based on the local patterns, one could cluster the local databases for processing relevant queries [10], [61].

2.7 Integration of Databases

In the context of a multi-database environment, different databases are required to be integrated with the data mining system. We need to respect the rights associated with each data set, as we deal with company’s own data, third-party data, and customers’ data. Lu [41] discussed the problem of seamless integration of data mining with DBMS and applications from three directions. These days, most of the database management systems have extended their functionality in data analysis. Such capability should be fully explored to develop DBMS-aware data mining algorithms. Another type of integration includes algorithm selection and parameter setting. Reducing or eliminating mining parameters as much as possible, and developing automatic or semi-automatic mining algorithm selection techniques will greatly increase the application friendliness of data mining systems. Lastly, standardizing the interface among databases, data mining algorithms, and applications can also facilitate the integration to certain extent.

2.8 Other Data Preparation Activities

Primitive data preprocessing techniques such as data cleaning, data transformation, data reduction, and data summarization make data fit for the mining tasks [31].

3.1 Local Pattern Analysis

In the context of a multi-database environment, the patterns in databases could be classified into local patterns, global patterns, and patterns that are neither local nor global. Patterns in a local (branch) database are termed as local patterns. Local patterns can be used for local data analysis and decision making problems [10], [57]. Global patterns are based on all the databases that are under consideration. They are useful for global data analyses [7], [58]. One could mine the global patterns by mining each local database, then by analyzing all the local patterns in different databases. This technique is simply called local pattern analysis. Zhang et al. [64] designed local pattern analysis for the purpose of addressing various problems related to multiple large databases.

Let us assume that there are n branches in a multi-branch company. Also, let D_i be the database corresponding to the i-th branch, i=1, 2, …, n. The essence of mining the multiple databases using local pattern analysis could be explained using Figure 1 (see Chapter 1 of Ref. [8] for more details).

Figure 1:

Mining Patterns in Multiple Databases Using Local Pattern Analysis.

Here, LPB_i denotes the local pattern base for D_i, i=1, 2, …, n. Multi-database mining using local pattern analysis could be considered as an approximate method for mining multiple large databases.

An extended model of local pattern analysis was reported by Adhikari and Rao [7]. It suggested an idea of mining the patterns in multiple databases using a systematic approach. In order to enhance the quality of knowledge, it might be required to synthesize a huge amount of patterns in different local databases. There is need to store the patterns efficiently. Thus, IS coding [10] and ACP coding [9] for storing the local frequent itemsets and local association rules, respectively, are proposed.

3.2 Pipelined Feedback Technique

Consider a multi-branch organization with n branch databases. Let W_i be the data warehouse corresponding to the i-th branch, i=1, 2, …, n. Then, the local patterns for the i-th branch are extracted from W_i, i=1, 2, …, n. We mine each data warehouse using an SDMT. PFT is elaborated using Figure 2 (see Chapter 6 of Ref. [4] for more details).

Figure 2:

PFT of Mining Multiple Databases.

In PFT, W₁ is mined using an SDMT and the local pattern base LPB₁ is extracted. While mining W₂, all the patterns in LPB₁ are extracted irrespective of their values of interestingness measures, such as minimum support and minimum confidence. Apart from these patterns, some new patterns that satisfy the user-defined threshold values of interestingness measures are also extracted. In general, while mining W_i, all the patterns in W_i−1 are mined irrespective of their values of interestingness measures, and some new patterns that satisfy the user-defined threshold values of interestingness measures, i=2, 3, …, n, are extracted. Due to this nature of mining each data warehouse, PFT is called a feedback technique. Thus, |LPB_i−1|≤|LPB_i|, for i=2, 3, …, n. There are n arrangements of pipelining for n databases. All the arrangements of data warehouses might not produce the same mining result. If the number of local patterns increases, we get more accurate global patterns and a better analysis of local patterns. An arrangement of data warehouses would produce near optimal result if |LPB_n| is maximal. Let size(W_i) be the size of W_i (in bytes), i=1, 2, …, n. We shall follow the following rule of thumb regarding the arrangements of data warehouses for the purpose of mining. The number of patterns in W_i is greater than or equal to the number of patterns in W_i−1 if size(W_i)≥size(W_i−1), i=2, 3, …, n. For the purpose of increasing the number of local patterns, W_i precedes W_i−1 in the pipelined arrangement of mining data warehouses if size(W_i)≥size(W_i−1), i=2, 3, …, n. Finally, we analyze the patterns in LPB₁, LPB₂, …, and LPB_n for synthesizing the global patterns or for analyzing the local patterns.

Let W be the collection of data from all the warehouses. For synthesizing the global patterns in W, we discuss here a simple pattern synthesizing formula. Without any loss of generality, let the itemset X be extracted from the first m databases, for 1≤m≤n. Then, the synthesized support of X in W could be obtained as follows:

3.3 Distributed Approaches

Applications with distributed data are very common. Data analysis using distributed data is a challenging issue. A wireless ad hoc network can have multiple sensors that collect data continuously. Moving such data to a central location may not be a desirable objective. Thus, computation must be done in a distributed manner. An earth observatory system may consist of multiple satellites, and they send the data to their control centers. A distributed data analysis might be a critical approach to gain knowledge inherent in these datasets. Park and Kargupta [50] presented different issues on DDM algorithms. Tsoumakas [58] discussed some classical distributed problems such as distributed classification and regression, distributed association rule mining, and distributed clustering.

3.4 Other Techniques

Khiat et al. [33] applied the maximum entropy method to query selectivity estimation, and it consists of the following steps:

1. Construct a graph clique tree structure to gather the common distributions.

2. Apply an iterative scaling algorithm as a convergence of maximum entropy solution for reducing the complexity estimation.

3. Perform the bucket elimination technique for each clique tree cluster for accelerating the estimation of common factors in the scale.

Mining multiple databases could be done using partition algorithm proposed by Savasere et al. [54]. Each database may require to be partitioned because a local database could be large. Thus, the original partition algorithm may need to be modified. The algorithm was designed for mining a very large database by partitioning it. The algorithm was stated as follows. It scans the database twice. The database is divided into disjoint partitions, where each partition is small enough to fit in the memory. In the first scan, the algorithm reads each partition and computes the locally frequent itemsets in each partition using the Apriori algorithm [16]. In the second scan, the algorithm counts the supports of all locally frequent itemsets toward the complete database. In this case, each local database could be considered as a partition. Though this partition algorithm mines frequent itemsets in a database exactly, it might be an expensive solution to mine multiple big databases because each database is required to be scanned twice. During the time of the second scanning, all the local patterns obtained during the first scan are analyzed. It remains an expensive technique for mining multiple big databases.

Wu and Zhang [62] have proposed the RuleSynthesizing algorithm for synthesizing high-frequency association rules in multiple databases. Using this technique, every local database is mined separately at random order using an SDMT for synthesizing high-frequent association rules. A pattern in a local database is assumed as zero if it does not get reported. Based on the association rules in different databases, the authors have estimated the weights of different databases. Let w_i be the weight of the i-th database, i=1, 2, …, n. Without any loss of generality, let the association rule r be extracted from the first m databases, 1≤m≤n. supp_a(r, D_i) has been assumed as 0, i=m+1, m+2, …, n. Here, D_i represents the i-th database, i=1, 2, …, n. Then, the support of r in D has been synthesized as follows:

where D represents the union of local databases. The algorithm RuleSynthesizing is an indirect approach for synthesizing the association rules in multiple databases. The authors used the same weights for local databases while synthesizing confidence of r in D. It requires further elaboration whether the same weights are justifiable to conditional probabilities while synthesizing the confidence of a rule.

Big data [18] is a broad term used for datasets that are so large or complex that traditional data processing applications are inadequate to process them. One could assume here that a big database cannot be mined using a traditional data mining technique. A technique similar to PFT, presented in Section 3.2, could be designed for mining multiple big databases, and we wish to address this issue in our future work.