Cluster-By: An Efficient Clustering Operator in Emergency Management Database Systems

Abstract

Database management systems (DBMS) have been widely used to efficiently store, manage and analysis large emergency management data. Despite the popularity of clustering as a general data mining method, current emergency management database systems lacked a unified and convenient way to support in-database clustering. In this paper we promote the advantages of integrating clustering into databases and propose a new Cluster-by SQL extension. We formally define the syntax and semantics of the Cluster-by clause, illustrate its query plan node in database engine and present two data preprocessing rules. Then we explore the query optimization opportunities, present a novel framework for multiquery optimization and define the cost model for multi-query scheduling. We also introduce DBSCAN-based Shrink and Expand algorithms to utilize the historical clustering results and present a heuristic cost model. To demonstrate the integration of the extension with existing DBMSs, we implemented the Cluster-by extension in PostgreSQL. We performed experiments on real data sets in PostgreSQL. Results show that Cluster-by extension is useful, the multiquery optimization techniques proposed are efficient.