A High Performance Hierarchical Cubing Algorithm and Efficient OLAP in High-Dimensional Data Warehouse

Abstract

Data cube has been playing an essential role in fast OLAP (online analytical processing) in many data warehouses. The pre-computation of data cubes is critical for improving the OLAP response time of in large high-dimensional data warehouses. However, as the sizes of data warehouses grow, the time it takes to perform this pre-computation becomes a significant performance bottleneck. In a high dimensional data warehouse, it might not be practical to build all these cuboids and their indices. In this paper, we propose a hierarchical cubing algorithm to partition the high dimensional data cube into low dimensional cube segments. It permits a significant reduction of CPU and I/O overhead for many queries by restricting the number of cube segments to be processed for both the fact table and bitmap indices. Experimental results show that the proposed method is significantly more efficient than other existing cubing methods.

The research in the paper is supported by the National Natural Science Foundation of China under Grant No. 70472033 and 60673060; the National Facilities and Information Infrastructure for Science and Technology of China under Grant No. 2004DKA20310; the National Tenth-Five High Technology Key Project of China under Grant No. 2003BA614A; the Natural Science Foundation of Jiangsu Province under Grant No. BK2005047 and BK2005046; the ‘Qing Lan’ Project Foundation of Jiangsu Province of China.