Skip to main content
SpringerLink
Log in

Revisiting the cube lifecycle in the presence of hierarchies

  • Regular Paper
  • Published:
The VLDB Journal Aims and scope Submit manuscript

Abstract

On-line analytical processing (OLAP) typically involves complex aggregate queries over large datasets. The data cube has been proposed as a structure that materializes the results of such queries in order to accelerate OLAP. A significant fraction of the related work has been on Relational-OLAP (ROLAP) techniques, which are based on relational technology. Existing ROLAP cubing solutions mainly focus on “flat” datasets, which do not include hierarchies in their dimensions. Nevertheless, as shown in this paper, the nature of hierarchies introduces several complications into the entire lifecycle of a data cube including the operations of construction, storage, indexing, query processing, and incremental maintenance. This fact renders existing techniques essentially inapplicable in a significant number of real-world applications and mandates revisiting the entire cube lifecycle under the new perspective. In order to overcome this problem, the CURE algorithm has been recently proposed as an efficient mechanism to construct complete cubes over large datasets with arbitrary hierarchies and store them in a highly compressed format, compatible with the relational model. In this paper, we study the remaining phases in the cube lifecycle and introduce query-processing and incremental-maintenance algorithms for CURE cubes. These are significantly different from earlier approaches, which have been proposed for flat cubes constructed by other techniques and are inadequate for CURE due to its high compression rate and the presence of hierarchies. Our methods address issues such as cube indexing, query optimization, and lazy update policies. Especially regarding updates, such lazy approaches are applied for the first time on cubes. We demonstrate the effectiveness of CURE in all phases of the cube lifecycle through experiments on both real-world and synthetic datasets. Among the experimental results, we distinguish those that have made CURE the first ROLAP technique to complete the construction and usage of the cube of the highest-density dataset in the APB-1 benchmark (12 GB). CURE was in fact quite efficient on this, showing great promise with respect to the potential of the technique overall.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Acharya, S., Gibbons, P.B., Poosala, V.: Congressional samples for approximate answering of group-by queries. In: Proceedings of ACM Special Interest Group on Management of Data (SIGMOD), pp. 487–498 (2000)

  2. Agarwal, S., Agrawal, R.,Deshpande, P., Gupta, A.,Naughton, J.F., Ramakrishnan, R., Sarawagi, S.: On the computation of multidimensional aggregates. In: Proceedings of Very Large Data Bases (VLDB), pp. 506–521 (1996)

  3. Beyer, K.S., Ramakrishnan, R.: Bottom-up computation of sparse and iceberg cubes. In: Proceedings of ACM Special Interest Group on Management of Data (SIGMOD), pp. 359–370 (1999)

  4. Blackard, J.A.: The forest covertype dataset. ftp://ftp.ics.uci.edu/pub/machine-learning-databases/covtype

  5. Chen M.S., Han J., Yu P.S.: Data mining: an overview from a database perspective. IEEE Trans. Knowl. Data Eng. 8(6), 866–883 (1996)

    Article  Google Scholar 

  6. Colby, L.S., Griffin, T., Libkin, L., Mumick, I.S., Trickey, H.: Algorithms for deferred view maintenance. In: Proceedings of ACM Special Interest Group on Management of Data (SIGMOD), pp. 469–480 (1996)

  7. Feng, J., Si, H., Feng, Y.: Indexing and incremental updating condensed data cube. In: Proceedings of International Conference on Scientific and Statistical Database Management (SSDBM), pp. 23–32 (2003)

  8. Feng, Y., Agrawal, D., Abbadi, A.E., Metwally, A.: Range cube: Efficient cube computation by exploiting data correlation. In: Proceedings of International Conference on Data Engineering (ICDE), pp. 658–670 (2004)

  9. Gray, J., Bosworth, A., Layman, A., Pirahesh, H.: Data cube: A relational aggregation operator generalizing group-by, cross-tab, and sub-total. In: Proceedings of International Conference on Data Engineering (ICDE), pp. 152–159 (1996)

  10. Hahn, C., Warren, S., London, J.: Synoptic cloud reports. http://cdiac.esd.ornl.gov/cdiac/ndps/ndp026b.html

  11. Han, J., Pei, J., Dong, G., Wang, K.: Efficient computation of ice-berg cubes with complex measures. In: Proceedings of ACM Special Interest Group on Management of Data (SIGMOD), pp. 1–12 (2001)

  12. Harinarayan, V., Rajaraman, A., Ullman, J.D.: Implementing data cubes efficiently. In: Proceedings of ACM Special Interest Group on Management of Data (SIGMOD), pp. 205–216 (1996)

  13. Jagadish, H.V., Lakshmanan, L.V.S., Srivastava, D.: What can hierarchies do for data warehouses? In: Proceedings of Very Large Data Bases (VLDB), pp. 530–541 (1999)

  14. Karayannidis, N., Sellis, T.K., Kouvaras, Y.: Cube file: a file structure for hierarchically clustered olap cubes. In: Proceedings of International Conference on Extending Database Technology (EDBT), pp. 621–638 (2004)

  15. Kotidis, Y., Roussopoulos, N.: An alternative storage organization for rolap aggregate views based on cubetrees. In: Proceedings of ACM Special Interest Group on Management of Data (SIGMOD), pp. 249–258 (1998)

  16. Kotsis, N., McGregor, D.R.: Elimination of redundant views in multidimensional aggregates. In: Proceedings of Data Warehousing and Knowledge Discovery (DaWaK), pp. 146–161 (2000)

  17. Lakshmanan, L.V.S., Pei, J., Han, J.: Quotient cube: How to summarize the semantics of a data cube. In: Proceedings of Very Large Data Bases (VLDB), pp. 778–789 (2002)

  18. Lakshmanan, L.V.S., Pei, J., Zhao, Y.: Qc-trees: an efficient summary structure for semantic olap. In: Proceedings of ACM Special Interest Group on Management of Data (SIGMOD), pp. 64–75 (2003)

  19. Lee, K.Y., Kim, M.H.: Efficient incremental maintenance of data cubes. In: Proceedings of Very Large Data Bases (VLDB), pp. 823–833 (2006)

  20. Li, C., Cong, G., Tung, A.K.H.,Wang, S.: Incremental maintenance of quotient cube for median. In: Proceedings of International Conference on Knowledge Discovery and Data Mining (KDD), pp. 226–235 (2004)

  21. Li C., Tung K.H., Wang S.: Incremental maintenance of quotient cube based on galois lattice. J. Comput. Sci. Technol. 19(3), 302–308 (2004)

    Article  Google Scholar 

  22. Morfonios, K., Ioannidis, Y.: CURE for cubes: cubing using a ROLAP engine. In: Proceedings of Very Large Data Bases (VLDB), pp. 379–390 (2006)

  23. Morfonios K., Ioannidis Y.E.: Supporting the data cube lifecycle: the power of ROLAP. VLDB J. 17(4), 729–764 (2008)

    Article  Google Scholar 

  24. Morfonios K., Konakas S., Ioannidis Y.E., Kotsis N.: ROLAP implementations of the data cube. ACM Comput. Surv. 39(4), 12:1–12:53 (2007)

    Article  Google Scholar 

  25. Mumick, I.S., Quass, D., Mumick, B.S.: Maintenance of data cubes and summary tables in a warehouse. In: Proceedings of ACM Special Interest Group on Management of Data (SIGMOD), pp. 100–111 (1997)

  26. OLAP Council: Apb-1 olap benchmark. http://www.olapcouncil.org

  27. Pedersen, T.B., Jensen, C.S., Dyreson, C.E.: The treescape system: Reuse of pre-computed aggregates over irregular olap hierarchies. In: Proceedings of Very Large Data Bases (VLDB), pp. 595–598 (2000)

  28. Ross, K.A., Srivastava, D.: Fast computation of sparse datacubes. In: Proceedings of Very Large Data Bases (VLDB), pp. 116–125 (1997)

  29. Roussopoulos N., Economou N., Stamenas A.: Adms: a test-bed for incremental access methods. IEEE Trans. Knowl. Data Eng. 5(5), 762–774 (1993)

    Article  Google Scholar 

  30. Roussopoulos, N., Kotidis, Y., Roussopoulos, M.: Cubetree: Orga nization of and bulk updates on the data cube. In: Proceedings of ACM Special Interest Group on Management of Data (SIGMOD), pp. 89–99 (1997)

  31. Sarawagi, S., Agrawal, R., Gupta, A.: On computing the data cube. In: Research report 10026. IBM Almaden Research Center, San Jose (1996)

  32. Shao, Z., Han, J., Xin, D.: Mm-cubing: computing iceberg cubes by factorizing the lattice space. In: Proceedings of International Conference on Scientific and Statistical Database Management (SSDBM), pp. 213–222 (2004)

  33. Shukla, A., Deshpande, P., Naughton, J.F.: Materialized view selection for multidimensional datasets. In: Proceedings of Very Large Data Bases (VLDB), pp. 488–499 (1998)

  34. Sismanis, Y., Deligiannakis, A., Kotidis, Y., Roussopoulos, N.: Hierarchical dwarfs for the rollup cube. In: Proceedings of ACM International Workshop on Data Warehousing and OLAP (DOLAP), pp. 17–24 (2003)

  35. Sismanis, Y., Deligiannakis, A., Roussopoulos, N., Kotidis, Y.: Dwarf: shrinking the petacube. In: Proceedings of ACM Special Interest Group on Management of Data (SIGMOD), pp. 464–475 (2002)

  36. Sismanis, Y., Roussopoulos, N.: The complexity of fully material ized coalesced cubes. In: Proceedings of Very Large Data Bases (VLDB), pp. 540–551 (2004)

  37. Timko, I., Dyreson, C.E., Pedersen, T.B.: Pre-aggregation with probability distributions. In: Proceedings of ACM International Workshop on Data Warehousing and OLAP (DOLAP), pp. 35–42 (2006)

  38. Vassiliadis P., Sellis T.K.: A survey of logical models for olap databases. SIGMOD Rec. 28(4), 64–69 (1999)

    Article  Google Scholar 

  39. Vitter, J.S., Wang, M.: Approximate computation of multidimensional aggregates of sparse data using wavelets. In: Proceedings of ACM Special Interest Group on Management of Data (SIGMOD), pp. 193–204 (1999)

  40. Wang, W., Feng, J., Lu, H., Yu, J.X.: Condensed cube: an efficient approach to reducing data cube size. In: Proceedings of International Conference on Data Engineering (ICDE), pp. 155–165 (2002)

  41. Xin, D., Han, J., Li, X., Wah, B.W.: Star-cubing: computing ice-berg cubes by top-down and bottom-up integration. In: Proceedings of Very Large Data Bases (VLDB), pp. 476–487 (2003)

  42. Zhao, Y., Deshpande, P., Naughton, J.F.: An array-based algorithm for simultaneous multidimensional aggregates. In: Proceedings of ACM Special Interest Group on Management of Data (SIGMOD), pp. 159–170 (1997)

  43. Zhou, J., Larson, P.Å., Elmongui, H.G.: Lazy maintenance of materialized views. In: Proceedings of Very Large Data Bases (VLDB), pp. 231–242 (2007)

Download references

Author information

Authors and Affiliations

  1. IBM Almaden Research Center, 650 Harry Road, San Jose, CA, 95120, USA

    Konstantinos Morfonios

  2. Department of Informatics and Telecommunications, University of Athens, Athens, Greece

    Yannis Ioannidis

Authors
  1. Konstantinos Morfonios
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yannis Ioannidis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Konstantinos Morfonios.

Additional information

Work done while K. Morfonios was at the Department of Informatics and Telecommunications, University of Athens.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Morfonios, K., Ioannidis, Y. Revisiting the cube lifecycle in the presence of hierarchies. The VLDB Journal 19, 257–282 (2010). https://doi.org/10.1007/s00778-009-0160-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00778-009-0160-3

Keywords

Search

Navigation