Abstract
As data warehousing technology gains a ubiquitous presence in business today, companies are becoming increasingly reliant upon the information contained in their data warehouses to inform their operational decisions. This information, known as business intelligence (BI), traditionally has taken the form of nightly or monthly reports and batched analytical queries that are run at specific times of day. However, as the time needed for data to migrate into data warehouses has decreased, and as the amount of data stored has increased, business intelligence has come to include metrics, streaming analysis, and reports with expected delivery times that are measured in hours, minutes, or seconds. The challenge is that in order to meet the necessary response times for these operational business intelligence queries, a given warehouse must be able to support at any given time multiple types of queries, possibly with different sets of performance objectives for each type. In this paper, we discuss why these dynamic mixed workloads make workload management for operational business intelligence (BI) databases so challenging, review current and proposed attempts to address these challenges, and describe our own approach. We have carried out an extensive set of experiments, and report on a few of our results.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Arlitt, M.F.: Characterizing Web user sessions. SIGMETRICS Performance Evaluation Review 28(2), 50–63 (2000)
Bach, F.R., Jordan, M.I.: Kernel Independent Component Analysis. Journal of Machine Learning Research 3, 1–48 (2003)
Benoit, D.G.: Automated Diagnosis and Control of DBMS Resources. In: EDBT PhD. Workshop (2000)
Carey, M.J., Livny, M., Lu, H.: Dynamic Task Allocation In A Distributed Database System. In: ICDCS, pp. 282–291 (1985)
Chaudhuri, S., Kaushik, R., Ramamurthy, R.: When Can We Trust Progress Estimators for SQL Queries? In: Proc. of the ACM SIGMOD Intl. Conf. on Management of Data, pp. 575–586 (2005)
Chaudhuri, S., Narasayya, V., Ramamurthy, R.: Estimating Progress of Execution for SQL Queries. In: Proc. of the ACM SIGMOD Intl. Conf. on Management of Data, pp. 803–814 (2004)
Davison, D.L., Graefe, G.: Dynamic Resource Brokering for Multi-User Query Execution. In: Proc. of the ACM SIGMOD Intl. Conf. on Management of Data, pp. 281–292 (1995)
Dayal, U., Kuno, H., Wiener, J.L., Wilkinson, K., Ganapathi, A., Krompass, S.: Managing operational business intelligence workloads. SIGOPS Oper. Syst. Rev. 43(1), 92–98 (2009)
Eeckhout, L., Vandierendonck, H., Bosschere, K.D.: How Input Data Sets Change Program Behaviour. In: 5th Workshop on Computer Architecture Evaluation Using Commercial Workloads (2002)
Elnaffar, S., Martin, P., Horman, R.: Automatically Classifying Database Workloads. In: Proc. of ACM Conference on Information and Knowledge Management (CIKM), pp. 622–624 (2002)
Ganapathi, A., Kuno, H., Dayal, U., Wiener, J., Fox, A., Jordan, M., Patterson, D.: Predicting Multiple Metrics for Queries: Better Decisions Enabled by Machine Learning. In: Proc. of the 21st Intl. Conf. on Data Engineering, ICDE (2009)
Gillin, P.: BI @ the Speed of Business. Computer World Technology (December 2007)
Gupta, C., Mehta, A.: PQR: Predicting Query Execution Times for Autonomous Workload Management. In: Proc. Intl Conf on Autonomic Computing, ICAC (2008)
Keeton, K., Patterson, D.A., He, Y.Q., Raphael, R.C., Baker, W.E.: Performance Characterization of a Quad Pentium Pro SMP using OLTP Workloads. In: The 25th Intl. Symposium on Computer Architecture (ISCA), pp. 15–26 (1998)
Krompass, S., Gmach, D., Scholz, A., Seltzsam, S., Kemper, A.: Quality of Service Enabled Database Applications. In: Dan, A., Lamersdorf, W. (eds.) ICSOC 2006. LNCS, vol. 4294, pp. 215–226. Springer, Heidelberg (2006)
Krompass, S., Kuno, H., Dayal, U., Kemper, A.: Dynamic Workload Management for Very Large Data Warehouses: Juggling Feathers and Bowling Balls. In: Proc. of the 33rd Intl. Conf. on Very Large Data Bases, VLDB (2007)
Krompass, S., Kuno, H., Wiener, J.L., Wilkinson, K., Dayal, U., Kemper, A.: Managing long-running queries. In: EDBT 2009, pp. 132–143. ACM, New York (2009)
Lo, J.L., Barroso, L.A., Eggers, S.J., Gharachorloo, K., Levy, H.M., Parekh, S.S.: An Analysis of Database Workload Performance on Simultaneous Multithreaded Processors. In: The 25th Intl. Symposium on Computer Architecture (ISCA), pp. 39–50 (1998)
Luo, G., Naughton, J.F., Ellmann, C.J., Watzke, M.W.: Toward a Progress Indicator for Database Queries. In: Proc. of the ACM SIGMOD Intl. Conf. on Management of Data, pp. 791–802 (2004)
Luo, G., Naughton, J.F., Ellmann, C.J., Watzke, M.W.: Increasing the Accuracy and Coverage of SQL Progress Indicators. In: Proc. of the 21st Intl. Conf. on Data Engineering (ICDE), pp. 853–864 (2005)
Luo, G., Naughton, J.F., Yu, P.S.: Multi-query SQL Progress Indicators. In: Ioannidis, Y., Scholl, M.H., Schmidt, J.W., Matthes, F., Hatzopoulos, M., Böhm, K., Kemper, A., Grust, T., Böhm, C. (eds.) EDBT 2006. LNCS, vol. 3896, pp. 921–941. Springer, Heidelberg (2006)
Markl, V., Lohman, G.: Learning Table Access Cardinalities with LEO. In: Proc. of the ACM SIGMOD Intl. Conf. on Management of Data, p. 613 (2002)
Mehta, M., DeWitt, D.J.: Dynamic Memory Allocation for Multiple-Query Workload. In: Proc. of the 19th Intl. Conf. on Very Large Data Bases (VLDB) (August 1993)
Moore, J., Chase, J., Farkas, K., Ranganathan, P.: Data Center Workload Monitoring, Analysis, and Emulation (2005)
Schroeder, B., Harchol-Balter, M., Iyengar, A., Nahum, E.M.: Achieving Class-Based QoS for Transactional Workloads. In: Proc. of the 22nd Intl. Conf. on Data Engineering (ICDE), p. 153 (2006)
Stillger, M., Lohman, G.M., Markl, V., Kandil, M.: LEO - DB2’s LEarning Optimizer. In: Proc. of the 27th Intl. Conf. on Very Large Data Bases (VLDB), pp. 19–28 (2001)
Weikum, G., Hasse, C., Mönkeberg, A., Zabback, P.: The COMFORT Automatic Tuning Project. Information Systems 19(5), 381–432 (1994)
White, C.: The Next Generation of Business Intelligence: Operational BI. DM Review Magazine (May 2005)
Yoo, R.M., Lee, H., Chow, K., Lee, H.-H.S.: Constructing a Non-Linear Model with Neural Networks for Workload Characterization. In: IISWC, pp. 150–159 (2006)
Yu, P.S., Chen, M.-S., Heiss, H.-U., Lee, S.: On Workload Characterization of Relational Database Environments. Software Engineering 18(4), 347–355 (1992)
Zhang, N., Haas, P.J., Josifovski, V., Lohman, G.M., Zhang, C.: Statistical Learning Techniques for Costing XML Queries. In: Proc. of the 31st Intl. Conf. on Very Large Data Bases (VLDB), pp. 289–300 (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kuno, H., Dayal, U., Wiener, J.L., Wilkinson, K., Ganapathi, A., Krompass, S. (2010). Managing Dynamic Mixed Workloads for Operational Business Intelligence. In: Kikuchi, S., Sachdeva, S., Bhalla, S. (eds) Databases in Networked Information Systems. DNIS 2010. Lecture Notes in Computer Science, vol 5999. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-12038-1_2
Download citation
DOI: https://doi.org/10.1007/978-3-642-12038-1_2
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-12037-4
Online ISBN: 978-3-642-12038-1
eBook Packages: Computer ScienceComputer Science (R0)