Efficiency evaluation of data warehouse operations

doi:10.1016/j.dss.2007.10.011

Decision Support Systems

Volume 44, Issue 4, March 2008, Pages 883-898

https://doi.org/10.1016/j.dss.2007.10.011 Get rights and content

Abstract

We evaluate an efficiency model for data warehouse operations using data from USA and non-USA-based (mostly Korean) organizations. The analysis indicates wide dispersions in operational efficiency, industry and region differences, large differences in labor budgets between efficient and inefficient firms, few organizations efficient in both refresh processing and query production, and difficulty of providing some variables. Follow-up interviews provide insights about the value of efficiency comparisons of information technology organizations and suggestions to improve the model. Using this analysis, we propose a framework containing data warehouse characteristics and firm characteristics to explain IT operational efficiency at the subfirm level.

Introduction

Data warehouse, a term coined by William Inmon in 1990, refers to a central data repository where data from operational databases and other sources are integrated, cleaned, and archived to support decision-making. A data warehouse provides management with convenient access to large volumes of internal and external data. Because of the potential benefits, most medium to large organizations operate data warehouses. Many of these organizations have operated data warehouses for five years or more with continuing development to increase the size and scope of the data warehouses.

As data warehouse technology and deployment matures, efficient operation becomes a priority. Due to complex data requests, large volumes of data, incompatibilities among data sources, and other complicating factors, operating a data warehouse may involve high costs for complex hardware/software architectures and significant labor support. To measure and improve the efficiency of delivering information products, organizations should strive to compare with other peer organizations.

In this paper, we present a model to evaluate organizations operating significant data warehouses. The emphasis in our model is to measure relative efficiency of an information technology (IT) organization providing a significant information service, usually for internal usage. The model contains salient variables to evaluate refresh processing and query production, two major operations for data warehouses. The variables in the model include traditional resource consumption (labor usage and computing budgets), system usage measures (users and queries), data quality measures (timeliness and availability), and a size measure (amount of change data). To assess the model, we analyze a data set using Data Envelopment Analysis (DEA), comparing efficiencies for significant subsets of the model. In addition to the formal analysis, we propose a framework to explain the efficiency of information technology operations and provide anecdotal evidence about the difficulties of the evaluation.

The results of this study have important implications for quantitative evaluation of IT service organizations, particularly those providing complex data products. Efficiency models for complex data products support evaluation of tradeoffs between costs and data quality levels. Although there are numerous studies of information technology at the firm level, this paper emphasizes subfirm efficiency. As far as we are aware, this paper proposes the first efficiency model for data warehouse operations along with analysis of a data set and a framework to explain subfirm operations.

This paper is organized as follows. Section 2 summarizes related work about data warehouse operations, evaluations of data warehouse success, and efficiency evaluations of information technology. Section 3 presents the efficiency model and data collection details. Section 4 analyzes the efficiency results and discusses follow-up interviews with selected organizations about the impact of the results. Section 5 concludes the study.

Section snippets

Related work

Data warehouse operations have been studied in detail since the early 1990s primarily with a focus on the refresh process. Although most of the research involves algorithms, some of the research involves the representation and control of the refresh process. Bouzeghoub et al. [5] developed a flexible workflow model to represent the refresh process. Their conceptual model provides a framework to understand the details of refresh processes including constraints on data sources and data

Measuring efficiency of data warehouse operations

The emphasis in our model is to measure relative efficiency of an IT organization providing a significant information service, usually for internal usage. Efficiency is defined as the ratio of outputs to inputs. IT organizations are classified as efficient or inefficient relative to the peer IT organizations in the data set. A statement that an IT organization is efficient means that no other IT organizations are more efficient considering the combination of inputs and outputs in the data set.

Analysis of operational efficiency

Data was solicited from three groups in the second half of 2005. The Center for Information Technology Innovation (CITI) at the University of Colorado at Denver is group of Chief Information Officers from organizations with significant operations in the Denver, Colorado area. Surveys were sent to this group in June 2005 with a response rate of 10 out of 20 organizations.¹ The BI Network (//www.B-EYE-Network.com

Conclusion

We presented an efficiency model for evaluating data warehouse operations and evaluated the relative efficiencies of a data set of organizations operating significant data warehouses. The efficiency model supports evaluation of refresh and query production processes for a collection of data warehouses managed by an IT organization. The variables in the models include traditional resource consumption (labor usage and computing budgets), system usage measures (number of queries, number of users,

References (30)

M. Mannino et al.
A field study and framework about data warehouse refresh policies
Decision Support Systems
(September 2006)
E. Oz
Information technology productivity: in search of a definitive observation
Information & Management
(2005)
C. Serrano-Cinca et al.
Measuring DEA efficiency in Internet companies
Decision Support Systems
(2005)
S. Shafer et al.
A framework for measuring the efficiency of organizational investments in information technology using data envelopment analysis
Omega
(2000)
B. Shao et al.
Technical efficiency analysis of information technology investments: a two-stage empirical investigation
Information & Management
(2002)
R. Banker et al.
A model to evaluate variables impacting the productivity of software maintenance projects
Management Science
(1991)
R. Banker et al.
Measuring gains in operational efficiency from information technology: a study of Positran Deployment at Hardee's Inc.
JMIS
(1990)
R. Banker et al.
Some models for estimating technical and scale inefficiencies in DEA
Management Science
(1984)
A. Basilevsky
Statistical Factor Analysis and Related Methods: Theory and Applications
(1994)
M. Bouzeghoub et al.
Modeling data warehouse refreshment process as a workflow application

M. Breunig et al.

LOF: Identifying Distance Based Local Outliers

Y. Chen et al.

Measuring information technology's indirect impact on firm performance

Information Technology and Management

(2004)

W. Cooper et al.

Measuring the efficiency of decision making units

European Journal of Operational Research

(1978)

W. Cooper et al.

Data Envelopment Analysis — A Comprehensive Text with Models, Applications, References and DEA-Solver Software

(2000)

B. Haley et al.

The benefits of data warehousing at Whirlpool

Annals of Cases on Information Technology Applications and Management in Organizations

(1999)

Cited by (33)

Data Envelopment Analysis of linguistic features and passage relevance for open-domain Question Answering
2022, Knowledge-Based Systems
Question Answering (QA) systems play an important role in today’s human–computer interaction systems. QA performance can be significantly improved using effective answer passage retrieval and ranking techniques. Our focus in this paper is on both non machine learning-based and deep learning-based passage retrieval and ranking systems for QA to leverage linguistic features within the text of questions and passages and improve passage ranking effectiveness. We propose a decoupled linguistic and linear programming-based approach for passage ranking using the Data Envelopment Analysis (DEA) technique to improve over well-established answer passage retrieval techniques. Our method scores passages using information retrieval and deep learning relevance metrics, represents retrieved passages using their relevance scores and several linguistic features, and finally makes use of DEA to re-rank the retrieved list of passages. The high effectiveness and significance of our proposed passage ranking method is demonstrated based on several experiments that we have conducted on a standard benchmark data set.
Common Association Rules for Dispersed Information Systems
2022, Procedia Computer Science
Association rules are popular form for knowledge discovery domain. They are used for finding interesting relationships and patterns hidden in large data sets and in the area of associative classification, where usually rules with one item in the right-hand side are considered. There are many different approaches and algorithms for mining association rules. One of the most popular group are methods which are based on mining frequent itemsets, usually applied for data in transaction format. Such data can be transformed to binary information system which corresponds to matrix data format. Technological development means that we are dealing with an increasing amount of data that can be heterogeneous, taking into account their format and location. In this paper, we assume that dispersed data is represented by a finite set S of information systems with equal sets of attributes. We discuss one of the possible ways to the study association rules common to all information systems from the set S: building a joint information system for which the set of true association rules that are realizable for a given row r and have given attribute f on the right-hand side coincides with the set of association rules that are true for all information systems from S, are realizable for the row r, and have the attribute f on the right-hand side. We show how to build a joint information system in a polynomial time. When we build such an information system, we can apply to it various association rule learning algorithms.
Benchmarking IT operations cost based on working time and unit cost
2017, Science of Computer Programming
Recently, size of information system gets large, and the information system operation (IT operations) is often outsourced. When IT operations of the large system is outsourced, high cost is needed, and troubles in IT operations may affect the activity of the company. Therefore, the information system operation is important for the companies. Cost is one of the important factors when the system operation is outsourced. However, it is not easy for the customers (system users) to judge whether the operation cost is valid or not. To support the judgment, we focus on information which the customers can know (e.g., size of software), to estimate the working time. In the analysis, we clarified the factors which affect the working time. Then, data was stratified based on the factors, to show the benchmark of working time. Using the benchmark, customers estimate the working time roughly. Also, we clarified the factors which affect the unit cost, and showed the benchmark. Operation cost can be estimated, by estimated working time multiplied by the estimated unit cost. The analysis results showed that the process standardization relates to the working time of operators. Also, the network range and the contract type have a relationship to the unit cost of operators. Work efficiency and unit cost do not affect operation quality.
Business intelligence and organizational learning: An empirical investigation of value creation processes
2017, Information and Management
Citation Excerpt :
To the best of our knowledge, this study is the first to formally model BI infrastructure as mediating the value contribution of the BI team. Previous research has focused on the effects of BI infrastructure (e.g., Refs. [68,92]) or observed the effects of both BI infrastructure and team without formally analyzing the interrelationships between them [75,103][e.g.,75,103]. However, this mediated effect alone fails to capture the full relationship between BI assets and capabilities because the BI team may influence BI capabilities directly, without mediation by the BI infrastructure.
With the aim of bridging the gap between well-established research on information technology (IT) value creation and the emergent study of business intelligence (BI), this study develops and tests a model of BI value creation that is firmly anchored in both streams of research. The analysis draws on the resource-based view and on conceptualizations of organizational learning to hypothesize about the paths by which BI assets and BI capabilities create business value. The research model is first assessed in an exploratory analysis of data collected through interviews in three firms and then tested in a confirmatory analysis of data collected through a survey.
Efficient maintenance of basic statistical functions in data warehouses
2014, Decision Support Systems
Citation Excerpt :
In other words, when a base table is changed, the data warehouse will use these auxiliary data to maintain the view instead of reaccessing the entire base table, significantly improving the maintenance processes. Since data warehouses often contain huge amounts of data, in order to keep performance smooth and constant, maintenance of a data warehouse is crucial. [16] suggested an efficiency model for data warehouse operations which contains two major processes (refresh processing and query production), and concluded that only few organizations can obtain good efficiency for both processes.
In general, some simple but very meaningful statistical functions are often used to retrieve valuable summary information in corporate databases. However, it is not uncommon that such information is obtained from computerized information systems which spend a great deal of time calculating the large volume of collected data. In practice, such data is usually stored in a data warehouse in which a large number of summary tables or materialized aggregate views are built in order to improve the system performance. Upon changes, most notable new transactional data are collected from various data sources, and all summary tables in the data warehouse that correspond to the transactional data must be updated accordingly. Since the number of summary tables that need to be maintained is often large, efficiently maintaining these is thus a critical issue for managing a data warehouse. In this study, an efficient maintenance approach to enhance the performance of a data warehouse is proposed, in which some additional auxiliary tables are kept inside a data warehouse with the role of improving the maintenance processes of some statistical functions, such as MIN, MAX, MEAN, and MEDIAN. Finally, a comparative analysis is performed to verify the effectiveness of the proposal method.
An integrated multiple criteria decision making model applying axiomatic fuzzy set theory
2012, Applied Mathematical Modelling
This article presents a hybrid model for the multiple criteria decision making problems. The proposed decision model consists of three parts: (i) DEA (data envelopment analysis) is used to provide the best combination on the performance parameters of original data; (ii) By the application of AFS (axiomatic fuzzy set) theory and AHP (analytic hierarchy process) method, the weight of each attribute is calculated and (iii) TOPSIS (technique for order preference by similarity to ideal solution) is applied to provide the ranking order of that best combination based on the weights of attributes. In addition, we also provide the definitely semantic interpretations for the decision results by AFS theory. Specially, the model not only employs the performance parameters from raw data, but also considers the preferences from decision-makers that can make the decision results more reasonable. The proposed model is used for robot selection to verify the proposed model. Using the selection index, the evaluation of alternative robots and the selection of the most appropriate are eventually feasible. Moreover, a numerical example for supplier selection is included to illustrate the application of the model for the newly developed problems.

View all citing articles on Scopus

Michael V. Mannino is an associate professor in the Business School of the University of Colorado Denver. Previously he was on the faculty at the University of Florida, University of Texas at Austin, and University of Washington. He has been active in research in database management, knowledge representation, and organizational impacts of technology. He has articles published in major journals of the IEEE (Transactions on Knowledge and Data Engineering and Transactions on Software Engineering), ACM (Communications of the ACM and Computing Surveys), and INFORMS (Informs Journal on Computing and Information Systems Research). His research efforts have produced several popular survey and tutorial articles as well as many papers describing original research. He is the author of the textbook, Database Design, Application Development, and Administration, published by Irwin McGraw-Hill.

Sa Neung Hong is an Associate Professor of Faculty of Business at the University of Seoul, Korea. He received his Ph.D. degree from the University of Texas, Austin. His research has appeared in Decision Support Systems and ORSA Journal on Computing. His research interests include Information Management, Service Computing, Very Large Scale Systems, IT Governance and ITSM. He has worked as a CIO of a major bank in Korea and has been providing consulting service very actively for IT Management and development of large Information Systems to various Korean financial institutions.

Injun Choi received the Ph.D. degree in Management Information System from University of Texas at Austin, in 1991. He is currently a full professor in the department of industrial and management engineering at the Pohang University of Science and Technology. His research interests include workflow and business process management, and knowledge management, object-oriented modeling and reasoning, and database systems.

View full text

Efficiency evaluation of data warehouse operations

Abstract

Introduction

Section snippets

Related work

Measuring efficiency of data warehouse operations

Analysis of operational efficiency

Conclusion

Decision Support Systems

Information & Management

Decision Support Systems

Omega

Information & Management

A model to evaluate variables impacting the productivity of software maintenance projects

Management Science

Measuring gains in operational efficiency from information technology: a study of Positran Deployment at Hardee's Inc.

JMIS

Some models for estimating technical and scale inefficiencies in DEA

Management Science

Statistical Factor Analysis and Related Methods: Theory and Applications

Modeling data warehouse refreshment process as a workflow application

LOF: Identifying Distance Based Local Outliers

Measuring information technology's indirect impact on firm performance

Information Technology and Management

Measuring the efficiency of decision making units

European Journal of Operational Research

Data Envelopment Analysis — A Comprehensive Text with Models, Applications, References and DEA-Solver Software

The benefits of data warehousing at Whirlpool

Annals of Cases on Information Technology Applications and Management in Organizations