A data analytic benchmarking methodology for discovering common causal structures that describe context-diverse heterogeneous groups
Introduction
It is commonly acknowledged that operational excellence is one of the sources of competitive advantage of modern enterprises. Fundamentally, the concept refers to achieving a high level of efficiency of conversion of inputs into outputs, where a higher level of efficiency implies, ceteris paribus, a greater degree of excellence. Modern organizations typically regard information and communication technologies (ICTs) as one of the significant direct or indirect inputs for achieving such operational excellence and competitive advantage. Since the concept of competitive advantage involves a relative comparison of the performance of organizational entities then the concepts of organizational capabilities and benchmarking (e.g., Gouveia, Dias, Antunes, Boucinha, & Inácio, 2015) are relevant. Ayabakan, Bardhan, and Zheng (2017) noted that with regards to the investigation of this pair of concepts: “A dominant approach in IS research involves the use of survey instruments designed to elicit user responses on their perceptions about competencies and capabilities … A limitation of such perception-based approaches is that they represent a subjective measure of firm/organizational capabilities”; these researchers therefore proposed an approach that involves the use of non-subjective data and the data envelopment analysis (DEA) method for doing benchmarking (e.g., Adler, Liebert and Yazhemsky, 2013, LaPlante and Paradi, 2015) of the Input-Output conversion process. In this paper we also take a similar approach to benchmarking but are also interested in the context of the Input-Output conversion process, including those that involve ICTs as input(s). The motivating idea for this research project is that the process of benchmarking could possibly be enhanced by the discovery and application of non-obvious common causal structures that differentiate more efficient organizational entities from less efficient ones. This motivating idea triggered our intention to design an appropriate methodology artifact that involves the analysis of non-subjective data. This research project can be considered to fall within the realm of Information Systems (IS) research for at least the following reasons: (1) benchmarking research is an aspect of well-established IS/ICT & Productivity research stream (e.g. Hitt and Brynjolfsson, 1996, Ko and Osei‐Bryson, 2004); (2) benchmarking research has appeared in leading IS journals (e.g. Ayabakan et al., 2017); (3) the proposed solution artifact involves the a creative integration of several IS artifacts (i.e. multiple data mining methods) with DEA; and (4) our illustrative example falls within the well-established IS/ICT & Productivity research stream.
The presentation of the project starts with our analysis of the concept of benchmarking, followed by the presentation of the research problem and associated research questions of the study. Then we offer to our reader an overview on supporting data analytic and data mining methods, with the methodology of the study presented next. The illustrative example demonstrates the application of the methodology and is followed by conclusion of the paper.
Section snippets
A conceptualization of the benchmarking problem
The term “benchmarking” is a commonly used one, and popular terms tend to be vulnerable to falling prey to the unfortunate assumption of the universality of their meaning. The concept of benchmarking is important to our inquiry; thus, we feel it is warranted if we spend a few sentences making sure that we clarify the chosen meaning of the term to our readers. Fundamentally and historically, benchmarking means accurate application of a measure, whatever the measure of interest could be (
Research problem and research questions of the study
Fundamentally, any business process can be seen as a process of conversion of means into ends, where the primary goal is to minimize the cost of means and maximize the value of ends via increasing efficiency and effectiveness of mechanism of transformation. If we consider a concept of a business process from a structural perspective, we can identify three distinctive parts. First, there is a set of inputs, second, there is a set of outputs, and third, there is a mechanism of transformation of
Overviews on supporting methods
Various DEA-based methods have been proposed for benchmarking including Cook, Seiford, and Zhu (2004) and Dai and Kuosmanen (2014). Both approaches to benchmarking have some similarity to that presented in this paper. Cook, Tone, and Zhu (2014) method addressed the situation “where multiple performance measures are needed to examine the performance and productivity changes”. The method of Dai and Kuosmanen (2014) involved combining DEA with clustering such that the resulting “cluster-specific
The proposed methodology
We now present a description of the phases of the methodology followed by its justification.
Description of the methodologyPhase Empty Cell Description 1 Define the Transformation Framework Such a framework would involve the specification of the relevant Driver and Impact constructs and their indicator variables. It would form the basis for identifying the potential causal paths that are to be evaluated. An existing established transformation framework (e.g. Networking Readiness Index) could be utilized, or a
Justification & benefits of the methodology
The first phase of our methodology is associated with defining, or adapting, a context-specific transformation framework according to which inputs are converted into outputs. For all intents and purposes the goal of the transformation framework is to create a set of established pathways by which the transformation of inputs into outputs takes place. We refer to the transformation framework as the Universe of Discourse of Transformation (UoDoT) - an established common set of means by which
Illustrative example – application to Sub-Saharan economies
The example we present to our readers involves the application of the methodology to a dataset that covers a subset of the countries of Sub-Saharan Africa (SSA). We obtained the data from a publicly available source- the World Economic Forum’ Global Information Technology Report (GITR, 2015). In 2012 the representation of the Networked Readiness Index (NRI) was partially changed so we decided to concentrate on the new version of NRI and use the data for the 2012 – 2015 period. However, for some
Conclusion
In this paper we have presented a new multi-method methodology for benchmarking that uncovers relevant non-obvious context-specific causal structures. We then provided an illustrative application of this methodology to an IS/ICT & Productivity research problem in the ‘developing’ countries context that resulted in the uncovering of non-obvious causal structures that describe relationships between Drivers and Impact of ICT. These results show that the relationships between the factors
References (67)
- et al.
Benchmarking airports from a managerial perspective
Omega
(2013) - et al.
How to better identify the true managerial performance: State of the art using DEA
Omega
(2008) - et al.
Governance and efficiency in the Brazilian water utilities: A dynamic analysis in the process of universal access
Utilities Policy
(2016) Probability models in partitional cluster analysis
Computational Statistics and Data Analysis
(1996)- et al.
An integrated neural network and data envelopment analysis for supplier evaluation under incomplete information
Expert Systems with Applications
(2008) - et al.
Using Malmquist indexes to measure changes in the productivity and efficiency of US accounting firms before and after the Sarbanes–Oxley Act
Omega
(2009) - et al.
Models for performance benchmarking: Measuring the effect of e-business activities on banking performance
Omega
(2004) - et al.
Data envelopment analysis: Prior to choosing a model
Omega
(2014) - et al.
Best-practice benchmarking using clustering methods: Application to energy regulation
Omega
(2014) - et al.
Pitfalls and protocols in DEA
European Journal of Operational Research
(2001)
Benchmarking of maintenance and outage repair in an electricity distribution company using the value-based DEA method
Omega
The use of data envelopment analysis for technology selection
Computers and Industrial Engineering
A multi-criteria approach to selecting an optimal portfolio of refinery upgrade projects under margin and tax regime uncertainty
Omega
Evaluation of bank branch growth potential using data envelopment analysis
Omega
DEA and ranking with the network-based approach: A case of R&D performance
Omega
A survey of DEA applications
Omega
Towards supporting expert evaluation of clustering results using a data mining process model
Information Sciences
A hybrid clustering algorithm
Computers & Operations Research
Incorporating cost and environmental factors in quality function deployment using data envelopment analysis
Omega
An exploration of the effects of the interaction between ict and labor force on economic growth in transitional economies
International Journal of Production Economics
Determining sources of relative inefficiency in heterogeneous samples: Methodology using cluster analysis, DEA and neural networks
European Journal of Operational Research
Using data envelopment analysis (DEA) for monitoring efficiency-based performance of productivity-driven organizations: Design and implementation of a decision support system
Omega
Understanding software project risk: A cluster analysis
Information and Management
Supplier selection: A hybrid model using DEA, decision tree and neural network
Expert Systems with Applications
Efficiency and effectiveness in railway performance using a multi-activity network DEA model
Omega
Fast algorithms for mining association rules in large databases
Business process reengineering: A survey of international experience
Business Process Management Journal
Perspectives on organizational change: Systems and complexity theories
The Innovation Journal: The Public Sector Innovation Journal
A data envelopment analysis approach to estimate IT-enabled production capability
MIS Quarterly
Evaluating contextual variables affecting productivity using data envelopment analysis
Operations Research
Pattern recognition with fuzzy objective function algorithms
A critique of BPR from a holistic perspective
Business Process Management Journal
Data envelopment analysis: Theory, methodology and applications
Cited by (8)
Integrated data envelopment analysis, multi-criteria decision making, and cluster analysis methods: Trends and perspectives
2023, Decision Analytics JournalTwo-stage additive network DEA: Duality, frontier projection and divisional efficiency
2020, Expert Systems with ApplicationsCitation Excerpt :Data envelopment analysis (DEA) introduced by Charnes, Cooper, and Rhodes (1978), which is used to identify best practices or efficient decision-making units (DMUs) when multiple inputs and multiple outputs are present, has been widely applied to various practical problems along with new model structures that extend in many directions (see, e.g., Cook & Seiford, 2009; Liu, Lu, Lu, & Lin, 2013). DEA not only provides efficiency scores but also efficient projections for inefficient DMUs onto the best-practice frontier (Samoilenko & Osei-Bryson, 2019). In recent years, a significant research has been done on DMUs with multistage or network structures.
Representation matters: An exploration of the socio-economic impacts of ICT-enabled public value in the context of sub-Saharan economies
2019, International Journal of Information ManagementCitation Excerpt :The NRI framework is reflected by a single composite index comprised of four sub-indexes- three drivers (Environment, Readiness, and Usage subindexes) and one Impact subindex (Table 1). Since its development in 2002 the NRI framework has been highly regarded as an authoritative source of data and an assessment tool geared towards global leaders, practitioners, policy makers (Kirkman, Osorio, & Sachs, 2002), which was also available to academic researchers (Chanyagorn & Kungwannarongkun, 2011; Ifinedo, 2005; Indjikian & Siegel, 2005; Milenkovic, Brajovic, Milenkovic, Vukmirovic, & Jeremic, 2016; Samoilenko & Osei-Bryson, 2017b; Vehovar, Sicherl, Hüsing, & Dolnicar, 2006; Wielicki & Arendt, 2010, 2019). One of the underlying principles of the framework is that “NRI should provide clear policy guidance” (p.xi) to its users, which is not an easy undertaking to accomplish because within the framework “…the complex relationships between ICTs and socio-economic performance are not fully understood and their causality not fully established” (Di Battista, Dutta, Geiger, & Lanvin, 2015, p.4).
Using Big Data to enhance data envelopment analysis of retail store productivity
2023, International Journal of Productivity and Performance ManagementAn approach to modelling complex ICT4D investment problems: towards a solution-oriented framework and data analytics methodology
2023, European Journal of Information SystemsApplication of DEA in benchmarking: A systematic literature review from 2003–2020
2021, Technological and Economic Development of Economy