Profile distance method—a multi-attribute decision making approach for information system investments
Introduction
Information systems (IS) play a major role in developing and sustaining competitive advantage in the global marketplace [16], [31]. IS can transform business and revolutionize the way business is conducted [6].
Literature reports extensively on diverse problems associated with IS evaluation [26]. Those problems can be derived from the difficulty of understanding the complex factors involved in IS decision making, such as scope and impact of the decision, the concept of value and its multi-dimensional facets, natures of IS benefits and costs, associated risks, strategy alignment, human and organizational mechanics or political issues.
The evaluation of IS investment proposals has been a recognized research area for a long time. Already in the 1960s, researchers began working on IS related evaluation issues [20], [28]. Since then, IS evaluation has become one of the most researched and written about topics in IS research, resulting in a large number of evaluation techniques available today. Research exists helping to assess the wide spectrum of methodical aids through taxonomies [25], classifications [17], [18], and surveys of methods [39]. Olson shows how major decision aids supporting selection problems work and concludes with a comparison of techniques with respect to practical implementation [32]. Despite these efforts, organizational studies have shown that in practice business management fails to appreciate the portfolio of investment appraisal techniques available. Many companies are justifying their IS expenditure on the basis of what could be called ‘acts of faith’ [5]. Where formal evaluation takes place, it is predominantly based on conventional accountancy approaches such as methods implementing discounted cash flow analysis [8]. The accountancy methods can be applied to any corporate investment proposal and are widely understood by senior managers, but do not accurately capture the benefits resulting from investments in IS [30]. More complex or IS-specific evaluation methods to capture the full consequences of an IS investment are seldom adopted by business managers. This situation further worsens in small to medium sized compared to large enterprises [8]. If a formal, non-financial technique for IS evaluation is chosen to support the IS investment appraisal, it is very likely a simple scoring and ranking technique [35]. These methods appeal to management due to their intuitive, simple and cost-effective application. They are relatively transparent, allowing others to see the logic of the results and enabling the inclusion of the full range of intangible consequences. These criteria seem to be the key to application success of any methodical aid.
In this paper we propose a new, conceptual approach, named profile distance method, to support IS selection problems. By combining the basic concept of the popular utility scoring and ranking technique with data envelopment analysis (DEA), we recognize their appealing benefits while making up for a number of their limitations. The results can be visualized to support the decision maker in justifying and communicating the model outcomes. Because the method consists of solving a linear program (LP), it is easy to implement on any personal computer. Its underlying concepts are logically sound and comprehensible. We illustrate this with a real-world case on a prototype implementation, which refers to an enterprise resource planning (ERP) software selection.
The remainder of this article is structured as follows. First, we provide more information on the methodical background for the incorporated concepts. Next, we summarize the research issues. In Section 4, we develop the new approach and subsequently show its application. Finally, Section 6 concludes the results and sketches the on-going research.
Section snippets
Utility ranking method
Four basic approaches can be identified in the evaluation methods available: the financial, the ratio, the portfolio, and the multi-criteria approaches [34], [36]. The method described in this article follows a multi-criteria, more precisely “multiple attribute decision making” (MADM) approach, which in general is applicable to a wide range of human choices [42]. MADM refers to making preference decisions over a finite number of alternatives that are characterized by multiple, usually
Research issues
This paper develops a method combining the merits of the two prominent concepts presented in the previous sections, URM and DEA, by utilizing their compatibility in terms of MADM. Thereby, we seek to advance the state of the art by addressing the following limitations in IS decision making:
- (1)
The difficulty to assess, understand and compare the full spectrum of decision making attributes as a whole, i.e., the profiles of the alternatives under evaluation.
- (2)
The problem of identifying the selection
Developing the new model
The original fractional model optimizes the weighted output per weighted input, where the weights are the variables. We start with the original CCR-DEA model as described in, e.g., [2], which translates the fractional program into an LP by adding constraint (3) keeping the denominator equal to 1. Here we have n DMUs each with m input attributes represented through the m × n matrix X and s output attributes stored in the s × n matrix Y. The vectors v and u are the weight vectors for input and
Case application
The developed model was tested within a real-life case study concerning an ERP software selection problem faced by the Austrian subsidiary called Primagaz Austria of an international wholesaler of liquid and gaseous fuels and related products (SHV Holdings N.V.). Before presenting the results, we provide short information on the company and ERP project background as well as on the selection methodology applied by the company. For a more detailed description about the company and the undertaken
Conclusions and further research
Although the analyzed company based their ERP decision making on a structured and methodical approach starting with a strategic assessment of their needs, their methodology showed limitations in the context of the given research issues. The case therefore provided us with an ideal setting to demonstrate the newly introduced profile distance method. The method utilizes the important concept of organizational fit, i.e., by exploring the distance to the desired product profile. It improves ranking
Acknowledgements
The authors thank Ulrike Andres (CEO of Primagaz) for offering us the opportunity to investigate their decision case, Andrew B. Whinston (Editor-in-Chief) and his team for their editorial services, as well as three anonymous reviewers for their valuable comments and advice.
Edward W.N. Bernroider received an MS degree in applied informatics from the University of Salzburg (1997) and a PhD degree in business administration from the Vienna University of Economics and Business Administration (2001), where he is employed as Assistant Professor since 1998. His current research and industry projects focus on evaluation/selection methodologies in IT/IS decisions, IT/IS controlling, and the international software industry.
References (43)
- et al.
The efficiency of Australian universities: A data envelopment analysis
Economics of Education Review
(2003) - et al.
Review of ranking methods in the data envelopment analysis context
European Journal of Operations Research
(2002) - et al.
Improving envelopment in data envelopment analysis
European Journal of Operational Research
(2004) - et al.
The paradox of information systems: Strategic value and low status
The Journal of Strategic Information Systems
(1999) - et al.
A closer look at the use of data envelopment analysis for technology selection
Computers and Industrial Engineering
(1997) Evaluating bank branch location and performance—a case-study
European Journal of Operational Research
(1995)- et al.
Simulation techniques for the sensitivity analysis of multicriteria decision models
European Journal of Operational Research
(1997) - et al.
Measuring the efficiency of decision making units
European Journal of Operational Research
(1978) - et al.
The great IT benefit hunt
European Management Journal
(1994) - et al.
A taxonomy for data envelopment analysis
Socio Economic Planning Sciences
(2004)
The critical success factors for eRP implementation: An organizational fit perspective
Information and Management
Information systems evaluation: Navigating through the problem domain
Information and Management
A decision support system for multiattribute utility evaluation based on imprecise assignments
Decision Support Systems
The use of data envelopment analysis for technology selection
Computers and Industrial Engineering
Methodologies for information systems investment evaluation at the proposal stage: A comparative review
Information and Software Technology
A comparative analysis of DEA as a discrete alternative multiple criteria decision tool
European Journal of Operational Research
Multi-attribute decision making: A simulation comparison of select methods
European Journal of Operational Research
A procedure for ranking efficient units in data envelopment analysis
Management Science
ERP selection process in midsize and large organizations
Business Process Management Journal
The evaluation of ERP systems using data envelopment analysis
Cited by (67)
From dynamic capabilities to ERP enabled business improvements: The mediating effect of the implementation project
2014, International Journal of Project ManagementQuality evaluation of floss projects: Application to ERP systems
2013, Information and Software TechnologyCitation Excerpt :This is a technique for supporting multiple-criteria decision problems that suggests how to prioritise a set of alternatives and importance of the relative attributes. Bernroider and Stix [7] defined an approach that combines the merits of two prominent concepts individually applied in decision-making: the Utility Ranking Method (URM) and the Data Envelopment Analysis (DEA). In addition, the method calculates the distances between the desired system profile, defined in the additive multi-attribute utility model, and the individual alternative profiles, calculated by the DEA-derived optimisation process.
Group decision making based on induced uncertain linguistic OWA operators
2013, Decision Support SystemsCitation Excerpt :Group decision making based on linguistic preference relations, has received a great deal of attention from researchers recently [6,19,21,22,30,32]. In the group decision making process, aggregating each decision maker's decision information is the key [1,5]. Since the ordered weighted averaging (OWA) operator was first generated by Yager [25] in 1988, many aggregation operators such as the ordered weighted geometric averaging (OWGA) operator [2,23], the linguistic weighted ordered weighted averaging (LOWA) operator [6,8] and linguistic ordered weighted geometric averaging (LOWGA) operator [19] have been developed.
A use of DEA-DA to measure importance of R&D expenditure in Japanese information technology industry
2013, Decision Support SystemsA technological, organisational, and environmental analysis of decision making methodologies and satisfaction in the context of IT induced business transformations
2013, European Journal of Operational Research
Edward W.N. Bernroider received an MS degree in applied informatics from the University of Salzburg (1997) and a PhD degree in business administration from the Vienna University of Economics and Business Administration (2001), where he is employed as Assistant Professor since 1998. His current research and industry projects focus on evaluation/selection methodologies in IT/IS decisions, IT/IS controlling, and the international software industry.
Volker Stix studied computer science at the Technical University of Vienna. His PhD thesis covered optimizational and computational aspects in quadratic non-convex global programming. Since 2001, he is assistant professor at the Vienna University of Economics and Business Administration. His research interests include decision support systems in information technologies especially with focus on adaptation and usability of theoretical models for decision makers.