Elsevier

Decision Support Systems

Volume 42, Issue 2, November 2006, Pages 988-998
Decision Support Systems

Profile distance method—a multi-attribute decision making approach for information system investments

https://doi.org/10.1016/j.dss.2005.02.006Get rights and content

Abstract

This article addresses the area of decision making for information systems (IS). We recognize the great demand for methods and techniques that can be of practical help by presenting a new, conceptual approach, the profile distance method, to support the IS selection problem. This approach combines the merits of two prominent concepts individually applied in decision making: the utility ranking method (URM) and the data envelopment analysis (DEA). In addition, the method involves calculating distances between the desired system profile defined in the additive multi-attribute utility model and the individual alternative profiles calculated by the DEA derived optimization process. The results can be visualized to support the decision maker in justifying and communicating the model outcomes. The proposed method is illustrated within a real-life case study concerning an enterprise resource planning (ERP) software selection problem.

Introduction

Information systems (IS) play a major role in developing and sustaining competitive advantage in the global marketplace [16], [31]. IS can transform business and revolutionize the way business is conducted [6].

Literature reports extensively on diverse problems associated with IS evaluation [26]. Those problems can be derived from the difficulty of understanding the complex factors involved in IS decision making, such as scope and impact of the decision, the concept of value and its multi-dimensional facets, natures of IS benefits and costs, associated risks, strategy alignment, human and organizational mechanics or political issues.

The evaluation of IS investment proposals has been a recognized research area for a long time. Already in the 1960s, researchers began working on IS related evaluation issues [20], [28]. Since then, IS evaluation has become one of the most researched and written about topics in IS research, resulting in a large number of evaluation techniques available today. Research exists helping to assess the wide spectrum of methodical aids through taxonomies [25], classifications [17], [18], and surveys of methods [39]. Olson shows how major decision aids supporting selection problems work and concludes with a comparison of techniques with respect to practical implementation [32]. Despite these efforts, organizational studies have shown that in practice business management fails to appreciate the portfolio of investment appraisal techniques available. Many companies are justifying their IS expenditure on the basis of what could be called ‘acts of faith’ [5]. Where formal evaluation takes place, it is predominantly based on conventional accountancy approaches such as methods implementing discounted cash flow analysis [8]. The accountancy methods can be applied to any corporate investment proposal and are widely understood by senior managers, but do not accurately capture the benefits resulting from investments in IS [30]. More complex or IS-specific evaluation methods to capture the full consequences of an IS investment are seldom adopted by business managers. This situation further worsens in small to medium sized compared to large enterprises [8]. If a formal, non-financial technique for IS evaluation is chosen to support the IS investment appraisal, it is very likely a simple scoring and ranking technique [35]. These methods appeal to management due to their intuitive, simple and cost-effective application. They are relatively transparent, allowing others to see the logic of the results and enabling the inclusion of the full range of intangible consequences. These criteria seem to be the key to application success of any methodical aid.

In this paper we propose a new, conceptual approach, named profile distance method, to support IS selection problems. By combining the basic concept of the popular utility scoring and ranking technique with data envelopment analysis (DEA), we recognize their appealing benefits while making up for a number of their limitations. The results can be visualized to support the decision maker in justifying and communicating the model outcomes. Because the method consists of solving a linear program (LP), it is easy to implement on any personal computer. Its underlying concepts are logically sound and comprehensible. We illustrate this with a real-world case on a prototype implementation, which refers to an enterprise resource planning (ERP) software selection.

The remainder of this article is structured as follows. First, we provide more information on the methodical background for the incorporated concepts. Next, we summarize the research issues. In Section 4, we develop the new approach and subsequently show its application. Finally, Section 6 concludes the results and sketches the on-going research.

Section snippets

Utility ranking method

Four basic approaches can be identified in the evaluation methods available: the financial, the ratio, the portfolio, and the multi-criteria approaches [34], [36]. The method described in this article follows a multi-criteria, more precisely “multiple attribute decision making” (MADM) approach, which in general is applicable to a wide range of human choices [42]. MADM refers to making preference decisions over a finite number of alternatives that are characterized by multiple, usually

Research issues

This paper develops a method combining the merits of the two prominent concepts presented in the previous sections, URM and DEA, by utilizing their compatibility in terms of MADM. Thereby, we seek to advance the state of the art by addressing the following limitations in IS decision making:

  • (1)

    The difficulty to assess, understand and compare the full spectrum of decision making attributes as a whole, i.e., the profiles of the alternatives under evaluation.

  • (2)

    The problem of identifying the selection

Developing the new model

The original fractional model optimizes the weighted output per weighted input, where the weights are the variables. We start with the original CCR-DEA model as described in, e.g., [2], which translates the fractional program into an LP by adding constraint (3) keeping the denominator equal to 1. Here we have n DMUs each with m input attributes represented through the m × n matrix X and s output attributes stored in the s × n matrix Y. The vectors v and u are the weight vectors for input and

Case application

The developed model was tested within a real-life case study concerning an ERP software selection problem faced by the Austrian subsidiary called Primagaz Austria of an international wholesaler of liquid and gaseous fuels and related products (SHV Holdings N.V.). Before presenting the results, we provide short information on the company and ERP project background as well as on the selection methodology applied by the company. For a more detailed description about the company and the undertaken

Conclusions and further research

Although the analyzed company based their ERP decision making on a structured and methodical approach starting with a strategic assessment of their needs, their methodology showed limitations in the context of the given research issues. The case therefore provided us with an ideal setting to demonstrate the newly introduced profile distance method. The method utilizes the important concept of organizational fit, i.e., by exploring the distance to the desired product profile. It improves ranking

Acknowledgements

The authors thank Ulrike Andres (CEO of Primagaz) for offering us the opportunity to investigate their decision case, Andrew B. Whinston (Editor-in-Chief) and his team for their editorial services, as well as three anonymous reviewers for their valuable comments and advice.

Edward W.N. Bernroider received an MS degree in applied informatics from the University of Salzburg (1997) and a PhD degree in business administration from the Vienna University of Economics and Business Administration (2001), where he is employed as Assistant Professor since 1998. His current research and industry projects focus on evaluation/selection methodologies in IT/IS decisions, IT/IS controlling, and the international software industry.

References (43)

  • K.-K. Hong et al.

    The critical success factors for eRP implementation: An organizational fit perspective

    Information and Management

    (2002)
  • Z. Irani

    Information systems evaluation: Navigating through the problem domain

    Information and Management

    (2002)
  • A. Jimenez et al.

    A decision support system for multiattribute utility evaluation based on imprecise assignments

    Decision Support Systems

    (2003)
  • M. Khouja

    The use of data envelopment analysis for technology selection

    Computers and Industrial Engineering

    (1995)
  • T.J.W. Renkema et al.

    Methodologies for information systems investment evaluation at the proposal stage: A comparative review

    Information and Software Technology

    (1997)
  • J. Sarkis

    A comparative analysis of DEA as a discrete alternative multiple criteria decision tool

    European Journal of Operational Research

    (2000)
  • S.H. Zanakis et al.

    Multi-attribute decision making: A simulation comparison of select methods

    European Journal of Operational Research

    (1998)
  • P. Anderson et al.

    A procedure for ranking efficient units in data envelopment analysis

    Management Science

    (1993)
  • J.L. Andresen, A Framework for Selecting an IT Evaluation Method—in the Context of Construction, BYG-DTU, Technical...
  • E.W.N. Bernroider et al.

    ERP selection process in midsize and large organizations

    Business Process Management Journal

    (2001)
  • E.W.N. Bernroider et al.

    The evaluation of ERP systems using data envelopment analysis

  • Cited by (67)

    • Quality evaluation of floss projects: Application to ERP systems

      2013, Information and Software Technology
      Citation Excerpt :

      This is a technique for supporting multiple-criteria decision problems that suggests how to prioritise a set of alternatives and importance of the relative attributes. Bernroider and Stix [7] defined an approach that combines the merits of two prominent concepts individually applied in decision-making: the Utility Ranking Method (URM) and the Data Envelopment Analysis (DEA). In addition, the method calculates the distances between the desired system profile, defined in the additive multi-attribute utility model, and the individual alternative profiles, calculated by the DEA-derived optimisation process.

    • Group decision making based on induced uncertain linguistic OWA operators

      2013, Decision Support Systems
      Citation Excerpt :

      Group decision making based on linguistic preference relations, has received a great deal of attention from researchers recently [6,19,21,22,30,32]. In the group decision making process, aggregating each decision maker's decision information is the key [1,5]. Since the ordered weighted averaging (OWA) operator was first generated by Yager [25] in 1988, many aggregation operators such as the ordered weighted geometric averaging (OWGA) operator [2,23], the linguistic weighted ordered weighted averaging (LOWA) operator [6,8] and linguistic ordered weighted geometric averaging (LOWGA) operator [19] have been developed.

    View all citing articles on Scopus

    Edward W.N. Bernroider received an MS degree in applied informatics from the University of Salzburg (1997) and a PhD degree in business administration from the Vienna University of Economics and Business Administration (2001), where he is employed as Assistant Professor since 1998. His current research and industry projects focus on evaluation/selection methodologies in IT/IS decisions, IT/IS controlling, and the international software industry.

    Volker Stix studied computer science at the Technical University of Vienna. His PhD thesis covered optimizational and computational aspects in quadratic non-convex global programming. Since 2001, he is assistant professor at the Vienna University of Economics and Business Administration. His research interests include decision support systems in information technologies especially with focus on adaptation and usability of theoretical models for decision makers.

    View full text