Skip to main content
Log in

Selecting most efficient information system projects in presence of user subjective opinions: a DEA approach

  • Original Paper
  • Published:
Central European Journal of Operations Research Aims and scope Submit manuscript

Abstract

Information System (IS) project selection is a critical decision making task that can significantly impact operational excellence and competitive advantage of modern enterprises and also can involve them in a long-term commitment. This decision making is complicated due to availability of numerous IS projects, their increasing complexities, importance of timely decisions in a dynamic environment, as well as existence of multiple qualitative and quantitative criteria. This paper proposes a Data Envelopment Analysis approach to find most efficient IS projects while considering subjective opinions and intuitive senses of decision makers. The proposed approach is validated by a real world case study involving 41 IS projects at a large financial institution as well as 18 artificial projects which are defined by the decision makers.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

Notes

  1. Available for free at www.gams.com.

References

  • Adler N, Friedman L, Sinuany-Stern Z (2002) Review of ranking methods in the data envelopment analysis context. Eur J Oper Res 140(2):249–265

    Article  Google Scholar 

  • Ali A, Seiford L (1993) Computational accuracy and infinitesimals in data envelopment analysis. INFOR 31(4):290–297

    Google Scholar 

  • Amin GR (2009) Comments on finding the most efficient DMUs in DEA: an improved integrated model. Comput Ind Eng 56(4):1701–1702

    Article  Google Scholar 

  • Amin GR, Toloo M (2004) A polynomial-time algorithm for finding epsilon in DEA models. Comput Oper Res 31(5):803–805

    Article  Google Scholar 

  • Amin GR, Toloo M (2007) Finding the most efficient DMUs in DEA: an improved integrated model. Comput Ind Eng 56(2):71–77

    Article  Google Scholar 

  • Andersen P, Petersen NC (1993) A procedure for ranking efficient units in data envelopment analysis. Manage Sci 39(10):1261–1264

    Article  Google Scholar 

  • Arabi B, Munisamy S, Emrouznejad A, Toloo M, Ghazizadeh MS (2016) Eco-efficiency considering the issue of heterogeneity among power plants. Energy 111:722–735

    Article  Google Scholar 

  • Asosheh A, Nalchigar S, Jamporazmey M (2010) Information technology project evaluation: an integrated data envelopment analysis and balanced scorecard approach. Expert Syst Appl 37(8):5931–5938

    Article  Google Scholar 

  • Badri MA, Davis D, Davis D (2001) A comprehensive 0–1 goal programming model for project selection. Int J Project Manage 19(4):243–252

    Article  Google Scholar 

  • Bai H, Zhan Z (2011) An IT Project selection method based on fuzzy analytic network process. In: 2011 International conference on system science, engineering design and manufacturing information, vol 2, pp 275–279

  • Baker RC, Talluri S (1997) A closer look at the use of data envelopment analysis for technology selection. Comput Ind Eng 32(1):101–108

    Article  Google Scholar 

  • Banker RD, Charnes A, Cooper WW (1984) Some models for estimating technical and scale inefficiencies in data envelopment analysis. Manage Sci 30(9):1078–1092

    Article  Google Scholar 

  • Bazaraa MS, Jarvis JJ, Sherali HD (2010) Linear programming and network flows, 4th edn. Wiley, New York

    Google Scholar 

  • Charnes A, Cooper WW, Rhodes E (1978) Measuring the efficiency of decision making units. Eur J Oper Res 2:429–444

    Article  Google Scholar 

  • Chen C-T, Cheng H-L (2009) A comprehensive model for selecting information system project under fuzzy environment. Int J Project Manage 27(4):389–399

    Article  Google Scholar 

  • Cook WD, Roll Y, Kazakov A (1990) A DEA model for measuring the relative efficiency of highway maintenance patrols. INFOR 28(2):113–124

    Google Scholar 

  • Cook WD, Kress M, Seiford LM (1996) Data envelopment analysis in the presence of both quantitative and qualitative factors. J Oper Res Soc 47(7):945–953

    Article  Google Scholar 

  • Deng H, Wibowo S (2008) Intelligent decision support for evaluating and selecting information systems projects. Eng Lett 16(3):412–418

    Google Scholar 

  • Doyle J, Green RH (1994) Efficiency and cross-efficiency in DEA: derivations, meanings and uses. J Oper Res Soc 45(5):567–578

    Article  Google Scholar 

  • Edirisinghe NCP, Zhang X (2007) Generalized DEA model of fundamental analysis and its application to portfolio optimization. J Bank Finance 31(11):3311–3335

    Article  Google Scholar 

  • Ertay T, Ruan D, Tuzkaya U (2006) Integrating data envelopment analysis and analytic hierarchy for the facility layout design in manufacturing systems. Inf Sci 176(3):237–262

    Article  Google Scholar 

  • Farzipoor Saen R (2011) Media selection in the presence of flexible factors and imprecise data. J Oper Res Soc 62(9):1695–1703

    Article  Google Scholar 

  • Gao P, Feng J, Yang L (2008) Fuzzy TOPSIS algorithm for multiple criteria decision making with an application in information systems project selection. In: WiCOM’08 4th international conference on wireless communications, networking and mobile computing, 2008, pp 1–4

  • Han CH, Kim JK, Choi SH, Kim SH (1998) Determination of information system development priority using quality function development. Comput Ind Eng 35(1):241–244

    Article  Google Scholar 

  • Holtsnider B, Jaffe BD (2012) IT manager’s handbook: getting your new job done. Morgan Kaufmann, Burlington

    Google Scholar 

  • Hou G (2011) IT/IS project selection: a grey multi-criteria decision model approach. In: 2011 international conference on E-business and E-government (ICEE), pp 1–4

  • Jablonsky J (2018) Ranking of countries in sporting events using two-stage data envelopment analysis models: a case of Summer Olympic Games 2016. Cent Eur J Oper Res. https://doi.org/10.1007/s10100-018-0537-8

    Article  Google Scholar 

  • Kapelko M (2018) Measuring inefficiency for specific inputs using data envelopment analysis: evidence from construction industry in Spain and Portugal. CEJOR 26(1):43–66

    Article  Google Scholar 

  • Karsak EE, Özogul CO (2009) An integrated decision making approach for ERP system selection. Expert Syst Appl 36(1):660–667

    Article  Google Scholar 

  • Kengpol A, Tuominen M (2006) A framework for group decision support systems: an application in the evaluation of information technology for logistics firms. Int J Prod Econ 101(1):159–171

    Article  Google Scholar 

  • Khoshnevis P, Teirlinck P (2018) Performance evaluation of R&D active firms. Socio Econ Plann Sci 61:16–28

    Article  Google Scholar 

  • Lall V, Teyarachakul S (2006) Enterprise Resource Planning (ERP) system selection: a Data Envelopment Anaysis (DEA) approach. J Comput Inf Syst 47(1):123–127

    Google Scholar 

  • Lawrence M, Zhu J (1999) Infeasibility of super-efficiency data envelopment analysis models. Inf Syst Oper Res 37(2):174–188

    Google Scholar 

  • Lee JW, Kim SH (2001) An integrated approach for interdependent information system project selection. Int J Project Manage 19(2):111–118

    Article  Google Scholar 

  • Lee H-S, Chu C-W, Zhu J (2011) Super-efficiency DEA in the presence of infeasibility. Eur J Oper Res 212(1):141–147

    Article  Google Scholar 

  • Masoumzadeh A, Toloo M, Amirteimoori A (2016) Performance assessment in production systems without explicit inputs: an application to basketball players. IMA J Manag Math 27(2):143–156

    Article  Google Scholar 

  • Nakhaeizadeh G, Schnabl A (1997) Development of multi-criteria metrics for evaluation of data mining algorithms. In: Proceedings of the third international conference on knowledge discovery and data mining. AAAI Press

  • Nalchigar S, Nasserzadeh SMR (2009) Application of DEA for selecting most efficient information system project with imprecise data. In: 2009 IEEE international conference on industrial engineering and engineering management. IEEE Publisher, pp 1653–1657

  • Paradi JC, Zhu H (2013) A survey on bank branch efficiency and performance research with data envelopment analysis. Omega 41(1):61–79

    Article  Google Scholar 

  • Ramón N, Ruiz JL, Sirvent I (2012) Common sets of weights as summaries of DEA profiles of weights: with an application to the ranking of professional tennis players. Expert Syst Appl 39(5):4882–4889

    Article  Google Scholar 

  • Roll Y, Cook WD, Golany B (1991) Controlling factor weights in data envelopment analysis. IIE Trans 23(1):2–9

    Article  Google Scholar 

  • Santhanam R, Kyparisis J (1995) A multiple criteria decision model for information system project selection. Comput Oper Res 22(8):807–818

    Article  Google Scholar 

  • Sarkis J, Sundarraj RP (2006) Evaluation of enterprise information technologies: a decision model for high-level consideration of strategic and operational issues. IEEE Trans Syst Man Cybern Part C Appl Rev 36(2):260–273

    Article  Google Scholar 

  • Schniederjans MJ, Santhanam R (1993) A multi-objective constrained resource information system project selection method. Eur J Oper Res 70(2):244–253

    Article  Google Scholar 

  • Schniederjans MJ, Wilson RL (1991) Using the analytic hierarchy process and goal programming for information system project selection. Inf Manag 20(5):333–342

    Article  Google Scholar 

  • Sexton TR, Silkman RH, Hogan AJ (1986) Data envelopment analysis: critique and extensions. N Dir Program Eval 1986(32):73–105

    Article  Google Scholar 

  • Shafer SM, Byrd TA (2000) A framework for measuring the efficiency of organizational investments in information technology using data envelopment analysis. Omega 28(2):125–141

    Article  Google Scholar 

  • Shang J, Sueyoshi T (1995) A unified framework for the selection of a flexible manufacturing system. Eur J Oper Res 85(2):297–315

    Article  Google Scholar 

  • Sohrabi B, Toloo M, Moeini A, Nalchigar S (2015) Evaluation of recommender systems: a multi-criteria decision making approach. Iran J Manag Stud 8(4):589–605

    Google Scholar 

  • Sowlati T, Paradi JC, Suld C (2005) Information systems project prioritization using data envelopment analysis. Math Comput Model 41(11):1279–1298

    Article  Google Scholar 

  • Thompson RG, Singleton FD, Thrall RM, Smith BA (1986) Comparative site evaluations for locating a high-energy physics lab in Texas. Interfaces 16(6):35–49

    Article  Google Scholar 

  • Toloo M (2012a) Alternative solutions for classifying inputs and outputs in data envelopment analysis. Comput Math Appl 63(6):1104–1110

    Article  Google Scholar 

  • Toloo M (2012b) On finding the most BCC-efficient DMU: a new integrated MIP-DEA model. Appl Math Model 36(11):5515–5520

    Article  Google Scholar 

  • Toloo M (2013) The most efficient unit without explicit inputs: an extended MILP-DEA model. Measurement J Int Measurement Confed 46(9):3628–3634

    Article  Google Scholar 

  • Toloo M (2014a) An epsilon-free approach for finding the most efficient unit in. Appl Math Model 38(13):3182–3192

    Article  Google Scholar 

  • Toloo M (2014b) The role of non-Archimedean epsilon in finding the most efficient unit: with an application of professional tennis players. Appl Math Model 38(21–22):5334–5346

    Article  Google Scholar 

  • Toloo M (2015) Alternative minimax model for finding the most efficient unit in data envelopment analysis. Comput Ind Eng 81:186–194

    Article  Google Scholar 

  • Toloo M (2016) A cost efficiency approach for strategic vendor selection problem under certain input prices assumption. Measurement 85:175–183

    Article  Google Scholar 

  • Toloo M, Ertay T (2014) The most cost efficient automotive vendor with price uncertainty: a new DEA approach. Measurement 52(1):135–144

    Article  Google Scholar 

  • Toloo M, Kresta A (2014) Finding the best asset financing alternative: a DEA-WEO approach. Measurement 55:288–294

    Article  Google Scholar 

  • Toloo M, Nalchigar S (2009) A new integrated DEA model for finding most BCC-efficient DMU. Appl Math Model 33(1):597–604

    Article  Google Scholar 

  • Toloo M, Nalchigar S (2011) A new DEA method for supplier selection in presence of both cardinal and ordinal data. Expert Syst Appl 38(12):14726–14731

    Article  Google Scholar 

  • Toloo M, Salahi M (2018) A powerful discriminative approach for selecting the most efficient unit in DEA. Comput Ind Eng 115:269–277

    Article  Google Scholar 

  • Toloo M, Tavana M (2017) A novel method for selecting a single efficient unit in data envelopment analysis without explicit inputs/outputs. Ann Oper Res 253(1):657–681

    Article  Google Scholar 

  • Toloo M, Masoumzadeh A, Barat M (2015) Finding an initial basic feasible solution for DEA models with an application on bank industry. Comput Econ 45(2):323–336

    Article  Google Scholar 

  • Toloo M, Tavana M, Santos-Arteaga FJ (2017) An integrated data envelopment analysis and mixed integer non-linear programming model for linearizing the common set of weights. Cent Eur J Oper Res. https://doi.org/10.1007/s10100-017-0510-y

    Article  Google Scholar 

  • Toloo M, Keshavarz E, Hatami-Marbini A (2018) Dual-role factors for imprecise data envelopment analysis. Omega 77:15–31

    Article  Google Scholar 

  • Yang C-L, Chiang S-J, Huang R-H, Lin Y-A (2013) Hybrid decision model for information project selection. Qual Quant 47(4):2129–2142

    Article  Google Scholar 

  • Yeh C-H, Deng H, Wibowo S, Xu Y (2010) Fuzzy multicriteria decision support for information systems project selection. Int J Fuzzy Syst 12(2):170–174

    Google Scholar 

  • Zhou P, Ang BW, Poh KL (2008) A survey of data envelopment analysis in energy and environmental studies. Eur J Oper Res 189(1):1–18

    Article  Google Scholar 

Download references

Acknowledgements

The research was supported by the European Social Fund (CZ.1.07/2.3.00/20.0296) and the Czech Science Foundation (GAČR 16-17810S). All support is greatly acknowledged.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Mehdi Toloo.

Appendix A

Appendix A

Consider the following input-oriented BCC model:

$$ \begin{array}{*{20}l} {\theta_{o}^{*} = \hbox{max} \mathop \sum \limits_{r = 1}^{s} u_{r} y_{r0} + u_{0} } \hfill \\ {{\text{s}}.{\text{t}}. } \hfill \\ {\begin{array}{*{20}l} {\mathop \sum \limits_{i = 1}^{m} v_{i} x_{io} = 1} \hfill & \hfill \\ {\mathop \sum \limits_{r = 1}^{s} u_{r} y_{rj} + u_{0} - \mathop \sum \limits_{i = 1}^{m} v_{i} x_{ij} \le 0} \hfill & {j = 1, \ldots ,n} \hfill \\ {u_{r} \ge \varepsilon } \hfill & {r = 1, \ldots ,s} \hfill \\ {v_{i} \ge \varepsilon } \hfill & { i = 1, \ldots ,m} \hfill \\ \end{array} } \hfill \\ \end{array} $$
(A.1)

Without loss of generality, suppose ysj = c for j = 1, …, n. Model (A.1) can be rewritten as:

$$ \begin{array}{*{20}l} {\theta_{o}^{*} = \hbox{max} \mathop \sum \limits_{r = 1}^{s - 1} u_{r} y_{r0} + u_{s} c + u_{0} } \hfill \\ {{\text{s}}.{\text{t}}. } \hfill \\ {\begin{array}{*{20}l} {\mathop \sum \limits_{i = 1}^{m} v_{i} x_{io} = 1} \hfill & \hfill \\ {\mathop \sum \limits_{r = 1}^{s - 1} u_{r} y_{rj} + u_{s} c + u_{0} - \mathop \sum \limits_{i = 1}^{m} v_{i} x_{io} \le 0} \hfill & {j = 1, \ldots ,n} \hfill \\ {u_{r} \ge \varepsilon } \hfill & {r = 1, \ldots ,s} \hfill \\ {v_{i} \ge \varepsilon } \hfill & { i = 1, \ldots ,m} \hfill \\ \end{array} } \hfill \\ \end{array} $$
(A.2)

Now, by making the change of variable \( \bar{u}_{0} = u_{s} c + u_{0} \) we obtain the following model which is equivalent to the BCC model (A.1):

$$ \begin{array}{*{20}l} {\theta_{o}^{*} = \hbox{max} \mathop \sum \limits_{r = 1}^{s - 1} u_{r} y_{r0} + \bar{u}_{0} } \hfill \\ {{\text{s}}.{\text{t}}. } \hfill \\ {\begin{array}{*{20}l} {\mathop \sum \limits_{i = 1}^{m} v_{i} x_{io} = 1} \hfill & \hfill \\ {\mathop \sum \limits_{r = 1}^{s - 1} u_{r} y_{rj} + \bar{u}_{0} - \mathop \sum \limits_{i = 1}^{m} v_{i} x_{io} \le 0} \hfill & {j = 1, \ldots ,n} \hfill \\ {u_{r} \ge \varepsilon } \hfill & {r = 1, \ldots ,s - 1} \hfill \\ {v_{i} \ge \varepsilon } \hfill & { i = 1, \ldots ,m} \hfill \\ \end{array} } \hfill \\ \end{array} $$
(A.3)

Hence, the following theorem is proved:

Theorem A.1

A constant output in the input-oriented BCC model is redundant.

Analogously, the following theorem can be proved:

Theorem A.2

A constant input in the output-oriented BCC model is redundant.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Toloo, M., Nalchigar, S. & Sohrabi, B. Selecting most efficient information system projects in presence of user subjective opinions: a DEA approach. Cent Eur J Oper Res 26, 1027–1051 (2018). https://doi.org/10.1007/s10100-018-0549-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10100-018-0549-4

Keywords

Navigation