Skip to main content
SpringerLink
Log in

Towards an integrated measurement of IS project performance: The project performance scorecard

  • Published:
Information Systems Frontiers Aims and scope Submit manuscript

Abstract

Determining the contribution of information system (IS) projects is a difficult endeavour. This research presents a framework for evaluating and measuring IS project performance that seek to address this problem while moving away from the reliance on the traditional assessment method (time, cost, specifications). When IS projects fail or succeed is difficult to determine what conditions enabled the outcome. However, if we are to develop a higher level of IS project management competence we need frameworks that can help in assessing IS project outcomes while critically reflecting and learning from IS project management practices. The Project Performance Scorecard (PPS) recommended here is one framework that can help to advance our understanding IS project management and evaluation approaches. This framework integrates concepts from existing models of IS success, the Balanced Scorecard and project management practices. A case study approach is used to illustrate the use of the PPS, and offers insights into how it can be useful in assessing the performance of IS projects.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  • Agarwal, N., & Rathod, U. (2006). Defining ‘success’ for software projects: An exploratory revelation. International Journal of Project Management, 24(4), 358–370.

    Article  Google Scholar 

  • Alleman, G. B. (2005). Agile project management for IT projects. In Y. H. K. Elias, G. Carayannis, & F. T. Anbari (Eds.), The story of managing projects: An interdisciplinary approach (p. 373). Westport: Greenwood Press.

    Google Scholar 

  • Atkinson, R. (1999). Project management: cost, time and quality, two best guesses and a phenomenon, its time to accept other success criteria. International Journal of Project Management, 17(6), 337–342.

    Article  Google Scholar 

  • Baccarini, D. (1999). The logical framework method for defining project success. Project Management Journal, 30(4), 25–32.

    Google Scholar 

  • Bandi, R. K., Vaishnavi, V. K., & Turk, D. E. (2003). Predicting maintenance performance using object-oriented design complexity metrics. IEEE Transactions on Software Engineering, 29(1), 77–87.

    Article  Google Scholar 

  • Barclay, C. (2007). Towards an integrated framework for project performance IRMA 2007. Vancouver: IGI Publishing.

    Google Scholar 

  • Basili, V. R., Caldiera, G., & Rombach, H. D. (1994). The goal question metric approach. Encyclopedia of software engineering. Hoboken: Wiley.

    Google Scholar 

  • Bourne, M., Franco, M., & Wilkes, J. (2003). Corporate performance measurement. Measuring Business Excellence, 7, 15–21.

    Article  Google Scholar 

  • Brancheau, J. C., Janz, B. D., et al. (1996). Key issues in information systems management: 1994–95 SIM Delphi results. MIS Quarterly, 20(2), 225–242.

    Article  Google Scholar 

  • Brancheau, J. C., & Wetherbe, J. C. (1987). Key issues in information systems management. MIS Quarterly, 11(1), 23–44.

    Article  Google Scholar 

  • Brandon, D. (2006). Project management for modern information systems. Hershey: IRM.

    Google Scholar 

  • Bryde, D. J. (2003). Modelling project management performance. International Journal of Quality & Reliability Management, 20(2), 229–254.

    Article  Google Scholar 

  • Bryde, D. J. (2005). Methods for managing different perspectives of project success. British Journal of Management, 16(2), 119–131.

    Article  Google Scholar 

  • Brynjolfsson, E. (1993). The productivity paradox of information technology. Communications of the ACM, 36(12), 67–77.

    Article  Google Scholar 

  • Cohen, D. J., & Graham, R. J. (2001). The project manager’s MBA: How to translate project decisions into business success. Hoboken: Wiley.

    Google Scholar 

  • Collins, A., & Baccarini, D. (2004). Project success - A survey. Journal of Construction Research, 5(2), 211–231.

    Article  Google Scholar 

  • Cooke-Davies, T. (2002). The real success factors on projects. International Journal of Project Management, 20(3), 185–190.

    Google Scholar 

  • DeLone, W. H., & McLean, E. R. (1992). Information systems success: The quest for the dependent variable. Information Systems Research, 3(1), 60, 36p.

    Google Scholar 

  • DeLone, W. H., & McLean, E. R. (2003). The DeLone and McLean model of information systems success: ten year update. Journal of Management Information Systems, 19(4), 9–30.

    Google Scholar 

  • deWit, A. (1988). Measurement of project management success. International Journal of Project Management, 6(3), 164–170.

    Article  Google Scholar 

  • Duggan, E. W., & Reichgelt, H. (2006). Measuring information systems delivery quality. Hershey: Idea Group Publishing.

    Google Scholar 

  • Dvir, D., Lipovetsky, S., Shenhar, A. J., & Tishler, A. (2003). What is really important for project success? A refined, multivariate, comprehensive analysis. International Journal of Management and Decision Making, 4(4), 382–404.

    Article  Google Scholar 

  • Ewusi-Mensah, K., & Przasnyski, Z. H. (1991). On information systems project abandonment: An exploratory study of organisational practices. MIS Quarterly, 15(1), 67–89.

    Article  Google Scholar 

  • Fitzgerald, G. (1998). Evaluating information systems project: A multidimensional approach. Journal of Information Technology, 13(1), 15–27.

    Article  Google Scholar 

  • Freeman, M., & Beale, P. (1992). Measuring project success. Project Management Journal, 23(1), 8–17.

    Google Scholar 

  • Garvin, D. A. (1987). Competing on the eight dimensions of quality. Harvard Business Review, pp 101–109.

  • Glass, R. L. (1999). Evolving a new theory of project success. Communications of the ACM, 42(11), 17–19.

    Article  Google Scholar 

  • Glass, R. L. (2005). IT failure rates—70% or 10–15%. IEEE Software, 22(3), 112, 110–111.

    Article  Google Scholar 

  • Glass, R. L. (2006). The Standish report: Does it really describe a software crisis. Communications of the ACM, 49(8), 15–16.

    Article  Google Scholar 

  • ISO/IEC (2001). ISO/IEC standard 9126-1—Software Engineering—Product Quality—Part 1: quality model.

  • Jugdev, K., & Muller, R. (2005). A retrospective look at our evolving understanding of project success. Project Management Journal, 36(4), 19–31.

    Google Scholar 

  • Kaplan, R. S., & Norton, D. P. (1992). The balanced scorecard: measures that drive performance. Harvard Business Review, 70(1), 71–79.

    Google Scholar 

  • Kaplan, R. S., & Norton, D. P. (1996a). Balanced scorecard: Translating strategy into action. Boston: Harvard Business School Press.

    Google Scholar 

  • Kaplan, R. S., & Norton, D. P. (1996b). Linking the balance scorecard to strategy. California Management Review, 39(1), 53–79.

    Google Scholar 

  • Kleist, V. F. (2003). An approach to evaluating E-business information systems projects. Information Systems Frontiers, 5(3), 249–263.

    Article  Google Scholar 

  • KPMG. (2005). Global IT project management survey, KPMG information risk management, (pp 1–35).

  • Kumar, R. L. (2003). Understanding the value of information technology enabled responsiveness. EJISE, 1(1).

  • Lee, A. S. (1989). A scientific methodology for MIS case studies. MIS Quarterly, 13(1), 32.

    Article  Google Scholar 

  • Leonard-Barton, D. (1990). A dual methodology for case studies. synergistic use of longitudinal single site with replicated multiple sites. Organisation Science, 1(1), 248–266.

    Google Scholar 

  • Lim, C. S., & Mohamed, M. Z. (1999). Criteria of project success: An exploratory re-examination. International Journal of Project Management, 17(4), 243–248.

    Article  Google Scholar 

  • Lin, C., & Pervan, G. (2001). Review of IS/IT investment evaluation and benefits management issues, problems, and processes. New York: Wiley.

    Google Scholar 

  • Lu, J. (2003). A model for evaluating e-commerce based on cost/benefit and customer satisfaction. Information Systems Frontiers, 5(3), 265–277.

    Article  Google Scholar 

  • Lyytinen, K. (1988). Expectation failure concept and systems analysts’ view of information system failures: results of an exploratory study. Information & Management, 14(1), 45–56.

    Article  Google Scholar 

  • Markus, M. L., & Keil, M. (1994). If we build it they will come: Designing information systems that users want to use. Sloan Management Review, 35(4), 11–25.

    Google Scholar 

  • Martinsons, M., Davison, R., & Tse, D. (1999). The balanced scorecard: A foundation for the strategic management of information systems. Decision Support Systems, 25, 71–88.

    Article  Google Scholar 

  • Morris, P. W. G., & Hough, G. H. (1987). The anatomy of major projects: A study of the reality of project management. Hoboken: Wiley.

    Google Scholar 

  • Nelson, R. R. (2005). Project retrospectives: Evaluating project success, failure and everything in between. MIS Quarterly Executive, 4(3), 361–372.

    Google Scholar 

  • Okoli, C., & Pawlowski, S. D. (2004). The Delphi method as a research tool: an example, design considerations and applications. Information & Management, 42, 15–29.

    Google Scholar 

  • Palvia, S. C., Sharma, R. S., & Conrath, D. W. (2001). A socio-technical framework for quality assessment of computer information systems. Industrial Management & Data Systems, 101(5), 237–251.

    Article  Google Scholar 

  • Pinto, J. K., & Slevin, D. P. (1987). Critical success factors in successful project implementation. IEEE Transactions Engineering Management, 34(1), 22–27.

    Google Scholar 

  • Rai, A., Lang, S. S., & Welker, R. B. (2002). Assessing the validity of IS success models: An empirical testand theoretical analysis. Information Systems Research, 13(1), 50–69.

    Article  Google Scholar 

  • Ravichandran, R., & Rai, A. (1994). The dimensions and correlates of system development quality. In Annual SIG Computer Personnel Research Conference on Reinventing IS, Virginia.

  • Remenyi, D. (2004). The elusive nature of delivering benefits from IT investment. EJISE, 3(1).

  • Robey, D., & Markus, M. L. (1984). Rituals in information system design. MIS Quarterly, 8(1), 5.

    Article  Google Scholar 

  • Saaty, T. L. (1987). Risk—Its priority and probability: The analytic hierarchy process. Risk Analysis, 7(2), 159–172.

    Article  Google Scholar 

  • Saaty, T. L. (1990). How to make a decision: The analytic hierarchy process. European Journal of Operational Research, 48(1), 9–26.

    Article  Google Scholar 

  • Seddon, P. B. (1997). A respecification and extension of the Delone and McLean model of IS success. Information Systems Research, 8(3), 240–253.

    Article  Google Scholar 

  • Seddon, P. B., Staples, S., Patnayakuni, R., & Bowtell, M. (1999). Dimensions of information systems success. CAIS, 2(20), 5.

    Google Scholar 

  • Sheng, H., Nah, F. F.-H., & Siau, K. (2005). Strategic implications of mobile technology: A case study using Value-Focused Thinking. Journal of Strategic Information Systems, 14, 269–290.

    Article  Google Scholar 

  • Shenhar, A. J., Dvir, D., et al. (2001). Project success: A multi-dimensional strategic perspective. Long Range Planning, 34, 699–725.

    Article  Google Scholar 

  • Shenhar, A. J., & Levy, O. (1997). Mapping the dimensions of project success. Project Management Journal, 28(2), 5–13.

    Google Scholar 

  • Shenhar, A. J., Tisler, A., Dvir, D., Lipostevsky, S., & Lechler, T. (2002). Refining the search for project success factors: A multivariate typographical approach. R & D Management, 32(2), 111–126.

    Article  Google Scholar 

  • Standish (1994). The chaos report. Boston: The Standish Group International.

    Google Scholar 

  • Standish (2004). 2004 third quarter research report. Boston: The Standish Group International.

    Google Scholar 

  • Teltumbde, A. (2000). A framework for evaluating ERP projects. International Journal of Production Research, 38(17), 4507–4520.

    Article  Google Scholar 

  • Wateridge, J. (1998). How can IS/IT projects be measured for success. International Journal of Project Management, 16(1), 59–63.

    Article  Google Scholar 

  • White, D., & Fortune, J. (2002). Current practice in project management—An empirical study. International Journal of Project Management, 20, 1–11.

    Article  Google Scholar 

  • Winter, M., Smith, C., Morris, P., & Cicmil, S. (2006). Directions for future research in project management: The main findings of a UK government-funded research network. International Journal of Project Management, 24, 638–649.

    Article  Google Scholar 

  • Yin, R. K. (1994). Case study research: Design and methods. Beverly Hills: Sage Publishing.

    Google Scholar 

  • Yin, R. K. (2003). Case study research: Design and methods. Beverly Hills: Sage Publishing.

    Google Scholar 

  • Zayed, T., Amer, M., and Pan, J. (2007). Assessing risk and uncertainty inherent in Chinese highway projects using AHP. International Journal of Project Management. (in press) DOI 10:1016/j.ijproman.2007.05.012.

Download references

Author information

Authors and Affiliations

  1. University of the West Indies, Mona Campus, Kingston, WI, Jamaica

    Corlane Barclay

Authors
  1. Corlane Barclay
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Corlane Barclay.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Barclay, C. Towards an integrated measurement of IS project performance: The project performance scorecard. Inf Syst Front 10, 331–345 (2008). https://doi.org/10.1007/s10796-008-9083-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10796-008-9083-6

Keywords

Search

Navigation