Emerging evaluation processes in consortium‐based outsourced system development

In large system development efforts organizational and managerial issues are often more challenging than technical ones. One of the key concerns of managers is the control and evaluation of the overall development effort. The purpose of this paper was to analyze the evaluation of system development process in a setting where there are multiple stakeholders in multiple organizations that develop a common information system (IS). The case is a consortium of universities that has developed a common student registrar system over a period of more than ten years.

The authors used a social process model to analyze the episodes and encounters in the system development and evaluation process. The research approach can be characterized as action case.

The authors found that in the early stages of system development the evaluation was emergent or even non‐existent. As the system was developed further and issues, such as delays and missing features, arose, there was a need for better control of the process. Thus, the evaluation process evolved through a series of critical encounters into a more proactive one.

The authors studied only one case so broad generalizations directly from the case cannot be made. However, the results could hold true for similar settings. The authors believe that the results are interesting to both researchers and practitioners since in outsourcing relationships the consortium kind of arrangement is becoming more common.

The authors analyze the evolution of IS evaluation processes in a setting with multiple client and vendor organizations and reveal the complexities of system evaluation in such settings. Moreover, they introduce three new concepts: reactive, transitional and proactive evaluation processes. These concepts can help researchers and practitioners to better understand the nature of the IS evaluation processes and to possibly choose an evaluation approach that has a better “fit” with the development situation at hand.