Case study: factors for early prediction of software development success

Abstract

Project managers can make more effective and efficient project adjustments if they detect project high-risk elements early. We analyzed 42 software development projects in order to investigate some early risk factors and their effect on software project success. Developers in our organization found the most important factors for project success to be: (1) the presence of a committed sponsor and (2) the level of confidence that the customers and users have in the project manager and development team. However, several other software project factors, which are generally recognized as important, were not considered important by our respondents.

Introduction

Despite extensive research into successful software development [2], [8], [24], [26], many software development projects are still failing [15]. Research into software development is likely to continue until we have the development process under better control [8], [15], [16], [26]. In 1994, 31% of all corporate software development projects resulted in cancellation [8]. A more recent study found that 20% of software projects failed, and that 46% experienced cost and schedule overruns or significantly reduced functionality [15]. Another study suggested that failure rates for software development projects are as high as 85% [11]. Clearly, systems development projects can present serious risks to the well being of an organization [15]. Troubled projects can, for example, cause developers to suffer long hours of unpaid overtime, loss of motivation, and burnout, leading to excessive staff turnover and its associated costs.

Recent research that focuses on factors contributing to the failure of information systems projects usually takes a practitioner point of view (such as that of an IT manager, project manager or software developer) [7], [9], [14], [24]. Management also has a view of what constitutes a successful project and, in some cases, this view is different from that of the project manager and developers, while users may take another quite different view. (Note that when we refer to management, we are referring to corporate management; where appropriate, we will explicitly refer to a project manager.)

The major factors that contribute to the success or failure of software systems fall into seven categories: (1) management, (2) customers and users, (3) requirements, (4) estimation, and scheduling, (5) the project manager, (6) the software development process, and (7) development personnel [7], [15], [20], [22]. Our research investigates some of the most influential success factors early in the development process from a developer's perspective. Successful software development projects are commonly cited as having met agreed upon business objectives, and having been completed on time and within budget [12], [14], [19], [25]. Other definitions of success, include the degree to which the project achieved its goals; reliable, maintainable and met the requirements of users; user satisfaction; effective project teamwork and professional satisfaction on the part of the project manager [10], [21]; and the extent to which the software is actually used [4], [6]. Glass [9] noted a profound difference of opinion between managers and team members concerning software project success, and our recent research agrees with his view [23]. Management tends to consider a project a success if it is delivered on time, within budget and meets specified business objectives [25]. In contrast, developers are more likely to perceive a project to be a success whether it is ‘completed or canceled’ [15]. In a study of several projects, the only criteria for success that had strong agreement among all the involved parties, were ‘meets user requirements, achieves purpose, meets time scale, meets budget, happy users, and meets quality’ [14]. Glass [7] suggests that problems arising from expectations established at the outset of a project have a greater influence on a developer's perception of success and failure than what happens during project development. For developers, a successful project is one in which the product works the way it is supposed to work, their participation provides a technical challenge and a learning experience that develops new skills [15]. Developers also appreciate when a project's cost and schedule are comparable to normal industry standards, which should mean that excessive unpaid overtime will not be required [15]. On the other hand, developers perception of project failure can include unrealistic schedule expectations, lack of resources, and poor understanding of the project's scope at the outset [15].

Insight into early project factors likely to threaten project success will help project managers and other stakeholders to better predict the likelihood of their project's success. This will allow for corrective action early in the development process. This study is a first step towards the establishment of guidelines for project risk prioritization and mitigation in the early phases of software development. Because of the number of generic risks the project manager has to contend with, we wish to provide a severity roadmap showing the consequences of early project actions (taken or imposed). In addition, we will explore the developer's view of factors that impact project success; a successful project manager needs to be aware of success and failure factors from all stakeholder viewpoints, and to be able to respond to these perceptions, either directly or indirectly, in a timely manner [13]. We anticipate that results of this study (and other studies) will be incorporated into a project management decision support tool.