A longitudinal study of development and maintenance

https://doi.org/10.1016/j.infsof.2010.03.003Get rights and content

Abstract

Context

The information systems we see around us today are at first sight very different from those that were developed 15 years ago and more. On the other hand, it seems that we are still struggling with many of the same problems, such as late projects and unfulfilled customer demands.

Objective

The paper presents finding relative to the distribution of work between maintenance and development tasks, comparing to the results reported earlier to assess the stability of important metrics within the area.

Method

This paper presents the main results of a survey-investigation performed in 2008 in 67 Norwegian organizations comparing the distribution of work to results from similar investigations performed in Norway in 1993, 1998, and 2003. Some comparisons to similar investigations performed in USA before this is also provided.

Results

The amount of application portfolio upkeep (work made to keep up the functional coverage of the application system portfolio of the organization, including the development of replacement systems), is at the same level as reported in 1998 and 2003. The level of application maintenance is also on the same level as the similar investigations conducted in 2003 and 1998. There was a significant increase in both maintenance and application portfolio upkeep from 1993 to 1998, which could partly be attributed to be the extra maintenance and replacement-oriented work necessary to deal with the “year 2000 problem”, but this seemed to be reversed in 2003 and 2008. As for the 2003 investigation, the slow IT-market in general seemed to have influenced the results negatively seen from the point of view of application systems support efficiency in organization. No similar explanation can be used for the 2008 numbers.

Conclusion

Based on the last surveys it seems than a stable level of work distribution both on maintenance and application portfolio upkeep have been reached, although the underlying development technologies are still undergoing large changes. This is contrary to others claiming that the amount of maintenance is still increasing.

Introduction

Large changes in how we develop information systems and the underlying technology for information systems have been witnessed over the last 10–15 years. For instance, over this period the prevalent development methods, programming languages and general technological infrastructure have changed dramatically. In the early 90s, one was going from mainframe solutions to a client–server, and then to an internet architecture for many applications. Year 2000 and the dot.com crash had large impact locally on the development and maintenance of systems. More lately SOA, outsourcing, open source and agile development methodologies would also be expected to have an impact. According to [14] one of the impacts on the state of IS-development is the increasing amount of time used for maintenance of systems (instead of developing new systems). On the other hand, many of the intrinsic problems and aspects related to information systems support in organizations are similar now to what they were 15 years ago. Application systems are valuable when they provide information in a manner that enables people to meet their objectives more effectively [3]. Many have claimed that the large amount of system work that goes into maintenance is a sign on poor use of resources to meet these demands. On the other hand, as stated already in [4], it is one of the essential difficulties with application systems that they are under a constant pressure of change. Given the intrinsic evolutionary nature of the sources of system specifications, it should come as no surprise that specifications and the related information system must evolve as well [3], [24]. Thus, all successful application systems are changed; there is nothing detrimental about this. Unsuccessful systems or systems that can no longer be maintained and enhanced are removed or replaced. Talking about having a high percentage of maintenance work being done as a measure of information system support efficiency can thus be highly misleading.

The goal of both development activities and general maintenance is to keep the overall information system support of the organisation relevant to the organization, meaning that it supports the fulfilment of actual organizational needs. A lot of the activities usually labelled as maintenance, are in this light value-adding activates, enabling the users of the systems to do new task. On the other hand, a large proportion of the ‘new’ systems being developed are so-called replacement systems, mostly replacing the existing systems without adding much to what end-users can do with the overall application systems portfolio of the organization.

Based on this argumentation we have earlier developed the concept application portfolio upkeep1 as a high-level measure to evaluate to what extent an organisation is able to evolve their application system portfolio efficiently. How application portfolio upkeep is different from maintenance is described further below.

In this paper, we present the main results from a survey-investigation performed in Norwegian organisations during the end of 2008. The main motivation of the investigation is to compare the current development and maintenance situation in Norway with what has been reported in similar investigations in Norway and abroad earlier, to see if the pattern of resource use in IT departments is stable or if we can observe changes and in case why these changes are happening. Our main hypothesis in this regard is described in further detail in Section 3. A more comprehensive report from the investigation can be found in [7].

We will first give definitions of some of the main terms used within software development and maintenance, including the terms application portfolio upkeep and application portfolio evolution, notions which were introduced 15 years ago [18]. We then describe the research method and the main hypotheses investigated. The main results are then presented. In Section 5 we discuss threats to validity of the results. The last section summarises our results relative to the hypothesis, and presents ideas for further work.

Section snippets

Basic concepts

Maintenance has traditionally been divided into three types: corrective, adaptive and perfective [11] inspired by Swanson [35]. We here used the IEEE glossary terms in [11] with some clarifications.

Maintenance is defined as the process of modifying a software system or component after delivery.

  • 1.

    Corrective maintenance is performed to correct faults in hardware and software.

  • 2.

    Adaptive maintenance is performed to make the computer program usable in a changed environment.

  • 3.

    Perfective maintenance is

Research method

The survey form was implemented using the SurveyMonkey web-tool.3 SurveyMonkey is a much used web-based survey tool, which makes the set-up and follow-up of surveys easy to perform. Invitations were distributed through SurveyMonkey by e-mail to 300 Norwegian organizations. For different reasons, only 278 of the invitations was delivered to the organizations (e.g. since some potential respondents did not want to receive request from SurveyMonkey or had changed their

Results

First, we present some of the overall demographics of the survey. Similar results from our previous surveys conducted in 2003, 1998, and 1993 are included in parenthesis where the numbers are comparable.

42% (2003 – 20%; 1998 – 43%) of the organizations had a yearly data processing budget above 10 mill NKr (approx. 1.5 mill USD), and the average number of employees among the responding organizations was 1083 (2003 – 181; 1998 – 656; 1993 – 2347). The average number of full-time personnel in the

Validity of study

The results of our study should be interpreted cautiously as there are several threats to its validity. The discussion below is based on recommendations given in [15], [17].

Conclusion and further work

Revisiting our hypotheses, we conclude the following:

  • H1: There is no difference between the percentage of maintenance time in our survey and what are reported in previous surveys.

Not rejected: The overall time used for maintenance tasks is also in our investigation around 40% on average (when not including user support as maintenance. Obviously, as indicated by the SD-numbers, there are large variation between different organizations in the different years, but over a number of organizations,

Acknowledgements

We would like to thank all the participants of the survey-investigation for their effort in filling in the forms. We would also like to thank everyone helping us in the piloting and refinement of the questionnaire, and The Norwegian Computer Society for practical support in the arrangement of the survey. Finally thanks to the reviewers for useful comments and to Birgit Krogstie helping with the layout of the tables.

References (37)

  • J. Galtung

    Theory and Method of Social Research

    (1967)
  • IEEE Standard Glossary of Software Engineering Terminology,...
  • A. Gupta, The profile of software changes in reused vs. non-reused industrial software systems, Doctoral thesis 2009:90...
  • A. Gupta, O.P. Slyngstad, R. Conradi, P. Mohagheghi, H. Rønneberg, E. Landre, An empirical study of software changes in...
  • C. Jones, The Economics of Software Maintenance in the Twenty First Century, 2006....
  • M. Jørgensen, Empirical Studies of Software Maintenance, PhD thesis University of Oslo, 1994, Research Report 188, ISBN...
  • M. Kajko-Mattsson, C. Ahnlund, E. Lundberg, CM3: service level agreement, in: Proceedings of the 20th IEEE...
  • B.A. Kitchenham et al.

    Preliminary guidelines for empirical research in software engineering

    IEEE Transactions on Software Engineering

    (2002)
  • Cited by (0)

    View full text