Editorial
Special section: Controlled Experiments in Software Engineering

https://doi.org/10.1016/S0950-5849(01)00200-2Get rights and content

Introduction

To become a true engineering discipline, software engineering has to adopt and adapt research methods from other disciplines. Engineering means, among other things, that we should be able to understand, plan, monitor, control, estimate, predict and improve the way we engineer our products. One enabler for doing this is measurement. Software measurement forms the basis, but it is not sufficient. Empirical methods such as controlled experiments, case studies and surveys are needed to help us evaluate and validate the research results. These methods are needed so that it is possible to scientifically state whether something is better than something else is. Thus, empirical methods provide one important scientific basis for software engineering. For some type of problems, other methods are better suited; but in most cases, the best method is applying empiricism. The main reason is that software development is human intensive and hence it does not lend itself to analytical approaches. This means that empirical methods are essential to the researcher.

The empirical methods are however also crucial from an industrial point of view. Companies aspiring to be learning organizations have to consider the following definition of a learning organization:

A learning organization is an organization skilled at creating, acquiring, and transferring knowledge, and at modifying its behavior to reflect new knowledge and insights [1].

Garvin continues with stating that learning organizations are good at five activities: systematic problem solving, experimentation, learning from past experiences, learning from others, and transferring knowledge. This includes relying on scientific methods rather than guesswork. From the perspective of this special section, the key issue is the application of a scientific method and the use of experimentation.

In summary, the above means that software engineering researchers and learning organizations both have a need to embrace empirical methods. One of these methods is the use of controlled experiments. This special section focuses on showing how controlled experiments can be used in an engineering perspective, which then can be further enlarged in terms of the learning organization.

This special section contains three high-quality papers. They represent three different ways of applying the scientific method within software engineering research. This means that they have slightly different foci from an engineering perspective. Moreover, the papers also present controlled experiments from three different areas of software engineering.

The first paper by Thomas Thelin, Per Runeson and Björn Regnell entitled ‘Usage-Based Reading—An Experiment to Guide Reviewers with Use Cases’ focuses on software inspections and evaluates two competing approaches when guiding reviewers using use cases.

The second paper entitled ‘Impact of Effort Estimates on Software Project Work’ by Magne Jørgensen and Dag I.K. Sjøberg studies effort estimation to understand the influence of having early indications of the effort. The experiment is formulated based on two industrial case studies to better understand the findings from the case studies.

The third paper ‘Empirical Validation of Referential Integrity Metrics’ by Coral Calero, Mario Piattini and Marcela Genero is focused on validating two software metrics for controlling the quality of a relational database.

Thus, the papers provide a sample of how controlled experiments may be applied in software engineering research to understand, evaluate and validate. Before presenting the papers in some more detail, a brief introduction to controlled experiments in software engineering is provided. A more comprehensive introduction can be found in Ref. [2].

Section snippets

Articles

The three papers of this special section illustrate both different technical areas and different engineering perspectives.

References (6)

  • D.A. Garvin

    Building a Learning Organization

    (1998)
  • C. Wohlin et al.

    Experimentation in Software Engineering—An Introduction

    (1999)
  • J.W. Tukey

    Exploratory Data Analysis

    (1977)
There are more references available in the full text version of this article.

Cited by (0)

View full text