An industrial study of reuse, quality, and productivity

doi:10.1016/S0164-1212(00)00121-7

Journal of Systems and Software

Volume 57, Issue 2, 15 June 2001, Pages 99-106

https://doi.org/10.1016/S0164-1212(00)00121-7 Get rights and content

Abstract

The relationship between amount of reuse, quality, and productivity was studied using four sets of C and C++ modules collected from industrial organizations. The data domains are: text retrieval, user interface, distributed repository, medical records. Reuse in this paper is ad hoc, black box, compositional code reuse. The data generally show that more reuse results in higher quality, but are ambiguous regarding the relationship between amount of reuse and productivity.

Introduction

This paper presents a quasi-experimental study of four sets of industrial data to determine the effects of reuse on software quality and productivity. This is an important question since many organizations today are attempting to significantly increase the quality and productivity of their software systems via reuse (Frakes and Isoda, 1994).

Despite its importance, there is a paucity of industrial studies on this topic. Frakes and Terry (1996) provides a survey of these studies. Some recent studies on the relationship between reuse, quality and productivity based on academic data have been reported (Basili et al., 1996, Devanbu et al., 1996). A key difficulty is obtaining industrial data. One serious problem is the type of available industrial data. Industrial data are usually typified by small sample sizes and lack of strict experimental controls. Industrial data also often come in small sets, and conclusions have to be derived from analysis across data sets, rather than within a single set. There are two approaches to this situation. One is to not do empirical, industrial studies until more perfect data become available. As the lack of experimental studies in software engineering attests, this often means that studies are never done because “scientifically acceptable” data never become available.

The other approach is to analyze the data that are available, and learn what we can from them using exploratory quasi-experimental techniques. This is the approach taken in this paper. Our method is to do a correlational study (Campbell and Stanley, 1966) of quality and productivity measures and “amount of reuse” measures derived from four small sets of industrial data.

There are many types of software reuse. A typology of them is reported in Frakes and Terry (1996). The data in this paper are based on ad hoc, black box, compositional code reuse. Ad hoc means that the observed reuse is not part of a repeatable, mandated organizational process. Ad hoc reuse is by far more common than systematic reuse, though the latter is thought to be more powerful. Black box reuse is reuse of a software item without modification. Compositional reuse means that the software system was built by a human programmer out of components, as opposed to generating a system automatically from specifications.

This paper is organized as follows. In Section 2, we define the measures of amount of reuse, quality, and productivity used in this study. Section 3 presents a discussion of the experimental methods used. Section 4 presents the analysis of the four data sets, and Section 5 presents conclusions and suggestions for future work.

Section snippets

Quality and productivity measures

Since the first studies of McCall et al. (1977), it has been clear that it is not possible to measure the quality and productivity of software systems with a single number. Accordingly, many measures of software quality and productivity have been proposed (Conte et al., 1986, Arthur, 1993).

In this paper we use measures of quality and productivity that we have observed to be commonly used in industrial practice. Not all the measures that we use are the best possible measures for describing

Structure of the study

We collected four data sets from four different sources. Table 3 summarizes the sizes of the samples and the quality and productivity measures used. An “X” in a cell indicates that a given quality or productivity measure was used for a given data set.

This study is an exploratory correlational study (Campbell and Stanley, 1966) of quality and productivity measures and amount of reuse measures. A correlational study is a kind of quasi-experiment. Quasi-experiments are used in settings where

PRC Inc

Our first data set was obtained from PRC Inc., a large government contracting company located in Fairfax, VA, USA. The domain of the software was text processing and retrieval. We do not have demographic data on the developers of this software. ERL and ERF were measured for three software modules written in the C language. Deltas in the PRC data were extracted using SCCS, a change management tool on Unix. Table 4 summarizes these measures.

The rank correlation for ERL and Deltas is −0.5 and for

Conclusions and future work

This paper is an exploratory study of the relationships between amount of reuse, and quality and productivity in software modules from four industrial organizations. The data sets are from four different domains: text retrieval, user interface, distributed repository, and medical records. The data in this paper are based on ad hoc, black box, compositional code reuse. The modules in the study were written in C and C++.

The amount of reuse was measured with RL and RF. The rl tool was used to

Acknowledgements

We would like to thank Prof. G. Kovacs for his insightful comments, N. Scioscia and G. Cardino for their help in finding data, CIM-EXP (Budapest, Hungary), PRC (Falls Church, VA), and Software Uno S.r.l. (Padova, Italy) for the data they provided us. Thanks also to Omar Alonso, Gabriella Belli, Gloria Hasslacher, Eric Liu, Romana Spasojevic, Milorad Stefanovic, Raymond Wong, and Jason Yip for their reviews of a draft of the paper. The second author thanks the University of Calgary, the

William Frakes is an associate professor in the Computer Science Department at Virginia Tech. He chairs the IEEE TCSE committee on software reuse, and edits ReNews. He has a B.L.S. from the University of Louisville, an M.S. from the University of Illinois at Urbana-Champaign, and an M.S. and Ph.D. from the Syracuse University.

References (20)

W. Li et al.
Object oriented metrics that predict maintainability
Journal of Systems and Software
(1993)
L.J. Arthur
Improving Software Quality – An Insider's Guide to TQM
(1993)
V.R. Basili et al.
An investigation of human factors in software development
IEEE Computer
(1979)
V. Basili et al.
How reuse influences productivity in object-oriented systems
Communications of the ACM
(1996)
W. Berg et al.
Lessons learned from the OS/400 OO project
Communications of the ACM
(1995)
B. Boehm et al.
Cost models for future software life cycle processes: COCOMO 2.0
Annals of Software Engineering
(1995)
D.T. Campbell et al.
Experimental and Quasi-Experimental Designs for Research
(1966)
M.P. Cline
The pros and cons of adopting and applying design patterns in the real world
Communications of the ACM
(1996)
S.D. Conte et al.
Software Engineering Metrics and Models
(1986)
Curry, W., Succi, G., Smith, M., Liu, E., Wong, R., 1999. Empirical analysis of the correlation between amount-of-reuse...

There are more references available in the full text version of this article.

Cited by (61)

Understanding and evaluating software reuse costs and benefits from industrial cases—A systematic literature review
2024, Information and Software Technology
Software reuse costs and benefits have been investigated in several primary studies, which have been aggregated in multiple secondary studies as well. However, existing secondary studies on software reuse have not critically appraised the evidence in primary studies. Moreover, there has been relatively less focus on how software reuse costs and benefits were measured in the primary studies, and the aggregated evidence focuses more on software reuse benefits than reuse costs.
This study aims to cover the gaps mentioned in the context above by synthesizing and critically appraising the evidence reported on software reuse costs and benefits from industrial cases.
We used a systematic literature review (SLR) to conduct this study. The results of this SLR are based on a final set of 30 primary studies.
We identified nine software reuse benefits and six software reuse costs, in which better quality and improved productivity were investigated the most. The primary studies mostly used defect-based and development time-based metrics to measure reuse benefits and costs. Regarding the reuse practices, the results show that software product lines, verbatim reuse, and systematic reuse were the top investigated ones, contributing to more reuse benefits. The quality assessment of the primary studies showed that most of them are either of low (20%) or moderate (67%) quality.
Based on the number and quality of the studies, we conclude that the strength of evidence for better quality and improved productivity as reuse benefits is high. There is a need to conduct more high quality studies to investigate, not only other reuse costs and benefits, but also how relatively new reuse-related practices, such as InnerSource and microservices architecture, impact software reuse.
Metrics for reusability of java language components
2022, Journal of King Saud University - Computer and Information Sciences
Citation Excerpt :
Reusability has been an active part of software development for the past several decades and it has its roots back in 1969 when Mcllroy introduced the concept of reusable components in an international conference held at Garmisch, Germany (McIllroy, 1968). Based on the study conducted by Frakes and Succi aiming at determining the relationship between the extent of reuse, software quality, and productivity gain, it is concluded that promoting reuse always results in improved underlying software system quality (Frakes and Succi, 2001). With the gain in the popularity of FOSS (Free and Open Source Software), software components are managed in the form of packages (Cosmo et al., 2009).
Software reuse is a promising opportunity for a software development process that helps in achieving an improved quality of the developed software system. Improved quality also promotes software architecture reuse opportunities. From the past few decades, software reuse is actively gaining popularity in the form of Component Based Software Development (CBSD). The components being the basic unit in CBSD, accessing their reusability is a useful and open area of investigation. The extent of the reusability can be quantified based on the extent of intra and inter-connectivity of the attributes/ characteristics of the component. This can be computed by defining suitable metrics to measure the reusability of a component. Although, there exist some metric suites such as CK, MOOD, Halstead, McCabe’s metric, etc. for measuring the reusability of the object-oriented software systems, but they are not directly reusable in the CBSD environment. This is due to the fact that these metrics are applicable only at the class level and they do not target the interfaces of the component that are important and define the composability of the component in the CBSD environment. Therefore, this paper aims to study the reusability, flexibility, composability, and adaptability aspects of the software component. Further, based on the study conducted, we propose a reusability metric suite specially designed for the JavaBeans. This metric helps to measure the architecture level reusability of the JavaBeans. The JavaBeans component standard is specially targeted in this paper because components based on this standard are not fully explored and only a few metric suites exist for measuring the reusability of the JavaBeans. The proposed component reusability metric suite is validated both experimentally as well as against Briand’s framework.
What software reuse benefits have been transferred to the industry? A systematic mapping study
2018, Information and Software Technology
Citation Excerpt :
The basic principle of the CBD is the idea that similar functionalities can be useful in different situations, making the components that offer them reusable [81]. Frakes and Succi [S10] [41] presented a detailed research on the reuse of code modules in an industrial context. The authors found a direct relationship between the amount of reuse and quality, while the results for productivity were ambiguous, in line with [S9, S30] [11,61].
The term software reuse was first used in 1968 at the NATO conference. Since then, work in the scientific literature has stated that the application of software reuse offers benefits such as increase in quality and productivity. Nonetheless, in spite of many publications reporting software reuse experiences, evidence that such benefits having reached industrial settings is scarce.
To identify and classify the benefits transferred to real-world settings by the application of software reuse strategies.
We conducted a systematic mapping study (SMS). Our search strategies retrieved a set of 2,413 papers out of which 49 were selected as primary studies. We defined five facets to classify these studies: (a) the type of benefit, (b) the reuse process, (c) the industry's domain, (d) the type of reuse and (e) the type of research reported.
Quality increase (28 papers) and Productivity increase (25 papers) were the two most mentioned benefits. Component-Based Development (CBD) was the most reported reuse strategy (41%), followed by Software Product Lines (SPL, 30%). The selected papers mentioned fourteen industrial domains, of which four stand out: aerospace and defense, telecommunications, electronics and IT services. The application of systematic reuse was reported in 78% of the papers. Regarding the research type, 50% use evaluation research as the investigation method. Finally, 13 papers (27%) reported validity threats for the research method applied.
The literature analyzed presents a lack of empirical data, making it difficult to evaluate the effective transfer of benefits to the industry. This work did not find any relationship between the reported benefits and the reuse strategy applied by the industry or the industry domain. Although the most reported research method was industrial case studies (25 works), half of these works (12) did not report threats to validity.
CollabRDL: A language to coordinate collaborative reuse
2017, Journal of Systems and Software
Citation Excerpt :
To illustrate, in (Lim, 1994), two case studies were conducted: the first reduced by 51% defects (quality improvement) and increased productivity by 57%, and the second got 24% in reducing defects and 40% increasing productivity. However, other studies were not so conclusive (Frakes and Succi, 2001). Framework instantiation is a type of software reuse.
Coordinating software reuse activities is a complex problem when considering collaborative software development. This is mainly motivated due to the difficulty in specifying how the artifacts and the knowledge produced in previous projects can be applied in future ones. In addition, modern software systems are developed in group working in separate geographical locations. Therefore, techniques to enrich collaboration on software development are important to improve quality and reduce costs. Unfortunately, the current literature fails to address this problem by overlooking existing reuse techniques. There are many reuse approaches proposed in academia and industry, including Framework Instantiation, Software Product Line, Transformation Chains, and Staged Configuration. But, the current approaches do not support the representation and implementation of collaborative instantiations that involve individual and group roles, the simultaneous performance of multiple activities, restrictions related to concurrency and synchronization of activities, and allocation of activities to reuse actors as a coordination mechanism. These limitations are the main reasons why the Reuse Description Language (RDL) is unable to promote collaborative reuse, i.e., those related to reuse activities in collaborative software development. To overcome these shortcomings, this work, therefore, proposes CollabRDL, a language to coordinate collaborative reuse by providing essential concepts and constructs for allowing group-based reuse activities. For this purpose, we extend RDL by introducing three new commands, including role, parallel, and doparallel. To evaluate CollabRDL we have conducted a case study in which developer groups performed reuse activities collaboratively to instantiate a mainstream Java framework. The results indicated that CollabRDL was able to represent critical workflow patterns, including parallel split pattern, synchronization pattern, multiple-choice pattern, role-based distribution pattern, and multiple instances with decision at runtime. Overall, we believe that the provision of a new language that supports group-based activities in framework instantiation can help enable software organizations to document their coordinated efforts and achieve the benefits of software mass customization with significantly less development time and effort.
Comparing reuse strategies in different development environments
2015, Advances in Computers
There is a debate in the aerospace industry whether lessons from reuse successes and failures in nonembedded software can be applied to embedded software. This chapter analyzes and compares reuse success and failures in embedded versus nonembedded systems. A survey of the literature identifies empirical studies of reuse that can be used to compare reuse outcomes in embedded versus nonembedded systems. Reuse outcomes include amount of reuse, effort, quality, performance, and overall success. We also differentiate between types of development approaches to determine whether and how they influence reuse success or failure. In particular, for some development approaches, quality improvements and effort reduction are low for embedded systems. This is particularly relevant to the aerospace industry as it has been subject to reuse mandates for its many embedded systems.
Code Reuse and Sustainable Testing-A Comparative Survey and Study
2023, 2023 International Conference on Artificial Intelligence and Smart Communication, AISC 2023

View all citing articles on Scopus

Giancarlo Succi is a Professor of Electrical and Computer Engineering at the University of Alberta, Edmonton, AB. He holds a Laurea degree in Electrical Engineering (Genova, Italy, 1988) Italy, an M.Sc. in Computer Science (SUNY Buffalo, NY, 1991), and a Ph.D. in Electrical and Computer Engineering (Genova, Italy, 1993). His research interests include software reuse, software metrics, software product lines, and software engineering economics. He has chaired and co-chaired several international, scientific events and published papers in international journals and conferences.

View full text

An industrial study of reuse, quality, and productivity

Abstract

Introduction

Section snippets

Quality and productivity measures

Structure of the study

PRC Inc

Conclusions and future work

Acknowledgements

Journal of Systems and Software

Improving Software Quality – An Insider's Guide to TQM

An investigation of human factors in software development

IEEE Computer

How reuse influences productivity in object-oriented systems

Communications of the ACM

Lessons learned from the OS/400 OO project

Communications of the ACM

Cost models for future software life cycle processes: COCOMO 2.0

Annals of Software Engineering

Experimental and Quasi-Experimental Designs for Research

The pros and cons of adopting and applying design patterns in the real world

Communications of the ACM

Software Engineering Metrics and Models