Empirical evaluation of a decision support model for adopting software product line engineering

doi:10.1016/j.infsof.2014.12.007

Information and Software Technology

Volume 60, April 2015, Pages 77-101

https://doi.org/10.1016/j.infsof.2014.12.007 Get rights and content

Abstract

Context

The software product line engineering (SPLE) community has provided several different approaches for assessing the feasibility of SPLE adoption and selecting transition strategies. These approaches usually include many rules and guidelines which are very often implicit or scattered over different publications. Hence, for the practitioners it is not always easy to select and use these rules to support the decision making process. Even in case the rules are known, the lack of automated support for storing and executing the rules seriously impedes the decision making process.

Objective

We aim to evaluate the impact of a decision support system (DSS) on decision-making in SPLE adoption. In alignment with this goal, we provide a decision support model (DSM) and the corresponding DSS.

Method

First, we apply a systematic literature review (SLR) on the existing primary studies that discuss and present approaches for analyzing the feasibility of SPLE adoption and transition strategies. Second, based on the data extraction and synthesis activities of the SLR, the required questions and rules are derived and implemented in the DSS. Third, for validation of the approach we conduct multiple case studies.

Results

In the course of the SLR, 31 primary studies were identified from which we could construct 25 aspects, 39 questions and 312 rules. We have developed the DSS tool Transit-PL that embodies these elements.

Conclusions

The multiple case study validation showed that the adoption of the developed DSS tool is justified to support the decision making process in SPLE adoption.

Introduction

A software product line is a set of software-intensive systems sharing a common, managed set of features that satisfy the specific needs of a particular market segment or mission and that are developed from a common set of core assets in a prescribed way [1]. Software Product Line Engineering (SPLE) is a systematic and comprehensive process that aims to develop and maintain product lines. SPLE aims to provide pro-active, pre-planned reuse at a large granularity (domain and product level) to develop applications from a core asset base.

Currently an increasing number of companies aim to adopt an SPLE approach with the goals of enhancing the quality of products, reducing time-to-market and optimizing production costs. The benefits for adopting SPLE have been extensively documented (e.g. [1], [2], [3], [4]) and discussed in experience reports (e.g. [5], [6], [7]). On the other hand it is commonly accepted that transition to SPLE is not easy. In general, it requires large upfront investments and as such forms a serious risk if the desired return-on-investment is not achieved. To illustrate this situation, consider, for example, the following typical realistic scenario of a company:

The engineering group manager of an organization that builds simulation systems is in a junction. The organization has delivered a number of successful simulation system projects in the last few years. Even though the simulation systems have a lot of common features, traditionally these systems were implemented using a single systems development approach. The company is aware that the reuse potential is not used. Recently there is an increasing pressure to decrease the costs and time-to-market of these systems. The manager has heard about Software Product Line Engineering, which is adopted by some of his company’s competitors and is aware of the potential benefits of SPLE. However, because of the risks associated with SPLE, it is not easy to provide a decision. After all, lots of investments need to be made and a wrong decision in this case could be dramatic. On the other hand, if a proper decision is made for the adoption of SPLE and the corresponding strategy, the risks could be mitigated and the expected return-on-investment could be achieved.

In the context of this scenario, two different decisions need to be made. First of all, it should be decided whether adopting an SPLE approach is indeed feasible for the organization. Secondly, if the adoption of SPLE is feasible, a decision should be made regarding the selection of an appropriate SPLE transition strategy. The SPLE community has provided several different approaches for assessing the feasibility of SPLE adoption and selecting transition strategies. These approaches usually include many rules and guidelines that can assist the decision maker to analyze the feasibility of SPLE adoption and select a proper transition strategy. Unfortunately, there are many different rules which are very often implicit or scattered over different publications. Even in case the rules are known, processing these rules manually is not trivial and requires lots of knowledge and experience. Hence, for the practitioners it is not always easy to select and use these rules to support the decision making process. To support the decision making process, it would be worthwhile to accumulate the rules and provide automated support to assist the decision makers.

In this paper we aim to systematically investigate and evaluate the impact of a decision support system (DSS) on decision-making in SPLE adoption. In alignment with this goal, we provide a decision support model (DSM) and the corresponding DSS, Transit-PL. In general DSS is a computer-based information system that supports business or organizational decision-making activities. Transit-PL is a DSS for supporting the decision making on product line adoption. Using Transit-PL, we present an empirical evaluation study that assesses the DSS via case studies. For this, we first apply a systematic literature review (SLR) on the existing primary studies that discuss and present approaches for analyzing the feasibility of SPLE adoption and transition strategies. Second, based on the data extraction and synthesis activities of the SLR, the required questions and rules are derived and implemented in the DSS. Third, for evaluating the impact of decision support on SPLE adoption and the selection of transition strategies, we have conducted a multiple case study approach with two case studies from the literature and a case study from an industrial company.

The SLR yielded 31 primary studies from which we could identify 25 aspects, 39 questions and 312 rules which are implemented in the DSS, Transit-PL. The case study design research includes two different types of cases related to SPLE adoption. The first type of cases is based on a retrospective analysis of two case studies in the literature; the second case discusses the qualitative analysis of a large size IT company, CompanyX.¹

The remainder of the paper is organized as follows. In Section 2 we provide the background for SPLE adoption and decision support models. Section 3 presents the results of the systematic literature review for identifying the primary studies to extract the questions and rules for feasibility analysis and transition strategy selection. Section 4, presents the DSS, Transit-PL that we have developed. Section 5 describes the multiple case study design. Section 6 presents the multiple case study results. Section 7 reviews the related work and finally Section 8 concludes the paper.

Section snippets

SPLE adoption and decision support

The methodology and roadmap for switching to SPLE from a traditional way of software development is defined as adoption or transition, and an action plan for this process is called transition strategy [7]. To describe the concepts related to SPLE adoption process Fig. 1 shows the conceptual model that we have developed earlier [8]. In general, the SPLE adoption is triggered by a well-defined goal that can be either based on internal or external motivations. External motivation refers to

Primary study selection and analysis

To derive the required knowledge for decision support for SPLE adoption we will first review the existing SPLE adoption approaches in the literature. For this we have conducted a systematic literature review in accordance with the guidelines described by Kitchenham and Charters [10]. In particular we are interested in answers to the following research questions:

RQ1: What are the existing SPLE feasibility analysis approaches?
RQ2: What are the existing SPLE transition strategies?
RQ3: What are the

Decision support system

The tool Transit-PL [39] has been developed to provide the basis for the empirical research that has been applied to verify applicability of the DSM constructed as described in Section 2. In Section 4.1 we present the overall architecture of this tool. Section 4.2 discusses the process for configuring and using this DSS. Section 4.3 describes the usage of this tool.

Case study design

Our primary objective is to evaluate the impact of the developed DSM on decision making in SPLE adoption. In alignment with this goal, in the previous sections we have discussed the DSM, the corresponding DSS and the data content that was derived via SLR. In this section we describe the case study design for validating the defined objectives. For this we apply the guidelines as discussed by Runeson and Höst [40]. We have followed the five steps: (1) case study design, (2) preparation for data

Evaluation of case studies

In this section we report on the results of the execution of the case studies that were introduced in the previous section. In Section 6.1 we first discuss the results of the retrospective case studies, followed by Section 6.2 in which we discuss the results of the prospective case study. In Section 6.3 we present the threats to validity.

Related work

Several studies have described approaches to assess the preconditions for a successful adoption of SPLE. We can categorize these approaches into approaches that check the feasibility of SPLE adoption, the selection of a feasible strategy and the measurement of the progress of the evolution of the adopted SPLE approach. These approaches were discussed in Section 3 Systematic Review. Our study can be considered complementary to these existing studies. We have used the knowledge of various

Conclusion

In this paper we have provided an empirical evaluation of the impact of decision support on the decision making process in SPLE adoption. We have considered two important decisions including the decision for adopting SPLE, and the decision for the selection of the product line adoption strategy. Both decisions are important and critical for organizations who aim to adopt SPLE. Obviously, the decision for adopting SPLE and the selection of the strategy is not an all or nothing decision. This was

References (53)

K. Sherif et al.
Domain engineering for developing software repositories: a case study
Decis. Support Syst.
(2002)
P. Clements et al.
Software Product Lines: Practices and Patterns
(2001)
G. Bockle et al.
Calculating ROI for software product lines
IEEE Softw.
(2004)
K. Schmid et al.
The economic impact of product line adoption and evolution
IEEE Softw.
(2002)
A. Birk et al.
Product line engineering, the state of the practice
IEEE Softw.
(2003)
F. Van Der Linden et al.
Software Product Lines in Action: The Best Industrial Practice in Product Line Engineering
(2007)
K. Pohl et al.
(2005)
B. Tekinerdogan, E. Tüzün, E. Şaykol, Multidimensional classification approach for defining product line engineering...
P.C. Clements, L.M. Northrop, A Framework for Software Product Line Practice, Version 5.0, 2007....
B. Kitchenham, S. Charters, Guidelines for Performing Systematic Literature Reviews in Software Engineering, No. EBSE...

C. Fritsch, R. Hahn, Product line potential analysis, in: Software Product Lines, 2004, pp....

F. Van Der Linden, J. Bosch, E. Kamsties, K. Känsälä, H. Obbink, Software product family evaluation, in: Software...

F. Ahmed et al.

Maturity assessment framework for business dimension of software product family

J. Interpretability Bus.

(2006)

F. Ahmed et al.

An organizational maturity model of software product line engineering

Softw. Qual. J.

(2009)

J. Bosch, Maturity and evolution in software product lines: approaches, artefacts and organization, in: Software...

S. Bühne, G. Chastek, T. Kaköla, P. Knauber, L. Northrop, S. Thiel, Exploring the context of product line adoption, in:...

C.W. Krueger, Easing the transition to software mass customization, PFE 01 Revised Papers from the 4th International...

J.D. McGregor et al.

Initiating software product lines

IEEE Softw.

(2002)

G. Böckle et al.

Adopting and institutionalizing a product line culture

Lect. Notes Comput. Sci.

(2002)

J. Ferreira Bastos, P. Anselmo Da Mota Silveira Neto, E. Santana De Almeida, S. Romero De Lemos Meira, J.F. Bastos,...

M. Kircher, C. Schwanninger, I. Groher, Transitioning to a software product family approach – challenges and best...

F. Van Der Linden, Family Evaluation Framework Overview & Introduction, Philips Medical Systems, 2005....

E. Niemelä, Strategies of product family architecture development, in: SPLC’05 Proceedings of the 9th International...

L. Northrop, Software Product Line Adoption Roadmap,...

C. Person, R. Schmid, C. Number, State-of-the-art Survey for Product Lines,...

M. Raatikainen, T. Soininen, T. Männistö, Case study Questions for Studying Software Product Families,...

Cited by (31)

A systematic review on food recommender systems
2024, Expert Systems with Applications
The Internet has revolutionised the way information is retrieved, and the increase in the number of users has resulted in a surge in the volume and heterogeneity of available data. Recommender systems have become popular tools to help users retrieve relevant information quickly. Food Recommender Systems (FRS), in particular, have proven useful in overcoming the overload of information present in the food domain. However, the recommendation of food is a complex domain with specific characteristics causing many challenges. Additionally, very few systematic literature reviews have been conducted in the domain on FRS. This paper presents a systematic literature review that summarises the current state-of-the-art in FRS. Our systematic review examines the different methods and algorithms used for recommendation, the data and how it is processed, and evaluation methods. It also presents the advantages and disadvantages of FRS. To achieve this, a total of 67 high-quality studies were selected from a pool of 2,738 studies using strict quality criteria. The review reveals that the domain of food recommendation is very diverse, and most FRS are built using content-based filtering and ML approaches to provide non-personalised recommendations. The review provides valuable information to the research field, helping researchers in the domain to select a strategy to develop FRS. This review can help improve the efficiency of development, thus closing the gap between the development of FRS and other recommender systems.
Analyzing systems product line engineering process alternatives for physical protection systems
2023, Management and Engineering of Critical Infrastructures
Physical protection systems (PPS) play an important role in safeguarding critical infrastructures against malevolent attacks. A PPS is a comprehensive solution that integrates people, procedures, and tools to provide effective protection. Developing an effective PPS is a complex process that requires consideration of various factors. Although PPSs may differ, they all share a similar structure and set of features that can be developed through a systematic approach to large-scale reuse. One such approach is product line engineering (PLE), which has been successfully used in various application domains to shorten the time to market, reduce costs, and enhance overall system quality. This paper provides an overview of the PPS domain, with a specific focus on its importance for critical infrastructures. We describe the three different types of PLE methods, including single, multiple, and feature-based PLE, and present a method for selecting the appropriate PLE method. To illustrate the practical application of the method, we apply it to a real industrial case study for a large-scale systems company.
Intricacies of medical drones in healthcare delivery: Implications for Africa
2021, Technology in Society
Citation Excerpt :
With this, we arrived at a total of (17) included papers, see Table 2 (a summary of the search outcome and the included papers), which was further screened for quality based on the RQs. To achieve this quality assessment, we defined a quality checklist [26]. The quality assessment contains questions asked about the included papers (see Table 1) and we coded the response to each question in the quality checklist with either; one (1) for a ‘Yes’ response, half (0.5) for a ‘Somewhat’ response and zero (0) for a ‘No’ response.
Unmanned aerial vehicles, also known as drones, can play a critical role in delivering health care. In this paper, we reviewed the literature to ascertain (1) the various medical supplies delivered by drones, (2) the challenges to the successful use of medical drones and (3) the potential benefits of medical drones. Implications for the African context is then provided. In achieving this objective, we employed a systematic literature review methodology and defined search strings, searched for relevant literature from PubMed, Scopus and Web of Science databases following a review protocol. The snowball technique was similarly used to search for other papers as well. A total of 17 out of 69 papers were included in the review after screening and applying a quality appraisal criterion. The results indicate that blood, AEDs, drugs, vaccines, and laboratory test samples etc were identified as part of the medical supplies aerially delivered by drones. Regulations, cost, misuse (evasion of people's privacy) and psychological effect on people who experienced drones used for bombing are part of the challenges that could result from using the technology for delivery of medical supplies. The benefits derived from drones range from reducing response times during health emergencies thus helping to save more lives, to being environmentally friendly as the CO₂ emissions levels are lower than conventional delivery by trucks and cars. In conclusion, drone technology has opened a new phase for the health sector and to sustain this technological innovation in Africa, there is the need for inclusive innovation to make drones safer and acceptable. This could be achieved by developing the needed policy framework with the regulators, providing other health resources to complement the use of drones, embarking on sensitizations on the usefulness of drones through cultural underpinnings of rural communities about drones and training the needed health personnel to handle dispatches of medical supplies with drones.
Software product lines and variability modeling: A tertiary study
2019, Journal of Systems and Software
Citation Excerpt :
Finally, an increasingly popular trend, which became more evident during the search update, is the use of SRs as a multi-method or triangulation manner. An SR is combined in a study with another research method, such as a case study or survey, and used as one data source to extend, refine or evaluate the study propositions or findings (Bastos et al., 2017; Da Silva et al., 2015; Myllärniemi et al., 2016; Rabiser et al., 2010; Hohl et al., 2018; Tüzün et al., 2015). We identified tens of unique variability models from the 39 SRs that reported a variability model (cf. Tables 9 and 13).
Context: A software product line is a means to develop a set of products in which variability is a central phenomenon captured in variability models. The field of SPLs and variability have been topics of extensive research over the few past decades. Objective: This research characterizes systematic reviews (SRs) in the field, studies how SRs analyze and use evidence-based results, and identifies how variability is modeled. Method: We conducted a tertiary study as a form of systematic review. Results: 86 SRs were included. SRs have become a widely adopted methodology covering the field broadly otherwise except for variability realization. Numerous variability models exist that cover different development artifacts, but the evidence is insufficient in quantity and immature, and we argue for better evidence. SRs perform well in searching and selecting studies and presenting data. However, their analysis and use of the quality of and evidence in the primary studies often remains shallow, merely presenting of what kinds of evidence exist. Conclusions: There is a need for actionable, context-sensitive, and evaluated solutions rather than novel ones. Different kinds of SRs (SLRs and Maps) need to be better distinguished, and evidence and quality need to be better used in the resulting syntheses.
Reporting systematic reviews: Some lessons from a tertiary study
2018, Information and Software Technology
Many of the systematic reviews published in software engineering are related to research or methodological issues and hence are unlikely to be of direct benefit to practitioners or teachers. Those that are relevant to practice and teaching need to be presented in a form that makes their findings usable with minimum interpretation.
We have examined a sample of the many systematic reviews that have been published over a period of six years, in order to assess how well these are reported and identify useful lessons about how this might be done.
We undertook a tertiary study, performing a systematic review of systematic reviews. Our study found 178 systematic reviews published in a set of major software engineering journals over the period 2010–2015. Of these, 37 provided recommendations or conclusions of relevance to education and/or practice and we used the DARE criteria as well as other attributes related to the systematic review process to analyse how well they were reported.
We have derived a set of 12 ‘lessons’ that could help authors with reporting the outcomes of a systematic review in software engineering. We also provide an associated checklist for use by journal and conference referees.
There are several areas where better reporting is needed, including quality assessment, synthesis, and the procedures followed by the reviewers. Researchers, practitioners, teachers and journal referees would all benefit from better reporting of systematic reviews, both for clarity and also for establishing the provenance of any findings.
The contribution that empirical studies performed in industry make to the findings of systematic reviews: A tertiary study
2018, Information and Software Technology
Systematic reviews can provide useful knowledge for software engineering practice, by aggregating and synthesising empirical studies related to a specific topic.
We sought to assess how far the findings of systematic reviews addressing practice-oriented topics have been derived from empirical studies that were performed in industry or that used industry data.
We drew upon and augmented the data obtained from a tertiary study that performed a systematic review of systematic reviews published in the period up to the end of 2015, seeking to identify those with findings that are relevant for teaching and practice. For the supplementary analysis reported here, we then examined the profiles of the primary studies as reported in each systematic review.
We identified 48 systematic reviews as candidates for further analysis. The many differences that arise between systematic reviews, together with the incompleteness of reporting for these, mean that our counts should be treated as indicative rather than definitive. However, even when allowing for problems of classification, the findings from the majority of these systematic reviews were predominantly derived from using primary studies conducted in industry. There was also an emphasis upon the use of case studies, and a number of the systematic reviews also made some use of weaker ‘experience’ or even ‘opinion’ papers.
Primary studies from industry play an important role as inputs to systematic reviews. Using more rigorous industry-based primary studies can give greater authority to the findings of the systematic reviews, and should help with the creation of a corpus of sound empirical data to support evidence-informed decisions.

View all citing articles on Scopus

View full text

Empirical evaluation of a decision support model for adopting software product line engineering

Abstract

Context

Objective

Method

Results

Conclusions

Introduction

Section snippets

SPLE adoption and decision support

Primary study selection and analysis

Decision support system

Case study design

Evaluation of case studies

Related work

Conclusion

Decis. Support Syst.

Software Product Lines: Practices and Patterns

Calculating ROI for software product lines

IEEE Softw.

The economic impact of product line adoption and evolution

IEEE Softw.

Product line engineering, the state of the practice

IEEE Softw.

Software Product Lines in Action: The Best Industrial Practice in Product Line Engineering

Maturity assessment framework for business dimension of software product family

J. Interpretability Bus.

An organizational maturity model of software product line engineering

Softw. Qual. J.

Initiating software product lines

IEEE Softw.

Adopting and institutionalizing a product line culture

Lect. Notes Comput. Sci.