Environmental factors analysis and comparison affecting software reliability in development of multi-release software

Highlights

• Revisit the environmental factors based on their impact on software reliability for multi-release software by conducting a survey of environmental factors affecting software reliability for the next release's development.

• Investigate the significant environmental factors in each development phase to deliver valuable information for software management team.

• Compare the significant environmental factors between the development of multi-release software and the development of single release software.

Abstract

As the application of the principles of agile and lean software development, software multiple release becomes very common in the modern society. Short iteration and short release cycle have driven the significant changes of the development process of multi-release software product, compared with single release software product. Thus, it is time to conduct a new study investigating the impact level of environmental factors on affecting software reliability in the development of multi-release software to provide a sound and concise guidance to software practitioners and researchers. Statistical learning methods, like principle component analysis, stepwise backward elimination, lasso regression, multiple linear regression, and Tukey method, are applied in this study. Comparisons regarding significant environmental factors during the whole development process, principle components, significant environmental factors in each development phase and significance level of each development phase between the development of single release software and multi-release software are also discussed.

Introduction

As software systems are more deeply embedded in our everyday life, the dependence of our modern society on complex, intelligent and reliable large-scale software system is rapidly growing (Mohanty et al., 2013). Meanwhile, the possibility of carrying more faults in the large-scale software system is higher than decades ago. Software failures are becoming increasingly common in the field environment as the increasing complexity of software product (Sahoo et al., 2010). In order to continually align with the fast-changing customer's requirement and provide more reliable product to the market, most companies will release multiple versions of the software product since it is unlikely to deliver all the features in a single release and satisfy all the constraints within limited resource (Kapur et al., 2011; Saliu and Ruhe, 2005). The principles of multi-release software are: adding new features in the next release and fixing the remaining faults from previous release due to the fact that bug-free software product is not likely to be delivered in any release (Maurice et al., 2006). Software multiple releases not only makes company easily balance the competing stakeholder's demand and benefits but also increases reliability and customer satisfaction level during each release (Svahnberg et al., 2010; Ruhe and Momoh, 2005). It is very pragmatic yet interesting to investigate what are the significant environmental factors on affecting software reliability during the development of multi-release software.

Although no general definition has been given to define what are the environmental factors affecting software reliability during software development process, there have been a number of related works in the literature defined different types of factors in software development from various perspectives. For instance, Misra et al. (2009) identified some success factors in agile software development based on data analysis technique. Chow and Cao (2008) provided a survey study on the critical success factor of agile software development using quantitative approach. Clarke and O'Connor (2012) studied the situational factors affecting software development process. Zhang and Pham (2000) defined 32 environmental factors to capture the characteristics affecting software reliability during software development process for single release software. Another exploratory analysis is also provided (Zhang et al., 2001) to further analyze the detailed relation of these environmental factors. Furthermore, Zhu et al. (2015) presented a comparison analysis to compare the changes of these 32 environmental factors affecting software reliability after Zhang and Pham (2000) first proposed this idea. However, these researches mainly focus on the development process for single release software. To the extent of our knowledge, we have not seen any research studied the impact of the environmental factors affecting software reliability during the development of multi-release software.

In today's fast-changing technology environment, it is unlikely to include all the required features in a single release software. Most software companies will modify the parts of the existing modules to extend the current functionality and usability by adding new features in the next release and correcting the issues from previous release (Gorschek and Davis, 2008). The importance of software multiple release has been brought to our attention (Febrero et al., 2016). Compared with traditional release cycle, the modern software release cycle is shortened from traditional 18-month to 3-month (Al-Emran and Pfahl, 2007). Moreover, the challenges caused by modern realities are more wide-spreading. The rapid-changing market, the increasing involvement of end-users during software development and the new competitive threats all require reliable products (Beck et al., 2001). Hence, how to assure reliability and quality for multi-release software in such a challenging environment is critical to the company and research institution.

The resources and constraints for the development of multi-release software are different. Software development team needs to select the corresponding features presented in the next release with respect to customers’ feedback and market requirements. Since software will be released in increments for multi-release software, thus, coupling this concept with other principles such as continuous unit testing and pair programming will better arrange the cost distribution (Beck et al., 2001).

During the past decades, software version planning and release has been studied by many researchers. Szoke (2011) developed a staged-delivery global optimized model for agile release planning. Li et al. (2014) proposed a multi-objective optimization technique to optimize three main objectives with respect to cost, revenue, and uncertainty for robust next release problem. Etgar et al. (2017) explored several optimization approaches to determine the content and release date for each release in order to provide optimal net present value (NPV). In addition, software reliability model related to software multiple release is also investigated by some researchers. Hu et al. (2011) proposed a software reliability model considering the effect of multiple releases on the fault detection process in software development. Yang et al. (2016) incorporated fault detection and fault correction process in multi-release software reliability modeling. Time delay is assumed in fault repair after detecting faults. However, all of the past research related to release planning or multi-release software reliability modeling did not investigate the impact of environmental factors on software reliability in the development of multi-release software.

Therefore, it is plausible to conduct a new study to investigate the impact of these environmental factors on software reliability for multi-release software and compare the difference between single release software and multi-release software.

As discussed in the previous section, the development process for multi-release software is different with single release software. For example, how to select the features in which release and how to determine the removal percentage of detected software faults. This study aims to revisit the environmental factors in terms of their impact on software reliability for multi-release software. This study is carried out by conducting a survey of environmental factors affecting software reliability for the next release's development.

First, we need to find out what are the significant environmental factors affecting reliability in the development of multi-release software. Secondly, what are the correlations between environmental factors and is it possible to reduce the dimension of those variables to provide concise and sound information for researchers and practitioners? Moreover, the significant environmental factors in each development phase and the significance level for each development phase are also investigated in order to provide a helpful time/resource allocation matrix for software development team. Thirdly, we also compare the significant environmental factors during the whole development process, principle components, the significant environmental factors in each development phase, significance level of each development phase between the development of single release software and multi-release software. At the end of this study, other statistical method in terms of variable selection is also applied in this paper to provide an insightful matrix for readers according to their selection priority. Software practitioners can choose the results coming from different methodologies based on their requirements.

Section 2 briefly describes how we collected the data for this survey. Section 3 presents the detailed analysis for the environmental factors and each development phase in the development of multi-release software. Section 4 compares the new findings for multi-release software in this study with the previous findings for single release software (Zhu et al., 2015). Section 5 reveals the significant environmental factors based on other variable selection method. Section 6 concludes this study and discusses the sound and pressing suggestion for the future research.