Supervised machine learning approach to predict qualitative software product

Abstract

Software development process (SDP) is a framework imposed on software product development and is a multi-stage process wherein a wide range of tasks and activities pan out in each stage. Each stage requires careful observations to improve productivity, quality, etc. to ease the process of development. During each stage, problems surface likes constraint of on-time completion, proper utilization of available resources and appropriate traceability of work progress, etc. and may lead to reiteration due to the defects spotted during testing and then, results into the negative walk-through due to unsatisfactory outcomes. Working on such defects can help to take a step towards the proper steering of activities and thus to improve the expected performance of the software product. Handpicking the proper notable features of SDP and then analyzing their nature towards the outcome can greatly help in getting a reliable software product by meeting the expected objectives. This paper proposed supervised Machine Learning (ML) models for the predictions of better SDP, particularly focusing on cost estimation, defect prediction, and reusability. The experimental studies were conducted on the primary data, and the evaluation reveals the model suitability in terms of efficiency and effectiveness for SDP prediction (accuracy of cost estimation: 65%, defect prediction: 93% and reusability: 82%).

Appendices

Appendix A

Appendix B

See appendix Tables 12, 13, 14.

Table 12 Data model for cost estimation prediction

Table 13 Data model for defect prediction

Table 14 Data model for reusability prediction

2.1 Glossary

In order to make more understandable for the researchers who have no or little or limited prior experience in ML evaluation metrics, relevant key terminologies are provided.

2.1.1 Mean absolute error (MAE)

MAE is the overall average calculation of error, obtained under the differences between the predicted and the actual observation [11] and presented in Eq. G.1, where, y is the predicted result, $\widehat{\mathrm{y}}$ is the actual result, and n is the number of observations. It is to measure the closeness of the prediction of the eventual outcomes and the lower value signifies better prediction.

$$MAE = \frac{1}{n}\sum\limits_{j = 1}^{n} {\left| {y_{j} - \hat{y}_{j} } \right|}$$

(G.1)

2.1.2 Root mean square error (RMSE)

It is the square root of the overall average of the squared differences of data calculated between the predicted and actual observation [11] and is presented in Equation G.2, where, y is the predicted result, $\widehat{\mathrm{y}}$ is the actual result, and n is the number of observations. It represents the sample standard deviation of the differences between predicted values and observed values, and the lower value signifies better prediction.

$${\text{RMSE}} = \sqrt {\frac{1}{n}\sum\nolimits_{j = 1}^{n} {\left( {{\text{y}}_{j} - {\hat{\text{y}}}_{j} } \right)}^{2} }$$

(G.2)

2.1.3 Confusion matrix

It is the technique for conveying the performance of the classification algorithm. It contains four aspects of performance measures which are being compared over the actual and the predicated observations. The four aspects are: True Positive, True Negative, False Positive and False Negative [3].

True Positive (TP): It is for the correctly predicted event values.

True Negative (TN): It is for the correctly predicted no-event values.

False Positive (FP): It is for the incorrectly predicted event values.

False Negative (FN): It is for the incorrectly predicted no-event values.

2.1.4 Accuracy

It is the fraction of samples predicted correctly [15] and is presented in Eq. G.3, where, TP is True Positive, TN is True Negative, FP is False Positive, and FN is False Negative (refer to Confusion Matrix).

$${\text{Accuracy}} = \frac{{{\text{TP}} + {\text{TN}}}}{{{\text{TP}} + {\text{TN}} + {\text{FP}} + {\text{FN}}}}$$

(G.3)

2.1.5 Precision

It is the fraction of predicted positive events that are actually positive [15] and is presented in Equation G.4, where, TP is True Positive and FP is False Positive (refer to Confusion Matrix). It is to answer the question on what proportion of positive identifications were actually correct and higher value represents better relevant prediction over irrelevant ones.

$${\text{Precision}} = \frac{{{\text{TP}}}}{{{\text{TP}} + {\text{FP}}}}$$

(G.4)

2.1.6 Recall

It is the fraction of predicted positive results that are predicted correctly [15] and is presented in Equation G.5, where, TP is True Positive and FN is False Negative (refer to Confusion Matrix). It is to answer the question on what proportion of actual positives were identified correctly and higher value represents better relevant prediction. It is also called as True Positive Rate (TPR).

$${\text{Recall}} = \frac{{{\text{TP}}}}{{{\text{TP}} + {\text{FN}}}}$$

(G.5)

High precision relates to a low false positive rate, and high recall relates to a low false negative rate. High recall and high precision signify the classifier is very good.

2.1.7 F1-score

It is the harmonic mean of the Precision and Recall with a higher score signifies a better model accuracy [15] and is presented in Equation G.6.

$${\text{F1}} - {\text{Score}} = \frac{{2{\text{*}}\left( {{\text{precision*recall}}} \right)}}{{{\text{precision}} + {\text{recall}}}}$$

(G.6)