Abstract
By far, many package cohesion metrics have been proposed from internal structure view and external usage view. Based on whether client usage context (i.e., the way packages are used by their clients) is exploited, we group these metrics into two categories: non-context-based and context-based. Currently, there is no comprehensive empirical research devoted to understanding the actual value of client usage context for fault-proneness prediction. In this study, we conduct a thorough empirical study to investigate the value of considering client usage context in package cohesion for fault-proneness prediction. First, we use principal component analysis to examine the relationships between context-based and non-context-based cohesion metrics. Second, we employ univariate logistic regression analysis to investigate the correlation between context-based cohesion metrics and fault-proneness. Then, we build multivariate prediction models to analyze the ability of context-based cohesion metrics for fault-proneness prediction when used alone or used together with non-context-based cohesion metrics. To obtain comprehensive evaluations, we evaluate the effectiveness of these multivariate models in the ranking and classification scenarios from both cross-validation and across-version perspectives. The experimental results show that (1) context-based cohesion metrics are complementary to non-context-based cohesion metrics; (2) most of context-based cohesion metrics have a significantly negative association with fault-proneness; (3) when used alone or used together with non-context-based cohesion metrics, context-based cohesion metrics can substantially improve the effectiveness of fault-proneness prediction in most studied systems under both cross-validation and across-version evaluation. Client usage context has an important value in package cohesion for fault-proneness prediction.
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10515-016-0198-6/MediaObjects/10515_2016_198_Fig1_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10515-016-0198-6/MediaObjects/10515_2016_198_Fig2_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10515-016-0198-6/MediaObjects/10515_2016_198_Fig3_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10515-016-0198-6/MediaObjects/10515_2016_198_Fig4_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10515-016-0198-6/MediaObjects/10515_2016_198_Fig5_HTML.gif)
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Abdeen, H., Ducasse, S., Sahraoui, H. : Modularization metrics: assessing package organization in legacy large object-oriented software. In: WCRE, pp. 394-398 (2011)
Abdeen, H., Sahraoui, H., Shata, O., Anquetil, N., Ducasse, S.: Towards automatically improving package structure while respecting original design decisions. In: IEEE Working Conference on Reverse Engineering (WCRE), pp. 212–221 (2013)
Abdi, H.: Factor Rotations in Factor Analyses. Encyclopedia for Research Methods for the Social Sciences. Sage, Thousand Oaks (2003)
Allen, E.B., Khoshgoftaar, T.M., Chen, Y.: Measuring coupling and cohesion of software modules: an information theory approach. In: METRICS, pp. 124–134 (2001)
Arisholm, E., Briand, L.C., Fuglerud, M.: Data mining techniques for building fault-proneness models in telecom java software. In: Proceedings of the 18th IEEE International Symposium on Software Reliability, pp. 215–224 (2007)
Arisholm, E., Briand, L.C., Johannessen, B.: A systematic and comprehensive investigation of methods to build and evaluate fault prediction models. J. Syst. Softw. 83(1), 2–17 (2010)
Basili, V., Briand, L.C., Melo, W.: A validation of object-oriented design metrics as quality indicators. IEEE Trans. Softw. Eng. 22(10), 751–761 (1996)
Belsley, D., Kuh, E., Welsch, R.: Regression Diagnostics: Identifying Influential Data and Sources of Collinearity. Wiley, New York (1980)
Belsley, D., Kuh, E., Welsch, R.: Regression Diagnostics: Identifying Influential Data and Sources of Collinearity. Wiley, New York (2005)
Briand, L.C., Morasca, S., Basili, V.R.: Property-based software engineering measurement. IEEE Trans. Softw. Eng. 22(1), 68–86 (1996)
Briand, L.C., Morasca, S., Basili, V.R.: Defining and validating measures for object-based high-level design. IEEE Trans. Softw. Eng. 25(5), 722–743 (1999)
Briand, L., Wüst, J., Daly, J., Porter, D.: Exploring the relationships between design measures and software quality in object-oriented systems. J. Syst. Softw. 51(3), 245–273 (2000)
Chen, L., Fang, B., Shang, Z., Tang, Y.: Negative samples reduction in cross-company software defects prediction. Inf. Softw. Technol. 62, 67–77 (2015)
Chowdhury, I., Zulkernine, M.: Using complexity, coupling, and cohesion metrics as early indicators of vulnerabilities. J. Syst. Archit. 57(3), 294–313 (2011)
Constantine, L., Yourdon, E.: Structured Design. Prentice Hall, Upper Saddle River (1979)
Couto, C., Pires, P., Valente, M.T., Bigonha, R.S.: Predicting software defects with causality tests. J. Syst. Softw. 93, 24–41 (2014)
D’Ambros, M., Lanza, M., Robbes, R.: Evaluating defect prediction approaches: a benchmark and an extensive comparison. Empir. Softw. Eng. 17(4–5), 531–577 (2012)
Doval, D., Mancoridis, S., Mitchell, B.S. : Automatic clustering of software systems using a genetic algorithm. In: STEP, pp. 73–81 (1999)
Elish, M.O., Al-Yafei, A.H., Al-Mulhem, M.: Empirical comparison of three metrics suites for fault prediction in packages of object-oriented systems: A case study of Eclipse. Adv. Eng. Softw. 42(10), 852–859 (2011)
Fenton, N., Pfleeger, S.L.: Software Metrics: A Rigorous and Practical Approach, 2nd edn. International Thomson Computer Press, London (1996)
Gorman, J.: OO design principles & metrics. http://www.codemanship.co.uk/parlezuml/metrics/OO%20Design%20Principles%20&%20Metrics.pdf (2006)
Gupta, V., Chhabra, J.K.: Package level cohesion measurement in object-oriented software. J. Braz. Comput. Soc. 18(3), 251–266 (2012)
Hall, T., Beecham, S., Bowes, D., Gray, D., Counsell, S.: A systematic literature review on fault prediction performance in software engineering. IEEE Trans. Softw. Eng. 38(6), 1276–1304 (2012)
Hata, H., Mizuno, O., Kikuno, T.: Bug prediction based on fine-grained module histories. In: Proceedings of the 34th International Conference on Software Engineering, pp. 200–210 (2012)
He, P., Li, B., Liu, X., Chen, J., Ma, Y.: An Empirical Study on Software Defect Prediction with a Simplified Metric Set, vol. 59, pp. 170–190. Wuhan University, Wuhan (2014)
He, Z., Shu, F., Yang, Y., Li, M., Wang, Q.: An investigation on the feasibility of interproject defect prediction. Autom. Softw. Eng. 19(2), 167–199 (2012)
Hocking, R.R.: The analysis and selection of variables in linear regression. Biometrics 32(1), 1–49 (1976)
Howell, D.C.: Statistical Methods for Psychology. Duxbury Press, Belmont (2002)
Janes, A., Scotto, M., Pedrycz, W., Russo, B., Stefanovic, M., Succi, G.: Identification of defect-prone classes in telecommunication software systems using design metrics. Inf. Sci. 176(24), 3711–3734 (2006)
Kaiser, H.F.: The varimax criterion for analytic rotation in factor analysis. Psychometrika 23(3), 187–200 (1958)
Kamei, Y., Matsumoto, S., Monden, A., Matsumoto, K., Adams, B. , Hassan, A.E.: Revisiting common bug prediction findings using effort-aware models. In: Proceedings of the 26th IEEE International Conference on Software Maintenance, pp. 1–10 (2010)
Kamei, Y., Shihab, E., Adams, B., Hassan, A.E., Mockus, A., Sinha, A., Ubayashi, N.: A large-scale empirical study of just-in-time quality assurance. IEEE Trans. Softw. Eng. 39(6), 757–773 (2013)
Kouroshfar, E.: Studying the effect of co-change dispersion on software quality. In: International Conference on Software Engineering (ICSE), pp. 1450–1452 (2013)
Lee, Y.S., Liang, B.S.: Measuring the coupling and cohesion of an object-oriented program based on information flow. In: ICSQ, pp. 81–90 (1995)
Marcus, A., Poshyvanyk, D., Ferenc, R.: Using the conceptual cohesion of classes for fault prediction in object-oriented systems. IEEE Trans. Softw. Eng. 34(2), 287–300 (2008)
Martin, R.C.: Design principles and design patterns. Object Mentor 1, 34 (2000)
Martin, R.C.: Agile Software Development: Principles, Patterns, and Practices. Prentice-Hall, Upper Saddle River (2002)
Meneely, A., Williams, L., Snipes, W., Osborne, J.: Predicting failures with developer networks and social network analysis. In: Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of software engineering. ACM, pp. 13–23 (2008)
Menzies, T., Greenwald, J., Frank, A.: Data mining static code attributes to learn defect predictors. IEEE Trans. Softw. Eng. 33(1), 2–13 (2007)
Misic, V.B.: Cohesion is structural, coherence is functional: different views, different measures. In: METRICS, pp. 135–144 (2001)
Morris, K.: Metrics for object-oriented software development environments. Master’s thesis, Sloan School of Management MIT (1989)
Nagappan, N., Ball, T., Zeller, A.: Mining metrics to predict component failures. In: ICSE, pp. 452–461 (2006)
Patel, S., Chu, W., Baxter, R.: A measure for composite module cohesion. In: ICSE, pp. 38–48 (1992)
Peters, F., Menzies, T., Gong, L., Zhang, H.: Balancing privacy and utility in crosscompany defect prediction. IEEE Trans. Softw. Eng. 39(8), 1054–1068 (2013)
Ponisio, L.: Exploiting client usage to manage program modularity. Ph.D. thesis, University of Berne (2006)
Ponisio, L., Nierstrasz, O.: Using contextual information to assess package cohesion. Technical Report IAM-06-002, Institute of Applied Mathematics and Computer Sciences, University of Berne (2006)
Praditwong, K., Harman, M., Yao, X.: Software module clustering as a multi-objective search problem. IEEE Trans. Softw. Eng. 37(2), 264–282 (2011)
Rahman, F., Devanbu, P.: How and why, process metrics are better. In: ICSE, pp. 432–441 (2013)
Rahman, F., Posnett, D., Devanbu, P.: Recalling the imprecision of cross-project defect prediction. In : Proceeding of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering, p. 61 (2012)
Rahman, F., Posnett, D., Devanbu, P.: Recalling the imprecision of cross-project defect prediction. In: FSE, p. 61 (2012)
Reinard, J.C.: Communication Research Statistics. Sage Publications Inc, Thousand Oaks (2006)
Sarkar, S., Kak, A.C., Rama, G.M.: Metrics for measuring the quality of modularization of large-scale object-oriented software. IEEE Trans. Softw. Eng. 34(5), 700–720 (2008)
Schein, A.I., Saul, L.K., Ungar, L.H.: A generalized linear model for principal component analysis of binary data. In: Proceedings of the 9th International Workshop on Artificial Intelligence and Statistics (2003)
Schroter, A., Zimmermann, T., Zeller, A.: Predicting component failures at design time. In: Proceedings of the 2006 ACM/IEEE international symposium on Empirical software engineering. ACM, pp. 18–27 (2006)
Shihab, E., Kamei, Y., Adams, B., Hassan, A.E.: Is lines of code a good measure of effort in effort-aware models. Inf. Softw. Technol. 55(11), 1981–1993 (2013)
Shin, Y., Meneely, A., Williams, L., Osborne, J.A.: Evaluating complexity, code churn, and developer activity metrics as indictors of software vulnerabilities. IEEE Trans. Softw. Eng. 37(6), 772–787 (2011)
Shin, Y., Williams, L.: Can traditional fault prediction models be used for vulnerability prediction? Empir. Softw. Eng. 18(1), 25–59 (2013)
Singh, Y., Kaur, A., Malhotra, R.: Empirical validation of object-oriented metrics for predicting fault proneness models. Softw. Qual. J. 18(1), 3–35 (2010)
Tan, M., Tan, L., Dara, S., Mayuex, C.: Online defect prediction for imbalanced data. In: ICSE-SEIP (2015)
Turhan, B., Tosun, A.: Misirli, A. Bener, Empirical evaluation of the effects of mixed project data on learning defect predictors. Inf. Softw. Technol. 55(6), 1101–1118 (2013)
Vernazza, T., Granatella, G., Succi, G., Benedicenti, L., Mintchev, M.: Defining metrics for software components. In: WMSCI, pp. 16–23 (2000)
Weyuker, E.J., Ostrand, T.J., Bell, R.M.: Using developer information as a factor for fault prediction. In: Proceedings of the 3rd International Worksho Data Mining Techniques p on Predictor Models in Software Engineering, p. 8 (2007)
Xu, B., Chen, Z., Zhao, J.: Measuring cohesion of packages in Ada95. SIGAda, pp. 62–67 (2003)
Yang, Y., Zhou, Y., Lu, H., Chen, L., Chen, Z., Xu, B., Leung, H., Zhang, Z.: Are slice-based cohesion metrics actually useful in effort-aware post-release fault-proneness prediction? An empirical study. IEEE Trans. Softw. Eng. 41(4), 331–357 (2015)
Zhang, H., Cheung, S.C.: A cost-effectiveness criterion for applying software defect prediction models. In: ESEC/FSE, pp. 643–646 (2013)
Zhou, T., Xu, B., Shi, L., Zhou, Y., Chen, L.: Measuring package cohesion based on context. In: Proceedings of the IEEE international workshop on semantic computing and systems, pp. 127–132 (2008)
Zhou, T., Xu, B., Shi, L., Zhou, Y.: Measuring package cohesion based on client usages. J. Softw. 20(2), 256–270 (2009)
Zhou, Y., Xu, B., Leung, H., Chen, L.: An in-depth study of the potentially confounding effect of class size in fault prediction. ACM Trans. Softw. Eng. Methodol. 23(1), 10 (2014)
Zhou, Y., Leung, H.: Empirical analysis of object-oriented design metrics for predicting high and low severity faults. IEEE Trans. Softw. Eng. 32(10), 771–789 (2006)
Zimmermann, T., Nagappan, N.: Predicting defects with program dependencies. In: Proceedings of the 3rd International Symposium on Empirical Software Engineering and Measurement. IEEE Computer Society pp. 435–438 (2009)
Zimmermann, T., Premraj, R., Zeller, A.: Predicting defects for eclipse. In: Proceedings of the International Workshop on Predictor Models in Software Engineering. IEEE Computer Society, p. 9 (2007)
Acknowledgments
The work in this paper is supported by the National Key Basic Research and Development Program of China (2014CB340702), the National Natural Science Foundation of China (91318301, 61432001, 61321491, 61300051, 61272082), and the National Natural Science Foundation of Jiangsu Province (BK20130014)
Author information
Authors and Affiliations
Corresponding author
Appendices
Appendix 1: Principal component analysis results
Tables 15, 16, 17, 18, 19, 20, 21, 22, 23, and 24 show the rotated components from PCA for the six data sets. The second to fourth rows in the tables show the eigenvalue, the percentage of variance explained by each PC, and the cumulative percentage of variances. The remaining rows, starting with the fifth row, show the loading of each metric in each PC, which indicates the degree of correlation with the PC. In each row, the loading with the largest absolute value is shown in bold. This enables us to observe which metrics capture the same property and hence belong to the same PC.
Appendix 2: The multivariate logistic regression models
Table 25 summarizes the “N”, “C”, and “C+N” models for each data set. The first column indicates the data set and the second column is the type of the multivariate logistic regression model. The third column shows the R square. The remaining columns, starting from the fourth column, show the selected independent variables. Below each variable name, we present the corresponding coefficient and p value.
Appendix 3: Comparison between cohesion metrics and traditional metrics
Table 26 describes the most commonly used twenty traditional metrics, including SLOC, structural complexity, halstead, and process churn metrics. All these metrics are collected in package level in our study. We build two types of multivariate logistic regression models: (1) “Cohesion” model (using 16 package cohesion metrics introduced in Sect. 2); (2) “Tradition” model (using 20 traditional metrics in Table 26); Based on ten Java systems introduced in Sect. 3.6, we perform 30 times threefold cross-validation. Fig. 6 shows the fault prediction performance for each model in both ranking and classification scenarios. Each sub-figure corresponds to one distinct data set, and reveals the mean value in terms of CE at \(\pi =0.2\), ER-BPP, ER-BCE, and ER-MFM.
From Fig. 6, we can see that, cohesion metrics outperform traditional metrics in six out of the ten systems (i.e., Cxf 2.1, Flume 1.4, Lucene 2.4, Wicket 6.6, Jdtcore 3.4, and Eclipse 2.0). While in Hive 0.12.0, cohesion metrics perform a little worse than traditional metrics. In Camel 2.9.0, cohesion metrics performs worse in terms of CE at \(\pi =0.2\) and ER-BPP, but performs better in terms of ER-BCE and ER_MFM. For the other two systems Hbase0.96.0 and Eclipse3.0, cohesion metrics and traditional metrics have similar performance for fault prediction. Overall, the results show that cohesion metrics have a better performance than traditional metrics in most of the studied systems in both ranking and classification scenarios.
Rights and permissions
About this article
Cite this article
Zhao, Y., Yang, Y., Lu, H. et al. Understanding the value of considering client usage context in package cohesion for fault-proneness prediction. Autom Softw Eng 24, 393–453 (2017). https://doi.org/10.1007/s10515-016-0198-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10515-016-0198-6