skip to main content
10.1145/3345629.3345638acmotherconferencesArticle/Chapter ViewAbstractPublication PagesicseConference Proceedingsconference-collections
short-paper

Applying Cross Project Defect Prediction Approaches to Cross-Company Effort Estimation

Authors Info & Claims
Published:18 September 2019Publication History

ABSTRACT

BACKGROUND: Prediction systems in software engineering often suffer from the shortage of suitable data within a project. A promising solution is transfer learning that utilizes data from outside the project. Many transfer learning approaches have been proposed for defect prediction known as cross-project defect prediction (CPDP). In contrast, a few approaches have been proposed for software effort estimation known as cross-company software effort estimation (CCSEE). Both CCSEE and CPDP are engaged in a similar problem, and a few CPDP approaches are applicable as CCSEE in actual. It is thus beneficial for improving CCSEE performance to examine how well CPDP approaches can perform as CCSEE approaches. AIMS: To explore how well CPDP approaches work as CCSEE approaches. METHOD: An empirical experiment was conducted for evaluating the performance of CPDP approaches in CCSEE. We examined 7 CPDP approaches which were selected due to the easiness of application. Those approaches were applied to 8 data sets, each of which consists of a few subsets from different domains. The estimation results were evaluated with a common performance measure called SA. RESULTS: there were several CPDP approaches which could improve the estimation accuracy though the degree of improvement was not large. CONCLUSIONS: A straight forward application of selected CPDP approaches did not bring a clear effect. CCSEE may need specific transfer learning approaches for more improvement.

References

  1. Sousuke Amasaki, Kazuya Kawata, and Tomoyuki Yokogawa. 2015. Improving Cross-Project Defect Prediction Methods with Data Simplification. In Proc. of SEAA '15. IEEE, 96--103. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Camargo-Cruz Ana Erika and Koichiro Ochimizu. 2009. Towards logistic regression models for predicting fault-prone code across software projects. In Proc. of ESEM '09. IEEE, 460--463. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Steffen Herbold. 2016. CrossPare: A tool for benchmarking cross-project defect predictions. In Proc. of 30th IEEE/ACM International Conference on Automated Software Engineering Workshops (ASEW). IEEE, 90--95. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Steffen Herbold, Alexander Trautsch, and Jens Grabowski. 2018. A Comparative Study to Benchmark Cross-Project Defect Prediction Approaches. IEEE Transactions on Software Engineering 44, 9 (2018), 811--833.Google ScholarGoogle ScholarCross RefCross Ref
  5. Kazuya Kawata, Sousuke Amasaki, and Tomoyuki Yokogawa. 2015. Improving relevancy filter methods for cross-project defect prediction. In Proc. of ACIT-CSI '15. 2--7. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Barbara A. Kitchenham, Emilia Mendes, and Guilherme Horta Travassos. 2007. Cross versus Within-Company Cost Estimation Studies: A Systematic Review. IEEE Transactions on Software Engineering 33, 5 (2007), 316--329. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Ekrem Kocaguneli, Bojan Cukic, Tim Menzies, and Huihua Lu. 2013. Building a second opinion: learning cross-company data. In Proc. of PROMISE. ACM, 1--10. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Ekrem Kocaguneli and Tim Menzies. 2011. How to Find Relevant Data for Effort Estimation?. In Proc. of International Symposium on Empirical Software Engineering and Measurement (ESEM). IEEE, 255--264. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Ekrem Kocaguneli, Tim Menzies, and Emilia Mendes. 2015. Transfer learning in effort estimation. Empirical Software Engineering 20, 3 (2015), 813--843. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Ying Ma, Guangchun Luo, Xue Zeng, and Aiguo Chen. 2012. Transfer learning for cross-company software defect prediction. Information and Software Technology 54, 3 (2012), 248--256. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Emilia Mendes and Chris Lokan. 2009. Investigating the use of chronological splitting to compare software cross-company and single-company effort predictions: a replicated study. In Proc. of EASE. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Tim Menzies, A. Butcher, A. Marcus, Thomas Zimmermann, and D. Cok. 2011. Local versus global models for effort estimation and defect prediction. In Proc. of ASE '11. IEEE, 343--351. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Leandro L. Minku. 2016. On the Terms Within- and Cross-Company in Software Effort Estimation. In Proc. of PROMISE. ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Leandro L. Minku. 2019. A novel online supervised hyperparameter tuning procedure applied to cross-company software effort estimation. Empirical Software Engineering (2019), 1--52.Google ScholarGoogle Scholar
  15. Leandro L. Minku and Siqing Hou. 2017. Clustering Dycom. In Proc. of PROMISE. 12--21.Google ScholarGoogle Scholar
  16. Leandro L Minku and Xin Yao. 2012. Can Cross-company Data Improve Performance in Software Effort Estimation?. In Proc. of PROMISE. 69--78. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Leandro L Minku and Xin Yao. 2014. How to make best use of cross-company data in software effort estimation?. In Proc. of ICSE. 446--456. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Leandro L Minku and Xin Yao. 2017. Which models of the past are relevant to the present? A software effort estimation approach to exploiting useful past models. Automated Software Engineering 24, 3 (2017), 499--542. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Martin J Shepperd and Steve MacDonell. 2012. Evaluating prediction systems in software project estimation. Information and Software Technology 54, 8 (2012), 820--827. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Shensi Tong, Qing He, Yuting Chen, Ye Yang, and Beijun Shen. 2016. Heterogeneous Cross-Company Effort Estimation through Transfer Learning. In Proc. of Asia-Pacific Software Engineering Conference (APSEC). IEEE, 169--176.Google ScholarGoogle ScholarCross RefCross Ref
  21. Burak Turhan and Emilia Mendes. 2014. A Comparison of Cross-Versus Single-Company Effort Prediction Models for Web Projects. In Proc. of SEAA. IEEE, 285--292. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Burak Turhan, Tim Menzies, Ayşe B Bener, and Justin Di Stefano. 2009. On the relative value of cross-company and within-company data for defect prediction. Empirical Software Engineering 14, 5 (2009), 540--578. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Shinya Watanabe, Haruhiko Kaiya, and Kenji Kaijiri. 2008. Adapting a fault prediction model to allow inter languagereuse. In Proc. of PROMISE '08. ACM, New York, New York, USA, 19--24. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Applying Cross Project Defect Prediction Approaches to Cross-Company Effort Estimation

      Recommendations

      Reviews

      Barrett Hazeltine

      The problem is estimating the effort required to complete a software project. The problem is difficult because of the shortage of data within the project, so a promising strategy is to use data from other projects. Work has been done on predicting defects on a given project using data from other projects. The research reported here describes the results from using these outside data approaches, which predict defects, to estimate effort. Seven approaches are tested on eight datasets. Performance is measured using "standardized accuracy," that is, 1 - (the ratio of the mean error using the approach divided by the mean error resulting from random guessing). Several of these approaches appear promising, but none are clearly superior. Also, for each dataset, the most effective approach is determined. Again, no approach shows clear superiority. Future work would include other approaches and ways of modifying the defect prediction algorithms to estimate error. The paper is clear and the analysis is valid; but, clearly, this is work in progress.

      Access critical reviews of Computing literature here

      Become a reviewer for Computing Reviews.

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in
      • Published in

        cover image ACM Other conferences
        PROMISE'19: Proceedings of the Fifteenth International Conference on Predictive Models and Data Analytics in Software Engineering
        September 2019
        103 pages
        ISBN:9781450372336
        DOI:10.1145/3345629

        Copyright © 2019 ACM

        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 18 September 2019

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • short-paper
        • Research
        • Refereed limited

        Acceptance Rates

        Overall Acceptance Rate64of125submissions,51%

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader