skip to main content
10.1145/2372251.2372307acmconferencesArticle/Chapter ViewAbstractPublication PagesesemConference Proceedingsconference-collections
research-article

How many individuals to use in a QA task with fixed total effort?

Published:19 September 2012Publication History

ABSTRACT

Increasing the number of persons working on quality assurance (QA) tasks, e.g., reviews and testing, increases the number of defects detected -- but it also increases the total effort unless effort is controlled with fixed effort budgets. Our research investigates how QA tasks should be configured regarding two parameters, i.e., time and number of people. We define an optimization problem to answer this question. As a core element of the optimization problem we discuss and describe how defect detection probability should be modeled as a function of time. We apply the formulas used in the definition of the optimization problem to empirical defect data of an experiment previously conducted with university students. The results show that the optimal choice of the number of persons depends on the actual defect detection probabilities of the individual defects over time, but also on the size of the effort budget. Future work will focus on generalizing the optimization problem to a larger set of parameters, including not only task time and number of persons but also experience and knowledge of the personnel involved, and methods and tools applied when performing a QA task.

References

  1. E. Raymond, "The cathedral and the bazaar," Knowledge, Technology & Policy, vol. 12, no. 3, 1999, pp. 23--49.Google ScholarGoogle ScholarCross RefCross Ref
  2. C. Jones, "Software defect-removal efficiency," Computer, vol. 29, no. 4, 1996, pp. 94--95. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. S. Biffl and M. Halling, "Investigating the defect detection effectiveness and cost benefit of nominal inspection teams," Software Engineering, IEEE Transactions on, vol. 29, no. 5, 2003, pp. 385--397. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. M.V. Mäntylä and J. Itkonen, "The Effect of Adding People and Restricting Time in Software Testing -- Power of the Crowds," Submitted to a Journal, Under review,Google ScholarGoogle Scholar
  5. S. Biffl and W. Gutjahr, "Influence of team size and defect detection technique on inspection effectiveness," Software Metrics Symposium, 2001. METRICS 2001. Proceedings. Seventh International, 2001, pp. 63--75. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. C.F. Kemerer and M.C. Paulk, "The impact of design and code reviews on software quality: An empirical study based on PSP data," IEEE Trans.Software Eng., vol. 35, no. 4, 2009, pp. 534--550. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. K. Petersen, K. Rönkkö and C. Wohlin, "The impact of time controlled reading on software inspection effectiveness and efficiency: a controlled experiment," Proceedings of the Second ACM-IEEE international symposium on Empirical software engineering and measurement, 2008, pp. 139--148. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. G. Ruhe and M.O. Saliu, "The art and science of software release planning," Software, IEEE, vol. 22, no. 6, 2005, pp. 47--53. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. How many individuals to use in a QA task with fixed total effort?

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in
    • Published in

      cover image ACM Conferences
      ESEM '12: Proceedings of the ACM-IEEE international symposium on Empirical software engineering and measurement
      September 2012
      338 pages
      ISBN:9781450310567
      DOI:10.1145/2372251

      Copyright © 2012 ACM

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 19 September 2012

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article

      Acceptance Rates

      Overall Acceptance Rate130of594submissions,22%

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader