ABSTRACT
Increasing the number of persons working on quality assurance (QA) tasks, e.g., reviews and testing, increases the number of defects detected -- but it also increases the total effort unless effort is controlled with fixed effort budgets. Our research investigates how QA tasks should be configured regarding two parameters, i.e., time and number of people. We define an optimization problem to answer this question. As a core element of the optimization problem we discuss and describe how defect detection probability should be modeled as a function of time. We apply the formulas used in the definition of the optimization problem to empirical defect data of an experiment previously conducted with university students. The results show that the optimal choice of the number of persons depends on the actual defect detection probabilities of the individual defects over time, but also on the size of the effort budget. Future work will focus on generalizing the optimization problem to a larger set of parameters, including not only task time and number of persons but also experience and knowledge of the personnel involved, and methods and tools applied when performing a QA task.
- E. Raymond, "The cathedral and the bazaar," Knowledge, Technology & Policy, vol. 12, no. 3, 1999, pp. 23--49.Google ScholarCross Ref
- C. Jones, "Software defect-removal efficiency," Computer, vol. 29, no. 4, 1996, pp. 94--95. Google ScholarDigital Library
- S. Biffl and M. Halling, "Investigating the defect detection effectiveness and cost benefit of nominal inspection teams," Software Engineering, IEEE Transactions on, vol. 29, no. 5, 2003, pp. 385--397. Google ScholarDigital Library
- M.V. Mäntylä and J. Itkonen, "The Effect of Adding People and Restricting Time in Software Testing -- Power of the Crowds," Submitted to a Journal, Under review,Google Scholar
- S. Biffl and W. Gutjahr, "Influence of team size and defect detection technique on inspection effectiveness," Software Metrics Symposium, 2001. METRICS 2001. Proceedings. Seventh International, 2001, pp. 63--75. Google ScholarDigital Library
- C.F. Kemerer and M.C. Paulk, "The impact of design and code reviews on software quality: An empirical study based on PSP data," IEEE Trans.Software Eng., vol. 35, no. 4, 2009, pp. 534--550. Google ScholarDigital Library
- K. Petersen, K. Rönkkö and C. Wohlin, "The impact of time controlled reading on software inspection effectiveness and efficiency: a controlled experiment," Proceedings of the Second ACM-IEEE international symposium on Empirical software engineering and measurement, 2008, pp. 139--148. Google ScholarDigital Library
- G. Ruhe and M.O. Saliu, "The art and science of software release planning," Software, IEEE, vol. 22, no. 6, 2005, pp. 47--53. Google ScholarDigital Library
Index Terms
- How many individuals to use in a QA task with fixed total effort?
Recommendations
Is 'Additional' Effort Always Negative?: Understanding Discretionary Work in Interpersonal Communications
CSCW'15 Companion: Proceedings of the 18th ACM Conference Companion on Computer Supported Cooperative Work & Social ComputingThis paper describes a position on the meaning of effort in interaction design, particularly for communication systems. We make use of interview data to illustrate our ongoing research on how people invest discretionary effort when using communication ...
Detecting architecturally-relevant code anomalies: a case study of effectiveness and effort
SAC '14: Proceedings of the 29th Annual ACM Symposium on Applied ComputingCode anomalies are structural problems in the program. Even though they might represent symptoms of architecture degradation, several code anomalies do not contribute to this process. Source code inspection by developers might not support time-effective ...
On Obtaining Effort Based Judgements for Information Retrieval
WSDM '16: Proceedings of the Ninth ACM International Conference on Web Search and Data MiningDocument relevance has been the primary focus in the design, optimization and evaluation of retrieval systems. Traditional testcollections are constructed by asking judges the relevance grade for a document with respect to an input query. Recent work of ...
Comments