Muhammad Atif Javed
ABSTRACT
Context: None of the published empirical studies on software traceability have comparatively examined the support for differently sized systems. Objective: This paper reports on two controlled experiments performed with two Enterprise Service Bus (ESB) systems that are comparable in terms of support features and system structure, but are different in their size, in particular, UltraESB Version 2.3.0 and PetalsESB Version 4.2.0, to investigate the effects of system size on the use of traceability links. Method: We conducted two controlled experiments in which the same impact evaluation activities were performed and measured how the control groups (provided with no traceability information) and the experiment groups (provided with traceability information) performed these activities in terms of the quantity and quality of retrieved elements. Results: Our findings show that the 133.71% larger size of one of ESBs does not have a significant influence on the quantity and quality of retrieved elements in the experiment groups. In the control groups, in contrast, this increase in system size significantly increases the quantity of incorrect elements and reduces the overall quality of elements retrieved, while no conclusive evidence concerning the quantity of missing elements was found. Conclusion: It is concluded that traceability is more important in larger software systems.
- B. Boehm, H. D. Rombach, and M. V. Zelkowitz. Foundations of Empirical Software Engineering: The Legacy of Victor R. Basili. Springer-Verlag New York, Inc., Secaucus, NJ, USA, 2005. Google Scholar
Digital Library
- T. Cook and D. Campbell. Quasi-experimentation: design & analysis issues for field settings. Rand McNally College, 1979.Google Scholar
- B. Cornelissen, A. Zaidman, and A. van Deursen. A controlled experiment for program comprehension through trace visualization. IEEE Trans. Softw. Eng., 37(3): 341--355, May 2011. Google ScholarDigital Library
- D. Cuddeback, A. Dekhtyar, and J. Hayes. Automated requirements traceability: The study of human analysts. In Proceedings of the 2010 18th IEEE International Requirements Engineering Conference, RE '10, pages 231--240. IEEE Computer Society, 2010. Google ScholarDigital Library
- A. De Lucia, R. Oliveto, and G. Tortora. Assessing ir-based traceability recovery tools through controlled experiments. Empirical Softw. Engg., 14(1): 57--92, Feb. 2009. Google ScholarDigital Library
- A. De Lucia, R. Oliveto, F. Zurolo, and M. Di Penta. Improving comprehensibility of source code via traceability information: A controlled experiment. In Proceedings of the 14th IEEE International Conference on Program Comprehension, ICPC '06, pages 317--326. IEEE Computer Society, 2006. Google ScholarDigital Library
- A. Dekhtyar, O. Dekhtyar, J. Holden, J. Hayes, D. Cuddeback, and W.-K. Kong. On human analyst performance in assisted requirements tracing: Statistical analysis. In Proceedings of the 2011 19th IEEE International Requirements Engineering Conference, RE '11, pages 111--120. IEEE Computer Society, Aug 2011. Google ScholarDigital Library
- O. Gotel and A. Finkelstein. An analysis of the requirements traceability problem. In Proceedings of the 1994 First IEEE International Requirements Engineering Conference, pages 94--101. IEEE Computer Society, Apr 1994.Google ScholarCross Ref
- J. Hayes, A. Dekhtyar, S. Sundaram, E. Holbrook, S. Vadlamudi, and A. April. Requirements tracing on target (retro): improving software maintenance through traceability recovery. Innovations in Systems and Software Engineering, 3(3): 193--202, 2007.Google ScholarCross Ref
- M. A. Javed and U. Zdun. The supportive effect of traceability links in architecture-level software understanding: Two controlled experiments. In Proceedings of the 11th Working IEEE/IFIP Conference on Software Architecture, WICSA 2014, pages 215--224. IEEE, 2014. Google ScholarDigital Library
- M. A. Javed and U. Zdun. A systematic literature review of traceability approaches between software architecture and source code. In Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering, EASE '14, pages 16:1--16:10. ACM, 2014. Google ScholarDigital Library
- M. A. Javed and U. Zdun. The supportive effect of traceability links in change impact analysis for evolving architectures -- two controlled experiments. In 14th International Conference on Software Reuse. Springer, January 2015.Google Scholar
- A. Jedlitschka, D. Hamann, T. Göhlert, and A. Schröder. Adapting profes for use in an agile process: An industry experience report. In PROFES, pages 502--516, 2005. Google ScholarDigital Library
- B. A. Kitchenham, S. L. Pfleeger, L. M. Pickard, P. W. Jones, D. C. Hoaglin, K. El Emam, and J. Rosenberg. Preliminary guidelines for empirical research in software engineering. Software Engineering, IEEE Transactions on, 28(8): 721--734, Aug. 2002. Google ScholarDigital Library
- P. Mader and A. Egyed. Assessing the effect of requirements traceability for software maintenance. In Proceedings of the 2012 28th IEEE International Conference on Software Maintenance, ICSM '12, pages 171--180, 2012. Google ScholarDigital Library
- H. Mann and D. Whitney. On a Test of Whether One of Two Random Variables is Stochastically Larger Than the Other. Institute of Mathematical Statistics, 1947.Google ScholarCross Ref
- T. I. of Electrical and E. Engineers. Ieee standard glossary of software engineering terminology. IEEE Standard, September 1990.Google Scholar
- B. Ramesh and M. Jarke. Toward reference models for requirements traceability. IEEE Trans. Softw. Eng., 27(1): 58--93, Jan. 2001. Google ScholarDigital Library
- M. Shahin, P. Liang, and Z. Li. Architectural design decision visualization for architecture design: preliminary results of a controlled experiment. In Proceedings of the 5th European Conference on Software Architecture: Companion Volume, ECSA '11, pages 2:1--2:8. ACM, 2011. Google ScholarDigital Library
- S. S. Shapiro and M. B. Wilk. An analysis of variance test for normality (complete samples). Biometrika, 52(3/4): pp. 591--611, 1965.Google ScholarCross Ref
- C. Wohlin. Experimentation in Software Engineering: An Introduction: An Introduction. The Kluwer International Series in Software Engineering. Kluwer Academic, 2000. Google ScholarDigital Library
Index Terms
- On the effects of traceability links in differently sized software systems
Recommendations
Cost-Effective Traceability Links for Architecture-Level Software Understanding: A Controlled Experiment
ASWEC ' 15 Vol. II: Proceedings of the ASWEC 2015 24th Australasian Software Engineering ConferenceAn important architectural challenge is to recover traceability links between the software architecture and artifacts produced in the other activities of the development process, such as requirements, detailed design, architectural knowledge, and ...
Do developers benefit from requirements traceability when evolving and maintaining a software system?
Software traceability is a required component of many software development processes. Advocates of requirements traceability cite advantages like easier program comprehension and support for software maintenance (i.e., software change). However, despite ...
The Supportive Effect of Traceability Links in Architecture-Level Software Understanding: Two Controlled Experiments
WICSA '14: Proceedings of the 2014 IEEE/IFIP Conference on Software ArchitectureThe advocates of architecture traceability approaches regularly cite advantages like easier understanding of architectural designs and support for software quality control and maintenance. However, the lack of published empirical data on the usefulness ...
Comments