Abstract
[Context and motivation] Trace matrices are lynch pins for the development of mission- and safety-critical software systems and are useful for all software systems, yet automated methods for recovering trace links are far from perfect. This limitation makes the job of human analysts who must vet recovered trace links more difficult. [Question/Problem] Earlier studies suggested that certain analyst behaviors when performing trace recovery tasks lead to decreased accuracy of recovered trace relationships. We propose a three-step experimental study to: (a) determine if there really are behaviors that lead to errors of judgment for analysts, (b) enhance the requirements tracing software to curtail such behaviors, and (c) determine if curtailing such behaviors results in increased accuracy. [Principal ideas/results] We report on a preliminary study we undertook in which we modified the user interface of RETRO.NET to curtail two behaviors indicated by the earlier work. We report on observed results. [Contributions] We describe and discuss a major study of potentially unwanted analyst behaviors and present results of a preliminary study toward determining if curbing these behaviors with enhancements to tracing software leads to fewer human errors.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
As reported earlier [2], the accuracy of the starting RTM affected the changes in precision, recall, and f2-measure, and the final precision in statistically significant ways, but did not affect final recall or final f2-measure in statistically significant ways.
- 2.
Originally, the control and the experimental groups were of the same size, but we had a significantly larger number of non-completions in the control group.
References
Cuddeback, D., Dekhtyar, A., Hayes, J.H.: Automated requirements traceability: the study of human analysts. In: Proceedings of IEEE International Conference on requirements Engineering (RE), Sydney, Australia, pp. 231–240, September 2010
Dekhtyar, A., Dekhtyar, O., Holden, J., Hayes, J.H., Cuddeback, D., Kong, W.-K.: On human analyst performance in assisted requirements tracing: statistical analysis. In: The Proceedings of IEEE International Conference on Requirements Engineering (RE) 2011, Trento, Italy (2011)
Huang, J., Gotel, O., Zisman, A.: Software and Systems Traceability. Springer, London (2014). https://doi.org/10.1007/978-1-4471-2239-5
Borg, M., Runeson, P., Ardö, A.: Recovering from a decade: a systematic mapping of information retrieval approaches to software traceability. Empirical Softw. Eng. 19(6), 1565–1616 (2014)
Hayes, J.H., Dekhtyar, A., Sundaram, S., Holbrook, A., Vadlamudi, S., April, A.: REquirements TRacing On target (RETRO): improving software maintenance through traceability recovery. Innov. Syst. Softw. Eng. NASA J. (ISSE) 3(3), 193–202 (2007)
Kong, W.-K., Hayes, J., Dekhtyar, A., Holden, J.: How do we trace requirements? An initial study of analyst behavior in trace validation tasks. In: Proceedings of the 4th International Workshop on Cooperative and Human Aspects of Software Engineering, (CHASE 2011) (2011)
Hayes, J., Dekhtyar, A., Sundaram, S.: Advancing candidate link generation for requirements tracing: the study of methods. IEEE Trans. Softw. Eng. 32(1), 4–19 (2006)
Kahneman, D.: Thinking, Fast and Slow. Farrar, Straus, New York (2011)
Acknowledgment
We thank Dr. Dan Berry for insightful comments and suggestions on prior versions that resulted in a greatly improved paper. We thank all participants from upper division software engineering classes who took their time to participate in our study. We thank NASA and NSF as prior grants funded the development of RETRO.NET. We thank Jody Larsen, the developer of RETRO.NET. We thank NSF for partially funding this work under grants CCF-1511117 and CNS- 1642134.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Chitre, B., Hayes, J.H., Dekhtyar, A. (2018). Second-Guessing in Tracing Tasks Considered Harmful?. In: Kamsties, E., Horkoff, J., Dalpiaz, F. (eds) Requirements Engineering: Foundation for Software Quality. REFSQ 2018. Lecture Notes in Computer Science(), vol 10753. Springer, Cham. https://doi.org/10.1007/978-3-319-77243-1_6
Download citation
DOI: https://doi.org/10.1007/978-3-319-77243-1_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-77242-4
Online ISBN: 978-3-319-77243-1
eBook Packages: Computer ScienceComputer Science (R0)