Abstract
The requirements phase is the most critical phase of the software development life cycle. The quality of the requirements specification affects the overall quality of the subsequent phases and hence, the software product. An effective and efficient method to qualify the software requirements specification (SRS) is necessary to ensure the reliability and safety of software. In this paper, a requirements inspection method based on scenarios generated by model mutation (RIMSM) is proposed to detect defects in the functional requirements of a safety-critical system. The RIMSM method models software requirements using a High Level Extended Finite State Machine (HLEFSM). A method that executes the HLEFSM model is defined. The method uncovers the behaviors and generates the outputs of the system for a given scenario. To identify an adequate set of scenarios in which the model shall be executed, an analogue to mutation testing is defined which applies to the requirements phase. Twenty-one mutation operators are designed based on a taxonomy of defects defined for the requirements phase. Mutants of the HLEFSM model are generated using these operators. Further, an algorithm is developed to identify scenarios that can kill the mutants. The set of scenarios is considered to be adequate for detecting defects in the model when all mutants generated are killed. The HLEFSM model is then executed for the scenarios generated. The results of execution are used to detect defects in the model. A Requirements Inspection Tool based on Scenarios Generated by Model Mutation (RITSM) is developed to automate the application of the RIMSM method. The performance and usability of the RIMSM method are studied and demonstrated in an experiment by comparing the RIMSM method to the checklist-based reading method.
Similar content being viewed by others
References
Agrawal H, DeMillo RA, Hathaway B, Hsu W, Wynne H, Krauser EW, Martin RJ, Mathur AP, Spafford E. (1989) “Design of Mutant Operators for the C programming language.” Technical report SERC-TR-41-P, Software Engineering Research Center, Purdue University
Ali SW, Ahmed QA, Shafi I. (2018). “Process to Enhance the Quality of Software Requirement Specification Document.” 2018 International Conference on Engineering and Emerging Technologies
Alshazly AA, Elfatatry AM, Abougabal MS (2014) Detecting defects in software requirements specification. Alexandria Engineering Journal 53(3):513–527
Arndt SA, Alvarado R, Dittman B, Mott K, Wood R (2017) “NRC Technical Basis For Evaluation Of Its Position On Protection Against Common Cause Failure In Digital Systems Used In Nuclear Power Plants.” In Proceedings of 2017 NPIC-HMIT
Budd TA, Gopal AS (1985) Program testing by specification mutation. Comput Lang 10(1):63–73. https://doi.org/10.1016/0096-0551(85)90011-6
Budd TA, DeMillo RA, Lipton RJ, Sayward FG (1978) The Design of a Prototype Mutation System for program testing. Proceedings of the AFIPS National Computer Conference 74:623–627
Cisneros A, Rafael J, Garcia GDLR, and Fernandez-Y-Fernandez CA (2018) “Software Requirement Specification for the Automotive Sector: The Case of a Post-Collision Event Control System.” Proceedings of 5th International Conference in Software Engineering Research and Innovation, IEEE
Dalpiaz F, van der Schalk I, Brinkkemper S, Aydemir FB, Lucassen G (2019) Detecting terminological ambiguity in user stories: tool and experimentation. Inf Softw Technol 110:3–16. https://doi.org/10.1016/j.infsof.2018.12.007
Demillo RA, Lipton RJ, Sayward FG (1978) Hints on test data selection : help for the practicing programmer. Computer 11(4):33–41
Derenthal FE, Elks CR, Bakker T, and Fotouhi M (2017) “Virtualized Hardware Environments for Supporting Digital I&C Verification.” Proceedings of 2017 NPIC-HMIT, 1658–70
Fabbri SCPF, Maldonado JC, Sugeta T, Masiero PC (1999) Mutation Testing Applied to Validate Specifications Based on Statecharts. Proceedings of 10th International Symposium on Software Reliability Engineering, IEEE:210–219. https://doi.org/10.1109/ISSRE.1999.809326
Hamlet RG (1977) Testing programs with the aid of a compiler. IEEE Trans Softw Eng 4:279–290
He L, Carver J (2006) PBR vs. Checklist: A Replication in the N-Fold Inspection Context. Proceedings of the 2006 ACM/IEEE International Symposium on Empirical Software Engineering:95–104. https://doi.org/10.1145/1159733.1159750
Hierons RM, Merayo MG (2007) “Mutation testing from probabilistic finite state machines.” Proceedings of Testing: Academic and Industrial Conference Practice and Research Techniques, 141–50. https://doi.org/10.1109/TAIC.PART.2007.20
Hierons R, Merayo M (2009) Mutation testing from probabilistic and stochastic finite state machines. J Syst Softw 82(11):1804–1818. https://doi.org/10.1016/j.jss.2009.06.030
IEEE (2018) Systems and software engineering-life cycle processes: requirements engineering. ISO/IEC/IEEE 29148:2018
Jia Y, Harman M (2011) An analysis and survey of the development of mutation testing. IEEE Trans Softw Eng 37(5):649–678
Kim SW, Clark J, McDermid J (1999) “The rigorous generation of Java mutation operators using HAZOP.” Proceedings of the 12th international conference on software and systems engineering and their applications ICSSEA99, 1–20
King KN, Jefferson Offutt A (1991) A Fortran language system for mutation-based software testing. Software: Practice and Experience 21(7):685–718. https://doi.org/10.1002/spe.4380210704
Kirby J (1987) “Software requirements for an automobile cruise control and monitoring system.” NRL-SCR
Laitenberger O (2002) A survey of software inspection technologies. Handbook Softw Eng Knowl Eng 2:517–555
Lanubile F, Visaggio G (2000) “Evaluating Defect Detection Techniques for Software Requirements Inspections.” International Software Engineering Research Network Report No. 00–08
LeBlanc D, Bezzina D, Tiernan T, Gabel M, Pomerleau D (2008) “Functional requirements for integrated vehicle-based safety system (IVBSS).” The University of Michigan Transportation Research Institute, Visteon Corporation and Cognex Corporation
Lee S-M, Chao A (1994) Estimating population size via sample coverage for closed capture-recapture models. Biometrics 50(1):88–97
Li B, Smidts C, Huang F, Quinn T (2018) “A Quantification Framework for Software Safety in the Requirements Phase: Application to Nuclear Power Plants.” In Proceedings of 2018 Annual Reliability and Maintainability Symposium (RAMS)
Li B, Smidts CS (2017) “Extension of Mutation Testing for the Requirements and Design Faults.” In Proceedings of 2017 NPIC-HMIT
Li X, and Gupta J (2013) “ARPS: An Automated Reliability Prediction System Tool for Safety Critical Software.” PSA, 22–27
Li X, Mutha C, Smidts CS (2016) An automated software reliability prediction system for safety critical software. Empir Softw Eng 21(6):2413–2455. https://doi.org/10.1007/s10664-015-9412-6
Lipton RJ (1971) “Fault diagnosis of computer programs.” Student Report, Carnegie Mellon University
Lorber F, Larsen KG, Nielsen B (2018) “Model-based mutation testing of real-time systems via model checking.” Proceedings of 2018 IEEE 11th international conference on software testing, verification and validation workshops (ICSTW), 59–68. https://doi.org/10.1109/ICSTW.2018.00029
Ma YS, Offutt J, Kwon YR (2005) MuJava: an automated class mutation system. Software Testing Verification and Reliability 15(2):97–133. https://doi.org/10.1002/stvr.308
McHugh ML (2012b) Interrater reliability: the kappa statistic. Biochemia Medica: Biochemia Medica 22(3)
De Moura L, Bjørner N, Bjorner N (2008) Z3: an efficient SMT solver. In: International conference on tools and algorithms for the construction and analysis of systems, pp 337–340. https://doi.org/10.1007/978-3-540-78800-3_24
NCSS Statistical Software (2020) “Mann-Whitney U or Wilcoxon Rank-Sum Tests.” NCSS, LLC. Kaysville, Utah, USA, 503–15
Okun V. 2004. “Specification mutation for test generation and analysis.” Dissertation, University of Maryland Baltimore County
Pinto Ferraz Fabbri SC, Delamaro ME, Maldonado JC, Masiero PC (1994) Mutation Analysis Testing for Finite State Machines. Proceedings of 1994 IEEE International Symposium on Software Reliability Engineering:220–229. https://doi.org/10.1109/ISSRE.1994.341378
Porter AA, Votta LG, Basili VR (2002) Comparing detection methods for software requirements inspections: a replicated experiment. IEEE Trans Softw Eng 21(6):563–575. https://doi.org/10.1109/32.391380
Santoso S, Sudarno R, Maerani JS, Cahyono A (2019) Software requirement analysis for digital based reactor protection system of RDE design. J Phys Conf Ser 1198(2). https://doi.org/10.1088/1742-6596/1198/2/022046
Sawilowsky SS (2009) New Effect Size Rules of Thumb. Journal of Modern Applied Statistical Methods 8(2):597–599. https://doi.org/10.22237/jmasm/1257035100
Staron M, Kuzniarz L, Thurn C (2005) An empirical assessment of using stereotypes to improve Reading techniques in software inspections. Proceedings: international conference on software engineering:63–69. https://doi.org/10.1145/1083292.1083308
Strobl F, Wisspeintner A. 1999. “Specication of an elevator control system.” Technische Universitat Munchen
Sugeta T, Maldonado JC, Eric Wong W (2004) Mutation testing applied to validate SDL specifications. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 2978:193–208. https://doi.org/10.1007/978-3-540-24704-3_13
Sullivan GM, Feinn R (2012) Using effect size—or why the P value is not enough. Journal of Graduate Medical Education 4(3):279–282. https://doi.org/10.4300/jgme-d-12-00156.1
Yang XM. 2007. “Towards a self-evolving software defect detection process.” Dissertation, University of Saskatchewan
Zhang H, Yue T, Ali S, Liu C (2016) Towards mutation analysis for use cases. Proceedings - 19th ACM/IEEE international conference on model driven engineering languages and systems, MODELS 2016:363–373. https://doi.org/10.1145/2976767.2976784
Zhang Z, Li K, Yuan L, Guanhua Y (2019) Mutation Model-Based Test Case Generation of Chinese Train Control System with Automatic Train Operation Function. Proceedings of 2018 International Conference on Intelligent Rail Transportation (ICIRT). https://doi.org/10.1109/ICIRT.2018.8641582
Acknowledgments
This material is based upon work supported by the U.S. Department of Energy, Office of Nuclear Energy, Nuclear Energy Enabling Technologies (NEET) program under the Award Number DE-NE0008434. This research is performed using funding received from the DOE Office of Nuclear Energy’s Nuclear Energy University Program under the Award Number DE-NE308896. This research is also funded by the Department of Mechanical and Aerospace Engineering, Ohio State University.
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by: Jeff Offutt
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Li, B., Diao, X., Gao, W. et al. A requirements inspection method based on scenarios generated by model mutation and the experimental validation. Empir Software Eng 26, 108 (2021). https://doi.org/10.1007/s10664-021-10001-9
Accepted:
Published:
DOI: https://doi.org/10.1007/s10664-021-10001-9