Abstract
Because they are comparatively easy to implement, structural coverage criteria are commonly used for test derivation in model- and code-based testing. However, there is a lack of compelling evidence that they are useful for finding faults, specifically so when compared to random testing. This paper challenges the idea of using coverage criteria for test selection and instead proposes an approach based on fault models. We define a general fault model as a transformation from correct to incorrect programs and/or a partition of the input data space. Thereby, we leverage the idea of fault injection for test assessment to test derivation.
We instantiate the developed general fault model to describe existing fault models. We also show by example how to derive test cases.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Arcuri, A., Briand, L.: Formal analysis of the probability of interaction fault detection using random testing. IEEE Transactions on Software Engineering 38(5), 1088–1099 (2012)
Arcuri, A., Iqbal, M.Z., Briand, L.: Random testing: Theoretical results and practical implications. IEEE Transactions on Software Engineering 38(2), 258–277 (2012)
Binder, R.V.: Testing Object-Oriented Systems: Models, Patterns, and Tools. Object Technology Series. Addison-Wesley (1999)
Bochmann, G.v., Das, A., Dssouli, R., Dubuc, M., Ghedamsi, A., Luo, G.: Fault models in testing. In: Proceedings of the IFIP TC6/WG6.1 Fourth International Workshop on Protocol Test Systems IV, pp. 17–30. North-Holland Publishing Co., Amsterdam (1992), http://dl.acm.org/citation.cfm?id=648126.747577
Büchler, M., Oudinet, J., Pretschner, A.: Semi-automatic security testing of web applications from a secure model. In: 2012 IEEE Sixth International Conference on Software Security and Reliability (SERE), pp. 253–262 (2012)
Ceccato, M., Tonella, P., Ricca, F.: Is aop code easier or harder to test than oop code? In: On-line Proceedings of the First Workshop on Testing Aspect-Oriented Programs (WTAOP 2005) (March 2005)
Chillarege, R., Bhandari, I.S., Chaar, J.K., Halliday, M.J., Moebus, D.S., Ray, B.K., Wong, M.Y.: Orthogonal Defect Classification-A Concept for In-Process Measurements. IEEE Trans. Softw. Eng. 18(11), 943–956 (1992), http://dx.doi.org/10.1109/32.177364
Ciupa, I., Pretschner, A., Oriol, M., Leitner, A., Meyer, B.: On the number and nature of faults found by random testing. Softw. Test. Verif. Reliab. 21(1), 3–28 (2011)
Dupuy, A., Leveson, N.: An empirical evaluation of the mc/dc coverage criterion on the hete-2 satellite software. In: Proceedings of the 19th Digital Avionics Systems Conference, DASC 2000, vol. 1, pp. 1B6/1–1B6/7 (2000)
Fagan, M.E.: Design and code inspections to reduce errors in program development. IBM Syst. J. 15(3), 182–211 (1976), http://dx.doi.org/10.1147/sj.153.0182
Gutjahr, W.J.: Partition testing vs. random testing: The influence of uncertainty. IEEE Trans. Softw. Eng. 25(5), 661–674 (1999), http://dx.doi.org/10.1109/32.815325
Harris, I.G.: Fault models and test generation for hardware-software covalidation. IEEE Des. Test 20(04), 40–47 (2003), http://dx.doi.org/10.1109/MDT.2003.1214351 , doi:10.1109/MDT.2003.1214351
Heimdahl, M., Whalen, M., Rajan, A., Staats, M.: On mc/dc and implementation structure: An empirical study. In: IEEE/AIAA 27th Digital Avionics Systems Conference, DASC 2008, pp. 5.B.3–1–5.B.3–13 (2008)
Kuhn, D.R., Wallace, D.R., Gallo Jr., A.M.: Software fault interactions and implications for software testing. IEEE Trans. Softw. Eng. 30(6), 418–421 (2004)
Lu, S., Park, S., Seo, E., Zhou, Y.: Learning from mistakes: a comprehensive study on real world concurrency bug characteristics. In: Proceedings of the 13th International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS XIII, pp. 329–339. ACM, New York (2008), http://dx.doi.org/10.1145/1346281.1346323
Ma, Y.-S., Kwon, Y.-R., Offutt, J.: Inter-class mutation operators for java. In: ISSRE, pp. 352–366 (2002)
Malaiya, Y., Li, M., Bieman, J., Karcich, R.: Software reliability growth with test coverage. IEEE Transactions on Reliability 51(4), 420–426 (2002)
Martin, E., Xie, T.: A fault model and mutation testing of access control policies. In: Proceedings of the 16th International Conference on World Wide Web, WWW 2007, pp. 667–676. ACM, New York (2007), http://dx.doi.org/10.1145/1242572.1242663
McCluskey, E., Clegg, F.W.: Fault equivalence in combinational logic networks. IEEE Transactions on Computers C-20(11), 1286–1293 (1971)
Mockus, A., Nagappan, N., Dinh-Trong, T.T.: Test coverage and post-verification defects: A multiple case study. In: Proceedings of the 2009 3rd International Symposium on Empirical Software Engineering and Measurement, ESEM 2009, pp. 291–301 (2009)
Nagaraja, K., Li, X., Bianchini, R., Martin, R.P., Nguyen, T.D.: Using fault injection and modeling to evaluate the performability of cluster-based services. In: Proceedings of the 4th Conference on USENIX Symposium on Internet Technologies and Systems, USITS 2003, vol. 4, p. 2. USENIX Association, Berkeley (2003), http://dl.acm.org/citation.cfm?id=1251460.1251462
Offutt, J., Alexander, R., Wu, Y., Xiao, Q., Hutchinson, C.: A fault model for subtype inheritance and polymorphism. In: Proceedings of the 12th International Symposium on Software Reliability Engineering, ISSRE 2001, pp. 84–95. IEEE Computer Society, Washington, DC (2001), http://dl.acm.org/citation.cfm?id=851028.856258
Ostrand, T.J., Balcer, M.J.: The category-partition method for specifying and generating fuctional tests. Commun. ACM 31(6), 676–686 (1988), http://dx.doi.org/10.1145/62959.62964
Pretschner, A., Prenninger, W., Wagner, S., Kühnel, C., Baumgartner, M., Sostawa, B., Zölch, R., Stauner, T.: One evaluation of model-based testing and its automation. In: Proceedings of the 27th International Conference on Software Engineering, pp. 392–401 (2005)
Weyuker, E., Jeng, B.: Analyzing partition testing strategies. IEEE Transaction on Software Engineering 17(7), 703–711 (1991)
Zhu, H., Hall, P.A.V., May, J.H.R.: Software unit test coverage and adequacy. ACM Comput. Surv. 29(4), 366–427 (1997)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Pretschner, A., Holling, D., Eschbach, R., Gemmar, M. (2013). A Generic Fault Model for Quality Assurance. In: Moreira, A., Schätz, B., Gray, J., Vallecillo, A., Clarke, P. (eds) Model-Driven Engineering Languages and Systems. MODELS 2013. Lecture Notes in Computer Science, vol 8107. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-41533-3_6
Download citation
DOI: https://doi.org/10.1007/978-3-642-41533-3_6
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-41532-6
Online ISBN: 978-3-642-41533-3
eBook Packages: Computer ScienceComputer Science (R0)