Abstract
The oracle problem is an essential part in current research on automating software tests. Partial oracles seem to be a viable solution, but their suitability for different testing steps and general applicability for various systems remains still to be shown. This paper presents a study in which partial oracles are applied in order to automatically test a jpeg2000 encoder as an example for a modular software system with several integrated units and components. The effectiveness of the partial oracles is measured by means of mutation analysis to determine their adequacy for both unit and integration testing. Additionally, the paper presents possibilities of improving the effectiveness as well as the efficiency of the employed partial oracles. It shows how the knowledge of certain characteristics of the system to be tested, such as linearity or time-invariance, may lead to a better choice of partial oracles and thus to an improved effectiveness and efficiency.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Andrews, J. H., Briand, L. C., & Labiche, Y. (2005). Is mutation an appropriate tool for testing experiments? In: Proceedings of the 27th International Conference on Software Engineering, ICSE ’05 (pp. 402–411).
Bertolino, A. (2007). Software testing research: Achievements, challenges, dreams. In: 2007 Future of Software Engineering, FOSE ’07 (pp. 85–103).
Binder, R. (1999). Testing object-oriented systems: Models, patterns, and tools. Reading: Addison-Wesley.
Budd, T. A. (1980). Mutation analysis of program test data. Ph.D. thesis, Yale University.
Chen, T. Y., Huang, D. H., Tse, T. H., & Zhou, Z. Q. (2004). Case studies on the selection of useful relations in metamorphic testing. In: Proceedings of the 4th Ibero-American Symposium on Software Engineering and Knowledge Engineering, JIISIC ’04 (pp. 569–583).
Chen, T. Y., Tse, T. H., & Zhou, Z. Q. (2003). Fault-based testing without the need of oracles. Information and Software Technology, 45(1), 1–9.
Christopoulos, C., Skodras, A., & Ebrahimi, T. (2000). The jpeg2000 still image coding system: An overview. IEEE Transactions on Consumer Electronics, 46(4), 1103–1127.
DeMillo, R. A., Lipton, R. J., Sayward, F. G. (1978). Hints on test data selection: Help for the practicing programmer. IEEE Computer, 11(4), 34–41.
Guderlei, R., Just, R., Schneckenburger, C., & Schweiggert, F. (2008). Benchmarking testing strategies with tools from mutation analysis. In: Proceedings of the 2008 IEEE International Conference on Software Testing Verification and Validation Workshop, ICSTW ’08 (pp. 360–364).
(JJ2000. 2010) Official web site of the jj2000 project. http://jj2000.epfl.ch.
(Jpeg. 2010) Official web site of the joint photographic experts group. http://www.jpeg.org.
Just, R., Schweiggert, F. (2010). Automating software tests with partial oracles in integrated environments. In: Proceedings of the 5th Workshop on Automation of Software Test, AST ’10 (pp. 91–94).
Ma, Y. S., Kwon, Y. R., Offutt, J. (2002). Inter-class mutation operators for Java. In: Proceedings of the 13th International Symposium on Software Reliability Engineering, ISSRE ’02 (pp. 352–363).
Ma, Y. S., Offutt, J., & Kwon, Y. R. (2005). Mujava: An automated class mutation system. Software Testing, Verification, and Reliability, 15(2), 97–133.
McMinn, P. (2004). Search-based software test data generation: a survey. Software Testing, Verification, and Reliability, 14(2), 105–156.
MuJava. (2009). The web site of the MuJava project. http://cs.gmu.edu/offutt/mujava.
Offutt, A., & Pan, J. (1997). Automatically detecting equivalent mutants and infeasible paths. Software Testing, Verification, and Reliability, 7(3), 165–192.
Offutt, A. J., Lee, A., Rothermel, G., Untch, R. H., & Zapf, C. (1996). An experimental determination of sufficient mutant operators. ACM Transactions on Software Engineering and Methodology, 5(2), 99–118.
Skodras, A., Christopoulos, C., & Ebrahimi, T. (2001). The jpeg 2000 still image compression standard. IEEE Signal Processing Magazine, 18(5), 36–58.
Visser, W., Pasareanu, C. S., Khurshid, S. (2004). Test input generation with java PathFinder. In: Proceedings of the 2004 ACM SIGSOFT International Symposium on Software Testing and Analysis, ISSTA ’04 (pp. 97–107).
Voas, J. M. (1992). PIE: A dynamic failure-based technique. IEEE Transactions on Software Engineering, 18(8), 717–727.
Weyuker, E. J. (1982). On testing non-testable programs. The Computer Journal, 25(4), 465–470.
Zhou, Z. Q., Huang, D. H., Tse, T. H., Yang, Z., Huang, H., Chen, T. Y. (2004). Metamorphic testing and its applications. In: Proceedings of the 8th International Symposium on Future Software Technology, ISFST ’04.
Zhu, H., Hall, P. A. V., & May, J. H. R. (1997). Software unit test coverage and adequacy. ACM Computing Surveys, 29(4), 366–427.
Author information
Authors and Affiliations
Corresponding author
Additional information
A preliminary version of this paper was presented at the 5th International Workshop on Automation of Software Test (AST’10)(Just and Schweiggert 2010)
Rights and permissions
About this article
Cite this article
Just, R., Schweiggert, F. Automating unit and integration testing with partial oracles. Software Qual J 19, 753–769 (2011). https://doi.org/10.1007/s11219-011-9151-x
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11219-011-9151-x