Abstract
Testing of multi-threaded programs is a demanding work due to the many possible thread interleavings one should examine. The noise injection technique helps to increase the number of thread interleavings examined during repeated test executions provided that a suitable setting of noise injection heuristics is used. The problem of finding such a setting, i.e., the so called test and noise configuration search problem (TNCS problem), is not easy to solve. In this paper, we show how to apply a multi-objective genetic algorithm (MOGA) to the TNCS problem. In particular, we focus on generation of TNCS solutions that cover a high number of distinct interleavings (especially those which are rare) and provide stable results at the same time. To achieve this goal, we study suitable metrics and ways how to suppress effects of non-deterministic thread scheduling on the proposed MOGA-based approach. We also discuss a choice of a concrete MOGA and its parameters suitable for our setting. Finally, we show on a set of benchmark programs that our approach provides better results when compared to the commonly used random approach as well as to the sooner proposed use of a single-objective genetic approach.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Alba, E., Chicano, F.: Finding Safety Errors with ACO. In: Proc. of GECCO 2007. ACM (2007)
Ayewah, N., Pugh, W., Morgenthaler, J.D., Penix, J., Zhou, Y.: Using FindBugs on Production Software. In: Proc. of OOPSLA 2007. ACM (2007)
Bensalem, S., Havelund, K.: Dynamic Deadlock Analysis of Multi-threaded Programs. In: Ur, S., Bin, E., Wolfsthal, Y. (eds.) HVC 2005. LNCS, vol. 3875, pp. 208–223. Springer, Heidelberg (2006)
Bron, A., Farchi, E., Magid, Y., Nir, Y., Ur, S.: Applications of Synchronization Coverage. In: Proc. of PPoPP 2005. ACM (2005)
Chockler, H., Farchi, E., Godlin, B., Novikov, S.: Cross-entropy Based Testing. In: Proc. of FMCAD 2007. IEEE (2007)
Clarke, E., Grumberg, O., Peled, D.: Model Checking. MIT Press (1999)
Deb, K.: Multi-Objective Optimization Using Evolutionary Algorithms. Wiley paperback series. Wiley (2009)
Edelstein, O., Farchi, E., Goldin, E., Nir, Y., Ratsaby, G., Ur, S.: Framework for Testing Multi-threaded Java Programs. Concurrency and Computation: Practice and Experience 15(3-5) (2003)
Elmas, T., Qadeer, S., Tasiran, S.: Goldilocks: A Race and Transaction-aware Java Runtime. In: Proc. of PLDI 2007. ACM (2007)
Fraser, G., Arcuri, A.: Whole Test Suite Generation. IEEE Transactions on Software Engineering 39(2) (2013)
Godefroid, P., Khurshid, S.: Exploring Very Large State Spaces Using Genetic Algorithms. International Journal on Software Tools for Technology Transfer 6(2) (2004)
Hong, S., Ahn, J., Park, S., Kim, M., Harrold, M.J.: Testing Concurrent Programs to Achieve High Synchronization Coverage. In: Proc. of ISSTA 2012. ACM (2012)
Hrubá, V., Křena, B., Letko, Z., Ur, S., Vojnar, T.: Testing of Concurrent Programs Using Genetic Algorithms. In: Fraser, G., de Teixeira Souza, J. (eds.) SSBSE 2012. LNCS, vol. 7515, pp. 152–167. Springer, Heidelberg (2012)
Ishibuchi, H., Shibata, Y.: A Similarity-based Mating Scheme for Evolutionary Multiobjective Optimization. In: Cantú-Paz, E., et al. (eds.) GECCO 2003. LNCS, vol. 2723, pp. 1065–1076. Springer, Heidelberg (2003)
Jin, Y., Branke, J.: Evolutionary Optimization in Uncertain Environments – A Survey. IEEE Transactions on Evolutionary Computation 9(3) (2005)
Křena, B., Letko, Z., Nir-Buchbinder, Y., Tzoref-Brill, R., Ur, S., Vojnar, T.: A Concurrency Testing Tool and its Plug-ins for Dynamic Analysis and Runtime Healing. In: Bensalem, S., Peled, D.A. (eds.) RV 2009. LNCS, vol. 5779, pp. 101–114. Springer, Heidelberg (2009)
Křena, B., Letko, Z., Tzoref, R., Ur, S., Vojnar, T.: Healing Data Races On-the-fly. In: Proc. of PADTAD 2007. ACM (2007)
Křena, B., Letko, Z., Vojnar, T.: Coverage Metrics for Saturation-based and Search-based Testing of Concurrent Software. In: Khurshid, S., Sen, K. (eds.) RV 2011. LNCS, vol. 7186, pp. 177–192. Springer, Heidelberg (2012)
Křena, B., Letko, Z., Vojnar, T., Ur, S.: A Platform for Search-based Testing of Concurrent Software. In: Proc. of PADTAD 2010. ACM (2010)
Křena, B., Letko, Z., Vojnar, T.: Noise Injection Heuristics on Concurrency Coverage. In: Kotásek, Z., Bouda, J., Černá, I., Sekanina, L., Vojnar, T., Antoš, D. (eds.) MEMICS 2011. LNCS, vol. 7119, pp. 123–135. Springer, Heidelberg (2012)
Lu, S., Tucek, J., Qin, F., Zhou, Y.: AVIO: Detecting Atomicity Violations via Access Interleaving Invariants. In: Proc. of ASPLOS 2006. ACM (2006)
Musuvathi, M., Qadeer, S., Ball, T., Basler, G., Nainar, P.A., Neamtiu, I.: Finding and Reproducing Heisenbugs in Concurrent Programs. In: OSDI. USENIX Association (2008)
Peierls, T., Goetz, B., Bloch, J., Bowbeer, J., Lea, D., Holmes, D.: Java Concurrency in Practice. Addison-Wesley Professional (2005)
Savage, S., Burrows, M., Nelson, G., Sobalvarro, P., Anderson, T.: Eraser: A Dynamic Data Race Detector for Multi-threaded Programs. In: Proc. of SOSP 1997. ACM (1997)
Smith, L.A., Bull, J.M., Obdržálek, J.: A Parallel Java Grande Benchmark Suite. In: Proc. of Supercomputing 2001. ACM (2001)
Spillner, A., Linz, T., Schaefer, H.: Software Testing Foundations: A Study Guide for the Certified Tester Exam, 3rd edn. Rocky Nook (2011)
Staunton, J., Clark, J.A.: Searching for Safety Violations Using Estimation of Distribution Algorithms. In: Proc. of ICSTW 2010. IEEE (2010)
Steenbuck, S., Fraser, G.: Generating Unit Tests for Concurrent Classes. In: ICST 2013. IEEE (2013)
White, D.: Software Review: The ECJ Toolkit. Genetic Programming and Evolvable Machines 13 (2012)
Yu, J., Narayanasamy, S., Pereira, C., Pokam, G.: Maple: A Coverage-driven Testing Tool for Multithreaded Programs. In: Proc. of OOPSLA 2012. ACM (2012)
Zitzler, E.: Evolutionary Algorithms for Multiobjective Optimization: Methods and Applications. PhD thesis, ETH Zurich (1999)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Hrubá, V., Křena, B., Letko, Z., Pluháčková, H., Vojnar, T. (2014). Multi-objective Genetic Optimization for Noise-Based Testing of Concurrent Software. In: Le Goues, C., Yoo, S. (eds) Search-Based Software Engineering. SSBSE 2014. Lecture Notes in Computer Science, vol 8636. Springer, Cham. https://doi.org/10.1007/978-3-319-09940-8_8
Download citation
DOI: https://doi.org/10.1007/978-3-319-09940-8_8
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-09939-2
Online ISBN: 978-3-319-09940-8
eBook Packages: Computer ScienceComputer Science (R0)