skip to main content
research-article

Complete Shadow Symbolic Execution with Java PathFinder

Published: 12 December 2019 Publication History

Abstract

Regression testing ensures the correctness of the software during its evolution, with special attention on the absence of unintended side-e ects that might be introduced by changes. However, the manual creation of regression test cases, which expose divergent behavior, needs a lot of e ort. A solution is the idea of shadow symbolic execution, which takes a uni ed version of the old and the new programs and performs symbolic execution guided by concrete values to explore the changed behavior.
In this work, we adapt the idea of shadow symbolic execution (SSE) and combine complete/standard symbolic execution with the idea of four-way forking to expose diverging behavior. There- fore, our approach attempts to comprehensively test the new be- haviors introduced by a change. We implemented our approach in the tool ShadowJPF+, which performs complete shadow sym- bolic execution on Java bytecode. It is an extension of the tool ShadowJPF, which is based on Symbolic PathFinder. We applied our tool on 79 examples, for which it was able to reveal more di- verging behaviors than common shadow symbolic execution. Ad- ditionally, the approach has been applied on a real-world patch for the Joda-Time library, for which it successfully generated test cases that expose a regression error.

References

[1]
C. Cadar, D. Dunbar, and D. Engler. Klee: Unassisted and automatic generation of high-coverage tests for complex sys- tems programs. In Proceedings of the 8th USENIX Con- ference on Operating Systems Design and Implementation, OSDI'08, pages 209{224, Berkeley, CA, USA, 2008. USENIX Association.
[2]
C. Cadar and K. Sen. Symbolic execution for software test- ing: Three decades later. Commun. ACM, 56(2):82{90, Feb. 2013.
[3]
L. A. Clarke. A system to generate test data and symbolically execute programs. IEEE Transactions on Software Engineer- ing, SE-2(3):215{222, Sept 1976.
[4]
L. De Moura and N. Bjorner. Z3: An efficient smt solver. In Proceedings of the Theory and Practice of Software, 14th In- ternational Conference on Tools and Algorithms for the Con- struction and Analysis of Systems, TACAS'08/ETAPS'08, pages 337{340, Berlin, Heidelberg, 2008. Springer-Verlag.
[5]
T. L. Graves, M. J. Harrold, J.-M. Kim, A. Porter, and G. Rothermel. An empirical study of regression test selection techniques. ACM Trans. Softw. Eng. Methodol., 10(2):184{ 208, Apr. 2001.
[6]
Z. Gu, E. T. Barr, D. J. Hamilton, and Z. Su. Has the bug really been xed? In Proceedings of the 32Nd ACM/IEEE International Conference on Software Engineering - Volume 1, ICSE '10, pages 55{64, New York, NY, USA, 2010. ACM.
[7]
M. J. Harrold, J. A. Jones, T. Li, D. Liang, A. Orso, M. Pen- nings, S. Sinha, S. A. Spoon, and A. Gujarathi. Regres- sion test selection for java software. In Proceedings of the 16th ACM SIGPLAN Conference on Object-oriented Pro- gramming, Systems, Languages, and Applications, OOPSLA '01, pages 312{326, New York, NY, USA, 2001. ACM.
[8]
R. Just, F. Schweiggert, and G. M. Kapfhammer. Major: An efficient and extensible tool for mutation analysis in a java compiler. In Proceedings of the 2011 26th IEEE/ACM In- ternational Conference on Automated Software Engineering, ASE '11, pages 612{615, Washington, DC, USA, 2011. IEEE Computer Society.
[9]
J. C. King. Symbolic execution and program testing. Com- mun. ACM, 19(7):385{394, July 1976.
[10]
K.-K. Ma, K. Yit Phang, J. S. Foster, and M. Hicks. Di- rected Symbolic Execution, pages 95{111. Springer Berlin Heidelberg, Berlin, Heidelberg, 2011.
[11]
P. D. Marinescu and C. Cadar. Katch: High-coverage test- ing of software patches. In Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering, ES- EC/FSE 2013, pages 235{245, New York, NY, USA, 2013. ACM.
[12]
Y. Noller, H. L. Nguyen, M. Tang, and T. Kehrer. Shadow symbolic execution with java path nder. SIGSOFT Softw. Eng. Notes, 42(4):1{5, Jan. 2018.
[13]
H. Palikareva, T. Kuchta, and C. Cadar. Shadow of a doubt: Testing for divergences between software versions. In Pro- ceedings of the 38th International Conference on Software Engineering, ICSE '16, pages 1181{1192, New York, NY, USA, 2016. ACM.
[14]
C. S. Pasareanu, W. Visser, D. Bushnell, J. Geldenhuys, P. Mehlitz, and N. Rungta. Symbolic path nder: integrating symbolic execution with model checking for java bytecode analysis. Automated Software Engineering, 20(3):391{425, 2013.
[15]
W. Visser, C. S. Pasareanu, and S. Khurshid. Test input gen- eration with java path nder. SIGSOFT Softw. Eng. Notes, 29(4):97{107, July 2004.
[16]
Z. Yin, D. Yuan, Y. Zhou, S. Pasupathy, and L. Bairavasun- daram. How do xes become bugs? In Proceedings of the 19th ACM SIGSOFT Symposium and the 13th European Confer- ence on Foundations of Software Engineering, ESEC/FSE '11, pages 26{36, New York, NY, USA, 2011. ACM.

Cited By

View all
  • (2022)Test Suite Augmentation for Reconfigurable PLC Software in the Internet of ProductionFormal Methods for Industrial Critical Systems10.1007/978-3-031-15008-1_10(137-154)Online publication date: 14-Sep-2022
  • (2022)Hybrid Differential Software TestingErnst Denert Award for Software Engineering 202010.1007/978-3-030-83128-8_9(167-195)Online publication date: 28-Feb-2022
  • (2020)The Java Pathfinder Workshop 2019ACM SIGSOFT Software Engineering Notes10.1145/3385678.338568545:2(20-22)Online publication date: 3-May-2020

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM SIGSOFT Software Engineering Notes
ACM SIGSOFT Software Engineering Notes  Volume 44, Issue 4
October 2019
47 pages
ISSN:0163-5948
DOI:10.1145/3364452
Issue’s Table of Contents
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 December 2019
Published in SIGSOFT Volume 44, Issue 4

Check for updates

Author Tags

  1. regression testing
  2. symbolic execution
  3. symbolic pathfinder

Qualifiers

  • Research-article

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)10
  • Downloads (Last 6 weeks)1
Reflects downloads up to 08 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2022)Test Suite Augmentation for Reconfigurable PLC Software in the Internet of ProductionFormal Methods for Industrial Critical Systems10.1007/978-3-031-15008-1_10(137-154)Online publication date: 14-Sep-2022
  • (2022)Hybrid Differential Software TestingErnst Denert Award for Software Engineering 202010.1007/978-3-030-83128-8_9(167-195)Online publication date: 28-Feb-2022
  • (2020)The Java Pathfinder Workshop 2019ACM SIGSOFT Software Engineering Notes10.1145/3385678.338568545:2(20-22)Online publication date: 3-May-2020

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media