Skip to main content
SpringerLink
Log in

Tuning parallel symbolic execution engine for better performance

  • Research Article
  • Published:
Frontiers of Computer Science Aims and scope Submit manuscript

Abstract

Symbolic execution is widely used in many code analysis, testing, and verification tools. As symbolic execution exhaustively explores all feasible paths, it is quite time consuming. To handle the problem, researchers have paralleled existing symbolic execution tools (e.g., KLEE). In particular, Cloud9 is a widely used paralleled symbolic execution tool, and researchers have used the tool to analyze real code. However, researchers criticize that tools such as Cloud9 still cannot analyze large scale code. In this paper, we conduct a field study on Cloud9, in which we use KLEE and Cloud9 to analyze benchmarks in C. Our results confirm the criticism. Based on the results, we identify three bottlenecks that hinder the performance of Cloud9: the communication time gap, the job transfer policy, and the cache management of the solved constraints. To handle these problems, we tune the communication time gap with better parameters, modify the job transfer policy, and implement an approach for cache management of solved constraints. We conduct two evaluations on our benchmarks and a real application to understand our improvements. Our results show that our tuned Cloud9 reduces the execution time significantly, both on our benchmarks and the real application. Furthermore, our evaluation results show that our tuning techniques improve the effectiveness on all the devices, and the improvement can be achieved upto five times, depending upon a tuning value of our approach and the behaviour of program under test.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Ciortea L, Zamfir C, Bucur S, Chipounov V, Candea G. Cloud9: a software testing service. ACM SIGOPS Operating Systems Review, 2010, 43(4): 5–10

    Article  Google Scholar 

  2. Staats M, Pˇasˇareanu C S. Parallel symbolic execution for structural testgeneration. In: Proceedings of the 19th International Symposium on Software Testing and Analysis. 2010, 183–194

    Google Scholar 

  3. King A. Distributed parallel symbolic execution. Dissertation for the Doctoral Degree. Manhattan, KS: Kansas State University, 2009

    Google Scholar 

  4. Siddiqui J H, Khurshid S. ParSym: Parallel symbolic execution. In: Proceedings of the 2nd International Conference on Software Technology and Engineering. 2010, 405–409

    Google Scholar 

  5. Sasnauskas R, Dustmann O S, Kaminski B L, Wehrle K, Weise C, Kowalewski S. Scalable symbolic execution of distributed systems. In: Proceedings of the 31st International Conference on Distributed Computing Systems. 2011, 333–342

    Google Scholar 

  6. Griesmayer A, Aichernig B, Johnsen E B, Schlatte R. Dynamic symbolic execution of distributed concurrent objects. In: Lee D, Lopes A, Poetzsch-Heffter A, eds. Formal Techniques for Distributed Systems. Berlin: Springer, 2009, 225–230

    Chapter  Google Scholar 

  7. Aleb N, Kechid S. A new approach for distributed symbolic software testing. In: Proceedings of International Conference on Computational Science and Its Applications. 2013, 487–497

    Google Scholar 

  8. Cadar C, Dunbar D, Engler D R. KLEE: unassisted and automatic generation of high-coverage tests for complex systems programs. In: Proceeding of OSDI. 2008, 209–224

    Google Scholar 

  9. Bucur S, Ureche V, Zamfir C, Candea G. Parallel symbolic execution for automated real-world software testing. In: Proceedings of the 6th ACM Conference on Computer Systems. 2011, 183–198

    Google Scholar 

  10. Candea G, Bucur S, Zamfir C. Automated software testing as a service. In: Proceedings of the 1st ACM Symposium on Cloud Computing. 2010, 155–160

    Chapter  Google Scholar 

  11. Renshaw D, Kong S. Symbolic execution in difficult environments. Technical Report 15–745. 2011

    Google Scholar 

  12. Marinescu P D, Cadar C. Make test-zesti: A symbolic execution solution for improving regression testing. In: Proceedings of the 34th International Conference on Software Engineering. 2012, 716–726

    Google Scholar 

  13. Cui H M, Hu G, Wu J Y, Yang J. Verifying systems rules using ruledirected symbolic execution. ACM SIGPLAN Notices, 2013, 48(4): 329–342

    Article  Google Scholar 

  14. Zhang D Z, Liu D G, Lei Y, Kung D, Csallner C, Wang WH. Detecting vulnerabilities in c programs using trace-based testing. In: Proceedings of IEEE/IFIP International Conference on Dependable Systems and Networks. 2010, 241–250

    Google Scholar 

  15. Zitser M, Lippmann R, Leek T. Testing static analysis tools using exploitable buffer overflows from open source code. ACM SIGSOFT Software Engineering Notes, 2004, 29(6): 97–106

    Article  Google Scholar 

  16. Saxena P, Poosankam P, McCamant S, Song D. Loop-extended symbolic execution on binary programs. In: Proceedings of the 18th ACM International Symposium on Software Testing and Analysis. 2009, 225–236

    Google Scholar 

  17. Jaffar J, Murali V, Navas J A, Santosa A E. TRACER: a symbolic execution tool for verification. In: Proceedings of the 24th International Conference on Computer Aided Verification. 2012, 758–766

    Chapter  Google Scholar 

  18. Coward P D. Symbolic execution systems-a review. Software Engineering Journal, 1988, 3(6): 229–239

    Article  Google Scholar 

  19. King J C. Symbolic execution and program testing. Communications of the ACM, 1976, 19(7): 385–394

    Article  MathSciNet  MATH  Google Scholar 

  20. Sen K, Marinov D, Agha G. CUTE: a concolic unit testing engine for C. ACMSIGSOFT Software Engineering Notes, 2005, 30(5): 263–272

    Article  Google Scholar 

  21. Zhang Y F, Chen Z B, Wang J. Speculative symbolic execution. In: Proceedings of the 23rd IEEE International Symposium on Software Reliability Engineering. 2012, 101–110

    Google Scholar 

  22. Palikareva H, Cadar C. Multi-solver support in symbolic execution. In: Proceedings of International Conference on Computer Aided Verification. 2013, 53–68

    Chapter  Google Scholar 

Download references

Acknowledgements

The authors are thankful to the researchers of Software Theory and Practice lab, Shanghai Jiao Tong University and the researchers at Intel Asia-Pacific Research and Development lab for their continuous support and idea to improve the basic motivation of this paper. The authors are also grateful to anonymous referees for their valuable comments and suggestions to improve the presentation of this paper. This research was sponsored by the National Basic Research Program of China (973 Program) (2015CB352203), the National Nature Science Foundation of China (Grant Nos. 61572312 and 61572313), Intel Asia-Pacific Research and Development Ltd., Shanghai, China, Japan Society for the Promotion of Science, Grant-in-Aid for Research Activity Start-up (16H07031), and the Science and Technology Commission of Shanghai Municipality’s Innovation Action Plan (15DZ1100305).

Author information

Authors and Affiliations

  1. Department of Computer Science, Shanghai Jiao Tong University, Shanghai, 200240, China

    Anil Kumar Karna & Hao Zhong

  2. School of Software, Shanghai Jiao Tong University, Shanghai, 200240, China

    Jinbo Du & Haibo Yu

  3. Intel Asia-Pacific Research and Development Ltd., Shanghai, 200240, China

    Haihao Shen, Jiong Gong & Xiangning Ma

  4. Department of Advanced Information Technology, Kyushu University, Fukuoka, 819-0395, Japan

    Jianjun Zhao

Authors
  1. Anil Kumar Karna
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jinbo Du
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Haihao Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hao Zhong
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jiong Gong
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Haibo Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Xiangning Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Jianjun Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Anil Kumar Karna.

Additional information

Anil Kumar Karna received his BE in computer science from Huazhong University of Science and Technology, China in 2004, and received his MS degree in software engineering from Shanghai Jiao Tong University (SJTU), China in 2010. He worked as a web-master, lead IT support engineer, and IT teacher in Nepal, India, and China, respectively from 2004 to 2007. He joined STAP lab as a PhD candidate in the Department of Computer Science and Engineering at SJTU. His current research interests include software reliability, system crashes, program analysis, testing and debugging, and verification and validation.

Jinbo Du received his BS in computer science from Linyi University, China in 2005, and received his MS degree in Software Engineering from Shanghai Jiao Tong University (SJTU), China in 2010. He worked as software engineering at Bopsoft in Qingdao from 2006 to 2007, and is working as software engineer in Network and Information center, SJTU since 2010. He joined STAP lab as a PhD candidate in School of Software at SJTU. His research interest includes software testing and symbolic execution.

Haihao Shen is a senior software engineer, working on critical software development and quality at Intel Asia Pacific Research and Development, China. His research interests include program analysis, software testing and verification, and machine learning. He joined Intel in 2010 after receiving his MS degree in software engineering from Shanghai Jiao Tong University, China.

Hao Zhong received his BS degree from Civil Aviation University of China in 2002; and both his MS and PhD degrees from Peking University, China in 2005 and 2009, respectively. After graduation, he worked as an assistant professor at Institute of Software, Chinese Academy of Sciences, China, and was promoted as an associate professor in 2012. From 2013 to 2014, he was a visiting scholar at University of California, Davis, USA. Since 2014, he has become an associate professor at Shanghai Jiao Tong University, China. He is a member of the IEEE and ACM. His research interests are mainly in the area of software engineering, with an emphasis on mining software repositories.

Jiong Gong received his MS degree in Computer Science from Shanghai Jiao Tong University, China in 2006. After his graduation, he is working as a senior software engineer at Intel Asia Pacific Research and Development, China. His project experiences and research interests are in the fields of model checking, symbolic execution, and machine learning.

Haibo Yu received her BS degree in computer science from Tsinghua University, China in 1987. She received her MS and PhD degree in computer science from Kyushu University, Japan in 1999 and 2009, respectively. She joined the School of Software at Shanghai Jiao Tong University, China in March, 2010. Her research interests include software engineering, information retrieval, and web application systems.

Xiangning Ma received her PhD from Institute of Computing technology, Chinese Academy of Sciences, China in 2004. After receiving her PhD degree, she joined Intel Asia Pacific Research and Development, China. She is an engineering manager at Intel, currently working on binary translation, dynamic optimization, validation tools and methodology.

Jianjun Zhao received the PhD degree in computer science from Kyushu University, Japan in 1997. After that, he joined the Department of Computer Science and Engineering, Fukuoka Institute of Technology, Japan as an assistant professor, and then was promoted to be an associate professor in 2000. From November 2005 to March 2016, he had been with the School of Software, and then the Department of Computer Science and Engineering, Shanghai Jiao Tong University, China as a professor. From April, 2016, he joined the Department of Advanced Information Technology, Kyushu University as a professor. His research interests include software engineering and programming languages, especially program analysis, software testing, code searching and automatic programming.

Electronic supplementary material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Karna, A.K., Du, J., Shen, H. et al. Tuning parallel symbolic execution engine for better performance. Front. Comput. Sci. 12, 86–100 (2018). https://doi.org/10.1007/s11704-016-5459-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11704-016-5459-9

Keywords

Search

Navigation