Skip to main content
SpringerLink
Log in

Bootstrapping Library-Based Synthesis

  • Conference paper
  • First Online:
Static Analysis (SAS 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13790))

Included in the following conference series:

Abstract

Constraint-based program synthesis techniques have been widely used in numerous settings. However, synthesizing programs that use libraries remains a major challenge. To handle complex or black-box libraries, the state of the art is to provide carefully crafted mocks or models to the synthesizer, requiring extra manual work. We address this challenge by proposing Toshokan, a new synthesis framework as an alternative approach in which library-using programs can be generated without any user-provided artifacts at the cost of moderate performance overhead. The framework extends the classic counterexample-guided synthesis framework with a bootstrapping, log-based library model. The model collects input-output samples from running failed candidate programs on witness inputs. We prove that the framework is sound when a sound, bounded verifier is available, and also complete if the underlying synthesizer and verifier promise to produce minimal outputs. We implement and incorporate the framework to JSketch, a Java sketching tool. Experiments show that Toshokan can successfully synthesize programs that use a variety of libraries, ranging from mathematical functions to data structures. Comparing to state-of-the-art synthesis algorithms which use mocks or models, Toshokan reduces up to 159 lines of code of required manual inputs, at the cost of less than 40 s of performance overheads.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 79.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The validated artifact is available via DOI 10.5281/zenodo.7009051.

  2. 2.

    The actual library operates BigInteger objects; for simplicity, we adapt the signature to handle int’s.

  3. 3.

    Toshokan can actually solve this problem in 1 iteration (see Sect. 6); we use this 4-iteration run for illustration purpose.

  4. 4.

    This limitation is not fundamental and can be generalized in the future. Without calls between library functions, the implementation of library logger becomes easier since logging instrumentation would only need to be done on client code.

  5. 5.

    Here we assume all integer arguments are positive and use negative integers to represent methods. If negative integers are involved in the program, the array encoding has to have an extra bit to indicate a leaf node is a primitive value or a method call.

  6. 6.

    As a limitation of current JSketch, when generators are involved, the raw output of JSketch is not compilable and some manual adaptation is needed. This is impossible for Toshokan because the CEGIS loop must compile JSketch output every iteration.

References

  1. Ammons, G., Bodík, R., Larus, J.R.: Mining specifications. In: Proceedings of the 29th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages. POPL 2002. ACM (2002). https://doi.org/10.1145/503272.503275

  2. Astorga, A., Madhusudan, P., Saha, S., Wang, S., Xie, T.: Learning stateful preconditions modulo a test generator. In: Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation. PLDI 2019. ACM (2019). https://doi.org/10.1145/3314221.3314641

  3. Babić, D., et al.: Fudge: fuzz driver generation at scale. In: Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. ESEC/FSE 2019. ACM (2019). https://doi.org/10.1145/3338906.3340456

  4. Bastani, O., Anand, S., Aiken, A.: Specification inference using context-free language reachability. In: Proceedings of the 42nd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages. POPL 2015. ACM (2015). https://doi.org/10.1145/2676726.2676977

  5. Bastani, O., Sharma, R., Aiken, A., Liang, P.: Active learning of points-to specifications. In: Proceedings of the 39th ACM SIGPLAN Conference on Programming Language Design and Implementation. PLDI 2018. ACM (2018). https://doi.org/10.1145/3192366.3192383

  6. Bodik, R., Chandra, S., Galenson, J., Kimelman, D., Tung, N., Barman, S., Rodarmor, C.: Programming with angelic nondeterminism. In: Proceedings of the 37th annual ACM SIGPLAN-SIGACT symposium on Principles of programming languages. POPL 2010. ACM (2010). https://doi.org/10.1145/1706299.1706339

  7. Bornholt, J., Torlak, E.: Finding code that explodes under symbolic evaluation. In: Proc. of the ACM on Programming Languages. OOPSLA 2018, vol. 2. ACM, October 2018. https://doi.org/10.1145/3276519

  8. Cordeiro, L., Kesseli, P., Kroening, D., Schrammel, P., Trtik, M.: JBMC: a bounded model checking tool for verifying Java bytecode. In: Chockler, H., Weissenbacher, G. (eds.) CAV 2018. LNCS, vol. 10981, pp. 183–190. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-96145-3_10

    Chapter  Google Scholar 

  9. Delaware, B., Pit-Claudel, C., Gross, J., Chlipala, A.: Fiat: deductive synthesis of abstract data types in a proof assistant. In: Proceedings of the 42nd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages. POPL 2015. ACM (2015). https://doi.org/10.1145/2676726.2677006

  10. Doughty-White, P., Quick, M.: Codebases: millions of lines of code (2015). https://informationisbeautiful.net/visualizations/million-lines-of-code/

  11. Ernst, M.D., et al.: The daikon system for dynamic detection of likely invariants. Science of Computer Programming 69(1), 35–45 (2007). https://doi.org/10.1016/j.scico.2007.01.015, special issue on Experimental Software and Toolkits

  12. Feng, Y., Martins, R., Wang, Y., Dillig, I., Reps, T.W.: Component-based synthesis for complex APIs. In: Proceedings of the 44th ACM SIGPLAN Symposium on Principles of Programming Languages. POPL 2017. ACM (2017). https://doi.org/10.1145/3009837.3009851

  13. Floyd, R.W.: Nondeterministic algorithms. J. ACM (JACM) 14(4), 636–644 (1967). https://doi.org/10.1145/321420.321422

    Article  MATH  Google Scholar 

  14. Galenson, J., Reames, P., Bodik, R., Hartmann, B., Sen, K.: CodeHint: dynamic and interactive synthesis of code snippets. In: Proceedings of the 36th International Conference on Software Engineering. ICSE 2014. ACM (2014). https://doi.org/10.1145/2568225.2568250

  15. Gascón, A., Tiwari, A., Carmer, B., Mathur, U.: Look for the proof to find the program: decorated-component-based program synthesis. In: Majumdar, R., Kunčak, V. (eds.) CAV 2017. LNCS, vol. 10427, pp. 86–103. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-63390-9_5

    Chapter  Google Scholar 

  16. Heule, S., Sridharan, M., Chandra, S.: Mimic: computing models for opaque code. In: Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering - ESEC/FSE 2015 (2015). https://doi.org/10.1145/2786805.2786875

  17. Hu, Q., Samanta, R., Singh, R., D’Antoni, L.: Direct manipulation for imperative programs. In: Chang, B.-Y.E. (ed.) SAS 2019. LNCS, vol. 11822, pp. 347–367. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32304-2_17

    Chapter  Google Scholar 

  18. JDial Debugger (2021). https://github.com/JDial-Debugger/backend/tree/master/SkechObject/benchmarks

  19. Jeon, J., Qiu, X., Fetter-Degges, J., Foster, J.S., Solar-Lezama, A.: Synthesizing framework models for symbolic execution. In: ICSE 2016. ACM (2016). https://doi.org/10.1145/2884781.2884856

  20. Jeon, J., Qiu, X., Foster, J.S., Solar-Lezama, A.: Jsketch: sketching for Java. In: Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering. ESEC/FSE 2015. ACM (2015). https://doi.org/10.1145/2786805.2803189

  21. Jha, S., Gulwani, S., Seshia, S.A., Tiwari, A.: Oracle-guided component-based program synthesis. In: Proceedings of the 32nd ACM/IEEE International Conf. on Software Engineering. ICSE 2010, vol. 1. ACM (2010). https://doi.org/10.1145/1806799.1806833

  22. Li, W., Seshia, S.A.: Sparse coding for specification mining and error localization. In: Qadeer, S., Tasiran, S. (eds.) RV 2012. LNCS, vol. 7687, pp. 64–81. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-35632-2_9

    Chapter  Google Scholar 

  23. Lustig, Y., Vardi, M.Y.: Synthesis from component libraries. Int. J. Softw. Tools Technol. Transf. 603–618 (2012). https://doi.org/10.1007/s10009-012-0236-z

  24. Mandelin, D., Xu, L., Bodík, R., Kimelman, D.: Jungloid mining: helping to navigate the API jungle. In: Proceedings of the 2005 ACM SIGPLAN Conference on Programming Language Design and Implementation. PLDI 2005. ACM (2005). https://doi.org/10.1145/1065010.1065018

  25. Mariano, B., et al.: Program synthesis with algebraic library specifications. In: Proceedings of the ACM on Programming Languages. OOPSLA 2019, vol. 3. ACM (Oct 2019). https://doi.org/10.1145/3360558

  26. Miltner, A., Nuñez, A.T., Brendel, A., Chaudhuri, S., Dillig, I.: Bottom-up synthesis of recursive functional programs using angelic execution. In: Proceedings of the ACM on Programming Languages. POPL 2022, vol. 6. ACM, January 2022. https://doi.org/10.1145/3498682

  27. Murali, V., Qi, L., Chaudhuri, S., Jermaine, C.: Neural sketch learning for conditional program generation. In: International Conference on Learning Representations (2018). https://openreview.net/forum?id=HkfXMz-Ab

  28. OpenJDK (2014). https://hg.openjdk.java.net/jdk8/jdk8/jdk/file/tip/src/share/classes/java/util/ArrayList.java

  29. Pacheco, C., Lahiri, S.K., Ernst, M.D., Ball, T.: Feedback-directed random test generation. In: 29th International Conference on Software Engineering (ICSE 2007), May 2007. https://doi.org/10.1109/ICSE.2007.37

  30. Pei, Y., Furia, C.A., Nordio, M., Wei, Y., Meyer, B., Zeller, A.: Automated fixing of programs with contracts. IEEE Trans. Softw. Eng. 40(5), 427–449 (2014). https://doi.org/10.1109/TSE.2014.2312918

    Article  Google Scholar 

  31. Polgreen, E., Reynolds, A., Seshia, S.A.: Satisfiability and synthesis modulo oracles. In: Finkbeiner, B., Wies, T. (eds.) VMCAI 2022. LNCS, vol. 13182, pp. 263–284. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-94583-1_13

    Chapter  MATH  Google Scholar 

  32. Raychev, V., Bielik, P., Vechev, M., Krause, A.: Learning programs from noisy data. In: Proceedings of the 43rd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages. POPL 2016. ACM (2016). https://doi.org/10.1145/2837614.2837671

  33. Shi, K., Steinhardt, J., Liang, P.: Frangel: component-based synthesis with control structures. In: Proceedings of the ACM on Programming Languages. POPL 2019, vol. 3. ACM, January 2019. https://doi.org/10.1145/3290386

  34. Singh, R., Gulwani, S., Solar-Lezama, A.: Automated feedback generation for introductory programming assignments. In: Proceedings of the 34th ACM SIGPLAN Conference on Programming Language Design and Implementation. PLDI 2013. ACM (2013). https://doi.org/10.1145/2491956.2462195

  35. Singh, R., Singh, R., Xu, Z., Krosnick, R., Solar-Lezama, A.: Modular synthesis of sketches using models. In: McMillan, K.L., Rival, X. (eds.) VMCAI 2014. LNCS, vol. 8318, pp. 395–414. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-642-54013-4_22

    Chapter  Google Scholar 

  36. Sketch (2021). https://github.com/asolarlez/sketch-frontend/blob/master/src/experiments/sk/models/

  37. Skrupsky, N., Monshizadeh, M., Bisht, P., Hinrichs, T., Venkatakrishnan, V.N., Zuck, L.: Waves: automatic synthesis of client-side validation code for web applications. In: 2012 International Conference on Cyber Security, December 2012. https://doi.org/10.1109/CyberSecurity.2012.13

  38. Smith, C., Albarghouthi, A.: Program synthesis with equivalence reduction. In: Enea, C., Piskac, R. (eds.) VMCAI 2019. LNCS, vol. 11388, pp. 24–47. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-11245-5_2

    Chapter  Google Scholar 

  39. Solar-Lezama, A.: The sketch programmers manual (2020). https://people.csail.mit.edu/asolar/manual.pdf, version 1.7.6

  40. Solar-Lezama, A., Tancau, L., Bodik, R., Seshia, S., Saraswat, V.: Combinatorial sketching for finite programs. In: Proceedings of the 12th International Conference on Architectural Support for Programming Languages and Operating Systems. ACM (2006). https://doi.org/10.1145/1168857.1168907

  41. Srivastava, S., Gulwani, S., Chaudhuri, S., Foster, J.S.: Path-based inductive synthesis for program inversion. In: Proceedings of the 32nd ACM SIGPLAN Conference on Programming Language Design and Implementation. PLDI 2011. ACM (2011). https://doi.org/10.1145/1993498.1993557

  42. Torlak, E., Bodik, R.: A lightweight symbolic virtual machine for solver-aided host languages. In: Proceedings of the 35th ACM SIGPLAN Conference on Programming Language Design and Implementation. PLDI 2014. ACM (2014). https://doi.org/10.1145/2594291.2594340

  43. Yang, Z., Hua, J., Wang, K., Khurshid, S.: EdSynth: synthesizing API sequences with conditionals and loops. In: 2018 IEEE 11th International Conference on Software Testing, Verification and Validation. ICST 2018, April 2018. https://doi.org/10.1109/ICST.2018.00025

Download references

Acknowledgments

This research was supported in part by the National Science Foundation under Grant Nos. CCF-1919197 and CCF-2046071.

Author information

Authors and Affiliations

  1. Purdue University, West Lafayette, IN, 47907, USA

    Kangjing Huang & Xiaokang Qiu

Authors
  1. Kangjing Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaokang Qiu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaokang Qiu .

Editor information

Editors and Affiliations

  1. VMware Research and University of Illinois Urbana-Champaign, Urbana, IL, USA

    Gagandeep Singh

  2. Inria and ENS/PSL, Paris, France

    Caterina Urban

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Huang, K., Qiu, X. (2022). Bootstrapping Library-Based Synthesis. In: Singh, G., Urban, C. (eds) Static Analysis. SAS 2022. Lecture Notes in Computer Science, vol 13790. Springer, Cham. https://doi.org/10.1007/978-3-031-22308-2_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-22308-2_13

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-22307-5

  • Online ISBN: 978-3-031-22308-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation