Skip to main content
SpringerLink
Log in

Programming by Examples: PL Meets ML

  • Conference paper
  • First Online:
Programming Languages and Systems (APLAS 2017)

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 10695))

Included in the following conference series:

Abstract

Programming by Examples (PBE) involves synthesizing intended programs in an underlying domain-specific language from example-based specifications. PBE systems are already revolutionizing the application domain of data wrangling and are set to significantly impact several other domains including code refactoring.

There are three key components in a PBE system. (i) A search algorithm that can efficiently search for programs that are consistent with the examples provided by the user. We leverage a divide-and-conquer-based deductive search paradigm that inductively reduces the problem of synthesizing a program expression of a certain kind that satisfies a given specification into sub-problems that refer to sub-expressions or sub-specifications. (ii) Program ranking techniques to pick an intended program from among the many that satisfy the examples provided by the user. We leverage features of the program structure as well of the outputs generated by the program on test inputs. (iii) User interaction models to facilitate usability and debuggability. We leverage active-learning techniques based on clustering inputs and synthesizing multiple programs.

Each of these PBE components leverage both symbolic reasoning and heuristics. We make the case for synthesizing these heuristics from training data using appropriate machine learning methods. This can not only lead to better heuristics, but can also enable easier development, maintenance, and even personalization of a PBE system.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Gulwani, S., Polozov, O., Singh, R.: Program synthesis. Found. Trends Program. Lang. 4(1–2), 1–119 (2017)

    Google Scholar 

  2. Gulwani, S.: Programming by examples - and its applications in data wrangling. In: Dependable Software Systems Engineering, pp. 137–158 (2016)

    Google Scholar 

  3. Le, V., Gulwani, S.: FlashExtract: a framework for data extraction by examples. In: PLDI, pp. 542–553 (2014)

    Google Scholar 

  4. Gulwani, S.: Automating string processing in spreadsheets using input-output examples. In: POPL, pp. 317–330 (2011)

    Google Scholar 

  5. Singh, R., Gulwani, S.: Learning semantic string transformations from examples. PVLDB 5(8), 740–751 (2012)

    Google Scholar 

  6. Singh, R., Gulwani, S.: Synthesizing number transformations from input-output examples. In: CAV, pp. 634–651 (2012)

    Google Scholar 

  7. Singh, R., Gulwani, S.: Transforming spreadsheet data types using examples. In: Proceedings of the 43rd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL 2016, St. Petersburg, FL, USA, January 20–22, 2016, pp. 343–356 (2016)

    Google Scholar 

  8. Barowy, D.W., Gulwani, S., Hart, T., Zorn, B.G.: FlashRelate: extracting relational data from semi-structured spreadsheets using examples. In: Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation, Portland, OR, USA, June 15–17, 2015, pp. 218–228 (2015)

    Google Scholar 

  9. Harris, W.R., Gulwani, S.: Spreadsheet table transformations from examples. In: PLDI, pp. 317–328 (2011)

    Google Scholar 

  10. Raza, M., Gulwani, S., Milic-Frayling, N.: Programming by example using least general generalizations. In: AAAI, pp. 283–290 (2014)

    Google Scholar 

  11. Rolim, R., Soares, G., D’Antoni, L., Polozov, O., Gulwani, S., Gheyi, R., Suzuki, R., Hartmann, B.: Learning syntactic program transformations from examples. In: Proceedings of the 39th International Conference on Software Engineering, ICSE 2017, Buenos Aires, Argentina, May 20–28, 2017, pp. 404–415 (2017)

    Google Scholar 

  12. Graves, A., Wayne, G., Reynolds, M., Harley, T., Danihelka, I., Grabska-Barwinska, A., Colmenarejo, S.G., Grefenstette, E., Ramalho, T., Agapiou, J., Badia, A.P., Hermann, K.M., Zwols, Y., Ostrovski, G., Cain, A., King, H., Summerfield, C., Blunsom, P., Kavukcuoglu, K., Hassabis, D.: Hybrid computing using a neural network with dynamic external memory. Nature 538(7626), 471–476 (2016)

    Article  Google Scholar 

  13. Joulin, A., Mikolov, T.: Inferring algorithmic patterns with stack-augmented recurrent nets. In: NIPS, pp. 190–198 (2015)

    Google Scholar 

  14. Neelakantan, A., Le, Q.V., Sutskever, I.: Neural programmer: inducing latent programs with gradient descent. CoRR abs/1511.04834 (2015)

    Google Scholar 

  15. Alur, R., Bodík, R., Dallal, E., Fisman, D., Garg, P., Juniwal, G., Kress-Gazit, H., Madhusudan, P., Martin, M.M.K., Raghothaman, M., Saha, S., Seshia, S.A., Singh, R., Solar-Lezama, A., Torlak, E., Udupa, A.: Syntax-guided synthesis. In: Dependable Software Systems Engineering, pp. 1–25 (2015)

    Google Scholar 

  16. Gulwani, S., Harris, W.R., Singh, R.: Spreadsheet data manipulation using examples. Commun. ACM 55(8), 97–105 (2012)

    Article  Google Scholar 

  17. Alur, R., Radhakrishna, A., Udupa, A.: Scaling enumerative program synthesis via divide and conquer. In: TACAS, pp. 319–336 (2017)

    Google Scholar 

  18. Raza, M., Gulwani, S.: Automated data extraction using predictive program synthesis. In: AAAI, pp. 882–890 (2017)

    Google Scholar 

  19. Devlin, J., Uesato, J., Bhupatiraju, S., Singh, R., Mohamed, A., Kohli, P.: Robustfill: neural program learning under noisy I/O. In: ICML (2017)

    Google Scholar 

  20. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)

    Article  Google Scholar 

  21. Singh, R., Gulwani, S.: Predicting a correct program in programming by example. In: Kroening, D., Păsăreanu, C.S. (eds.) CAV 2015. LNCS, vol. 9206, pp. 398–414. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-21690-4_23

    Chapter  Google Scholar 

  22. Menon, A.K., Tamuz, O., Gulwani, S., Lampson, B.W., Kalai, A.: A machine learning framework for programming by example. In: Proceedings of the 30th International Conference on Machine Learning, ICML 2013, Atlanta, GA, USA, 16–21 June 2013, pp. 187–195 (2013)

    Google Scholar 

  23. Balog, M., Gaunt, A.L., Brockschmidt, M., Nowozin, S., Tarlow, D.: Deepcoder: learning to write programs. In: ICLR (2017)

    Google Scholar 

  24. Singh, R.: Blinkfill: semi-supervised programming by example for syntactic string transformations. PVLDB 9(10), 816–827 (2016)

    Google Scholar 

  25. Ellis, K., Gulwani, S.: Learning to learn programs from examples: going beyond program structure. In: IJCAI, pp. 1638–1645 (2017)

    Google Scholar 

  26. Mayer, M., Soares, G., Grechkin, M., Le, V., Marron, M., Polozov, O., Singh, R., Zorn, B.G., Gulwani, S.: User interaction models for disambiguation in programming by example. In: UIST, pp. 291–301 (2015)

    Google Scholar 

  27. Padhi, S., Jain, P., Perelman, D., Polozov, O., Gulwani, S., Millstein, T.: Flashprofile: interactive synthesis of syntactic profiles. arXiv preprint arXiv:1709.05725 (2017)

  28. Jha, S., Gulwani, S., Seshia, S.A., Tiwari, A.: Oracle-guided component-based program synthesis. In: ICSE, pp. 215–224 (2010)

    Google Scholar 

  29. Gulwani, S., Marron, M.: Nlyze: interactive programming by natural language for spreadsheet data analysis and manipulation. In: SIGMOD, pp. 803–814 (2014)

    Google Scholar 

  30. Le, V., Gulwani, S., Su, Z.: Smartsynth: synthesizing smartphone automation scripts from natural language. In: The 11th Annual International Conference on Mobile Systems, Applications, and Services (MobiSys), pp. 193–206 (2013)

    Google Scholar 

  31. Raza, M., Gulwani, S., Milic-Frayling, N.: Compositional program synthesis from natural language and examples. In: IJCAI, pp. 792–800 (2015)

    Google Scholar 

  32. Solar-Lezama, A.: Program Synthesis by Sketching. Ph.D. thesis, University of California, Berkeley (2008)

    Google Scholar 

  33. Simpkins, C.: Integrating reinforcement learning into a programming language. In: Proceedings of the Twenty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2010, Atlanta, Georgia, USA, July 11–15, 2010 (2010)

    Google Scholar 

  34. Bielik, P., Raychev, V., Vechev, M.T.: Programming with “big code”: Lessons, techniques and applications. In: 1st Summit on Advances in Programming Languages, SNAPL 2015, May 3–6, 2015, Asilomar, California, USA, pp. 41–50 (2015)

    Google Scholar 

  35. Feser, J.K., Brockschmidt, M., Gaunt, A.L., Tarlow, D.: Neural functional programming. CoRR abs/1611.01988 (2016)

    Google Scholar 

  36. Singh, R., Kohli, P.: AP: artificial programming. In: 2nd Summit on Advances in Programming Languages, SNAPL 2017, May 7–10, 2017, Asilomar, CA, USA, pp. 16:1–16:12 (2017)

    Google Scholar 

Download references

Acknowledgments

We thank Nagarajan Natarajan, Naren Datha, Danny Simmons, Abhishek Mohta, Alex Polozov, and Daniel Perelman for their participation in the ongoing unpublished work that we have described in this article related to using ML techniques for search and ranking. We thank the entire PROSE team (https://microsoft.github.io/prose/team/) that has been developing and productizing the PBE technology inside many Microsoft products while also inspiring relevant research problem definitions. We thank Sriram Rajamani, Rishabh Singh, and Joseph Sirosh for stimulating discussions related to the topics discussed in this article.

Author information

Authors and Affiliations

  1. Microsoft Corporation, Redmond, USA

    Sumit Gulwani

  2. Microsoft Research, Bangalore, India

    Prateek Jain

Authors
  1. Sumit Gulwani
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Prateek Jain
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sumit Gulwani .

Editor information

Editors and Affiliations

  1. University of Colorado, Boulder, Colorado, USA

    Bor-Yuh Evan Chang

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Gulwani, S., Jain, P. (2017). Programming by Examples: PL Meets ML. In: Chang, BY. (eds) Programming Languages and Systems. APLAS 2017. Lecture Notes in Computer Science(), vol 10695. Springer, Cham. https://doi.org/10.1007/978-3-319-71237-6_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-71237-6_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-71236-9

  • Online ISBN: 978-3-319-71237-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation