Skip to main content
SpringerLink
Log in

MerIt: improving neural program synthesis by merging collective intelligence

  • Published:
Automated Software Engineering Aims and scope Submit manuscript

Abstract

Program synthesis is the task of automatically generating programs from user intent, which is one of the central problems in automated software engineering. Recently many researchers use a neural network to learn the distribution over programs based on user intent (such as API and type name), known as neural program synthesis (NPS). The generated programs of NPS are highly dependent on user intent. However, it is difficult for users to provide an accurate and complete intent for the NPS model, which decreases the synthesis accuracy of NPS. Collective Intelligence (CI) is an emerging trend, which illustrates that collective wisdom surpasses individual wisdom. Inspired by CI techniques, we propose an automatic task-specific user intent merging framework for NPS named MerIt (Merge User Intent of Program Synthesis). The key point of our framework is that we propose an improved Unsupervised Ant Colony Optimization (UACO) algorithm to selectively merge effective intent from multiple developers, and design three selection strategies to guide the merge process. The experiments show that our approach is able to provide more adequate and efficient input for NPS and improve the synthesis accuracy. Besides, our evaluation shows that selectively merging knowledge from multiple developers could be a significant way of promoting automated software engineering.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Notes

  1. https://git-scm.com/docs/git-merge.

  2. https://docs.oracle.com/javase/8/docs/api/.

  3. https://www.programcreek.com.

  4. https://www.codota.com.

References

  • Abolafia, D.A., Norouzi, M., Le, Q.V.: Neural program synthesis with priority queue training. CoRR abs arXiv:1801.03526 (2018)

  • Abualigah, L.M., Khader, A.T.: Unsupervised text feature selection technique based on hybrid particle swarm optimization algorithm with genetic operators for the text clustering. J. Supercomput. 73(11), 4773–4795 (2017)

    Article  Google Scholar 

  • Acharya, M., Xie, T., Pei, J., Xu, J.: Mining API patterns as partial orders from source code: from usage scenarios to specifications. In: Proceedings of the 6th joint meeting of the European Software Engineering Conference and the ACM SIGSOFT International Symposium on Foundations of Software Engineering, 2007, Dubrovnik, Croatia, 3–7 Sept 2007, pp. 25–34 (2007). https://doi.org/10.1145/1287624.1287630

  • Al-Tashi, Q., Kadir, S.J.A., Rais, H.M., Mirjalili, S., Alhussian, H.: Binary optimization using hybrid grey wolf optimization for feature selection. IEEE Access 7, 39496–39508 (2019)

    Article  Google Scholar 

  • Allamanis, M., Brockschmidt, M., Khademi, M.: Learning to represent programs with graphs. In: 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, 30 April–3 May 2018, Conference Track Proceedings, OpenReview.net (2018)

  • Bai, X., Gao, X., Xue, B.: Particle swarm optimization based two-stage feature selection in text mining. In: 2018 IEEE Congress on Evolutionary Computation (CEC), pp. 1–8. IEEE (2018)

  • Balog, M., Gaunt, A.L., Brockschmidt, M., Nowozin, S., Tarlow, D.: Deepcoder: Learning to write programs. In: 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, 24–26 April 2017, Conference Track Proceedings, OpenReview.net (2017)

  • Bonabeau, E., Dorigo, M., Theraulaz, G.: Swarm Intelligence—From Natural to Artificial Systems. Studies in the Sciences of Complexity, Oxford University Press, Oxford (1999)

    Book  MATH  Google Scholar 

  • Brezočnik, L., Fister, I., Podgorelec, V.: Swarm intelligence algorithms for feature selection: a review. Appl. Sci. 8(9), 1521 (2018)

    Article  Google Scholar 

  • Brockschmidt, M., Allamanis, M., Gaunt, A.L., Polozov, O.: Generative code modeling with graphs. In: 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, 6–9 May 2019, OpenReview.net (2019)

  • Bunel, R., Hausknecht, M.J., Devlin, J., Singh, R., Kohli, P.: Leveraging grammar and reinforcement learning for neural program synthesis. In: 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, 30 April–3 May 2018, Conference Track Proceedings, OpenReview.net (2018)

  • Buse, R.P.L., Weimer, W.: Synthesizing API usage examples. In: 34th International Conference on Software Engineering, ICSE 2012, 2–9 June 2012, Zurich, Switzerland, pp. 782–792 (2012). https://doi.org/10.1109/ICSE.2012.6227140

  • Devlin, J., Uesato, J., Bhupatiraju, S., Singh, R., Mohamed, A., Kohli, P.: Robustfill: neural program learning under noisy I/O. In: Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6–11 Aug 2017, PMLR, Proceedings of Machine Learning Research, vol. 70, pp. 990–998 (2017)

  • D’Souza, A.R., Yang, D., Lopes, C.V.: Collective intelligence for smarter API recommendations in python. In: 16th IEEE International Working Conference on Source Code Analysis and Manipulation, SCAM 2016, Raleigh, NC, USA, 2–3 Oct 2016, pp. 51–60. IEEE Computer Society (2016). https://doi.org/10.1109/SCAM.2016.22

  • Feng, Y., Martins, R., Wang, Y., Dillig, I., Reps, T.W.: Component-based synthesis for complex apis. In: Proceedings of the 44th ACM SIGPLAN Symposium on Principles of Programming Languages, POPL 2017, Paris, France, 18–20 Jan 2017, pp. 599–612. ACM (2017)

  • Fong, S., Deb, S., Yang, X.S.: A heuristic optimization method inspired by wolf preying behavior. Neural Comput. Appl. 26(7), 1725–1738 (2015)

    Article  Google Scholar 

  • Fowkes, J.M., Sutton, C.: Parameter-free probabilistic API mining across github. In: Zimmermann, T., Cleland-Huang, J., Su, Z. (eds.) Proceedings of the 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering, FSE 2016, Seattle, WA, USA, 13–18 Nov 2016, pp. 254–265. ACM (2016). https://doi.org/10.1145/2950290.2950319

  • Gulwani, S., Harris, W.R., Singh, R.: Spreadsheet data manipulation using examples. Commun. ACM 55(8), 97–105 (2012). https://doi.org/10.1145/2240236.2240260

    Article  Google Scholar 

  • Gulwani, S., Polozov, O., Singh, R., et al.: Program synthesis. Found. Trends® Program. Lang. 4(1–2), 1–119 (2017)

    Google Scholar 

  • Han, J., Cheng, H., Xin, D., Yan, X.: Frequent pattern mining: current status and future directions. Data Min. Knowl Discov. 15(1), 55–86 (2007). https://doi.org/10.1007/s10618-006-0059-1

    Article  MathSciNet  Google Scholar 

  • Jain, D.K., Kumar, A., Sangwan, S.R., Nguyen, G.N., Tiwari, P.: A particle swarm optimized learning model of fault classification in web-apps. IEEE Access 7, 18480–18489 (2019)

    Article  Google Scholar 

  • Jain, P., Dixit, V.S.: Recommendations with context aware framework using particle swarm optimization and unsupervised learning. J. Intell. Fuzzy Syst. 36(5), 4479–4490 (2019)

    Article  Google Scholar 

  • Kennedy, J., Eberhart, R.: Particle swarm optimization. In: Proceedings of ICNN’95-International Conference on Neural Networks, vol. 4, pp. 1942–1948. IEEE (1995)

  • Kyaw, K.S., Limsiroratana, S.: Traditional and swarm intelligent based text feature selection for document classification. In: 2019 19th International Symposium on Communications and Information Technologies (ISCIT), pp. 226–231. IEEE (2019)

  • Lakhani, K.R., Garvin, D.A., Lonstein, E.: Topcoder (a): Developing software through crowdsourcing. Harvard Business School General Management Unit Case (610-032) (2010)

  • Lin, X.V., Wang, C., Zettlemoyer, L., Ernst, M.D.: Nl2bash: a corpus and semantic parser for natural language interface to the linux operating system. In: Proceedings of the Eleventh International Conference on Language Resources and Evaluation, LREC 2018, Miyazaki, Japan, 7–12 May 2018. European Language Resources Association (ELRA) (2018)

  • Ling, W., Blunsom, P., Grefenstette, E., Hermann, K.M., Kociský, T., Wang, F., Senior, A.W.: Latent predictor networks for code generation. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, 7–12 Aug 2016, Berlin, Germany, vol. 1: Long Papers. The Association for Computer Linguistics (2016) https://doi.org/10.18653/v1/p16-1057

  • Liu, B., Dong, W., Zhang, Y.: Accelerating API-based program synthesis via API usage pattern mining. IEEE Access 7, 159162–159176 (2019). https://doi.org/10.1109/ACCESS.2019.2950232

    Article  Google Scholar 

  • Liu, J., Liu, B., Dong, W., Zhang, Y., Wang, D.: How much support can API recommendation methods provide for component-based synthesis? In: 44th IEEE Annual Computers, Software, and Applications Conference, COMPSAC 2020, Madrid, Spain, 13–17 July 2020, pp. 872–881 (2020). https://doi.org/10.1109/COMPSAC48688.2020.0-155

  • Moslehi, F., Haeri, A.: A novel hybrid wrapper-filter approach based on genetic algorithm, particle swarm optimization for feature subset selection. J. Ambient. Intell. Humaniz. Comput. 11(3), 1105–1127 (2020)

    Article  Google Scholar 

  • Murali, V., Qi, L., Chaudhuri, S., Jermaine, C.: Neural sketch learning for conditional program generation. In: 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, 30 April–3 May 2018, Conference Track Proceedings, OpenReview.net (2018)

  • Peng, H., Ying, C., Tan, S., Hu, B., Sun, Z.: An improved feature selection algorithm based on ant colony optimization. IEEE Access 6, 69203–69209 (2018)

    Article  Google Scholar 

  • Peška, L., Tashu, T.M., Horváth, T.: Swarm intelligence techniques in recommender systems—a review of recent research. Swarm Evol. Comput. 48, 201–219 (2019)

    Article  Google Scholar 

  • Petrillo, F., Guéhéneuc, Y., Pimenta, M., Freitas, C.M.D.S., Khomh, F.: Swarm debugging: the collective intelligence on interactive debugging. J. Syst. Softw. 153, 152–174 (2019). https://doi.org/10.1016/j.jss.2019.04.028

    Article  Google Scholar 

  • Rabinovich, M., Stern, M., Klein, D.: Abstract syntax networks for code generation and semantic parsing. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, 30 July–4 Aug, vol. 1: Long Papers, pp. 1139–1149. Association for Computational Linguistics (2017). https://doi.org/10.18653/v1/P17-1105

  • Shen, B., Zhang, W., Zhao, H., Liang, G., Jin, Z., Wang, Q.: Intellimerge: a refactoring-aware software merging technique. Proc. ACM Program. Lang. 3(OOPSLA), 170:1-170:28 (2019). https://doi.org/10.1145/3360596

    Article  Google Scholar 

  • Shi, K., Steinhardt, J., Liang, P.: Frangel: component-based synthesis with control structures. Proc. ACM Program. Lang. 3(POPL), 73:1-73:29 (2019). https://doi.org/10.1145/3290386

    Article  Google Scholar 

  • Sun, Z., Zhu, Q., Mou, L., Xiong, Y., Li, G., Zhang, L.: A grammar-based structural CNN decoder for code generation. In: The Thirty-Third AAAI Conference on Artificial Intelligence, AAAI 2019, The Thirty-First Innovative Applications of Artificial Intelligence Conference, IAAI 2019, The Ninth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2019, Honolulu, Hawaii, USA, 27 Jan–1 Feb 1 2019, pp. 7055–7062. AAAI Press (2019). https://doi.org/10.1609/aaai.v33i01.33017055

  • Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. In: Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, 8–13 Dec 2014, Montreal, QC, Canada, pp. 3104–3112 (2014)

  • Wang, D., Dong, W., Zhang, Y.: Collective Intelligence for Smarter Neural Program Synthesis, pp. 98–104. Association for Computing Machinery, New York (2020). https://doi.org/10.1145/3417113.3423371

    Book  Google Scholar 

  • Wang, H., Wang, W., Yang, J., Yu, P.S.: (2002) Clustering by pattern similarity in large data sets. In: Proceedings of the 2002 ACM SIGMOD International Conference on Management of Data, Madison, Wisconsin, USA, 3–6 June 2002, pp. 394–405. https://doi.org/10.1145/564691.564737

  • Wang, J., Dang, Y., Zhang, H., Chen, K., Xie, T., Zhang, D.: Mining succinct and high-coverage API usage patterns from source code. In: Proceedings of the 10th Working Conference on Mining Software Repositories, MSR ’13, San Francisco, CA, USA, 18–19 May 2013, pp. 319–328 (2013). https://doi.org/10.1109/MSR.2013.6624045

  • Xu, X., Liu, C., Song, D.: Sqlnet: generating structured queries from natural language without reinforcement learning. CoRR abs arXiv:1711.04436 (2017)

  • Yang, X.: Bat algorithm for multi-objective optimisation. Int. J. Bio Inspired Comput. 3(5), 267–274 (2011). https://doi.org/10.1504/IJBIC.2011.042259

    Article  Google Scholar 

  • Yang, X.S., Karamanoglu, M., He, X.: Flower pollination algorithm: a novel approach for multiobjective optimization. Eng. Optim. 46(9), 1222–1237 (2014)

    Article  MathSciNet  Google Scholar 

  • Yin, P., Neubig, G.: A syntactic neural model for general-purpose code generation. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, 30 July–4 Aug, vol. 1: Long Papers, pp. 440–450. Association for Computational Linguistics (2017). https://doi.org/10.18653/v1/P17-1041

  • Yudong, Z., Praveen, A., Vishal, B., Saeed, B., Xuewu, Z.: Swarm intelligence and its applications (2014). https://doi.org/10.1155/2014/204294

  • Zhang, Y., Dong, W., Wang, D., Liu, B., Liu, J.: Accuracy improvement for neural program synthesis via attention mechanism and program slicing. In: 44th IEEE Annual Computers, Software, and Applications Conference, COMPSAC 2020, Madrid, Spain, 13–17 July 2020, pp. 963–972. IEEE (2020). https://doi.org/10.1109/COMPSAC48688.2020.0-146

  • Zhong, H., Xie, T., Zhang, L., Pei, J., Mei, H.: MAPO: mining and recommending API usage patterns. In: ECOOP 2009—Object-Oriented Programming, 23rd European Conference, Genoa, Italy, 6–10 July 2009. Proceedings, pp. 318–343 (2009). https://doi.org/10.1007/978-3-642-03013-0_15

  • Zohar, A., Wolf, L.: Automatic program synthesis of long programs with a learned garbage collector. In: Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018. NeurIPS 2018, 3–8 Dec 2018, pp. 2098–2107. Montréal, Canada (2018)

Download references

Acknowledgements

This work was supported by National Natural Science Foundation of China (Nos. 61690203, 62032019).

Author information

Authors and Affiliations

  1. College of Computer, National University of Defense Technology, Changsha, China

    Yating Zhang, Daiyan Wang & Wei Dong

Authors
  1. Yating Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Daiyan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wei Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yating Zhang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, Y., Wang, D. & Dong, W. MerIt: improving neural program synthesis by merging collective intelligence. Autom Softw Eng 29, 45 (2022). https://doi.org/10.1007/s10515-022-00343-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10515-022-00343-z

Keywords

Search

Navigation