Shuai Wang
ABSTRACT
With the fast development of Information Technology, program software and mobile applications have been widely used in the world, and are playing important roles in human's daily life. Thus, writing programming codes has been important work in many fields. however, it is a hard and time-cost task which presents a great amount of workload to programmers. To make programmers' work easier, intelligent code completion models have been a popular research topic in recent years. This paper designs Deep Learning based models to automatically complete programming codes, which are LSTM-based neural networks, and are combined with several techniques such as Word Embedding models in NLP (Natural Language Processing), and Multihead Attention Mechanism. Moreover, in the models, this paper raises a new algorithm of generating input sequences from partial AST (Abstract Syntax Tree) that have most relevance with nodes to be predicted which is named as RZT (Reverse Zig-zag Traverse) Algorithm, and is the first work of applying Multihead Attention Block into this task. This paper makes insight into codes of several different programming languages, and the models this paper presents show good performances in accuracy comparing with the state-of-art models.
- Allamanis, Miltiadis, Barr, Earl T, Devanbu, Premkumar, et al. 2018. A survey of machine learning for big code and naturalness. ACM Computing Surveys (CSUR), 51(4), 81.Google Scholar
Digital Library
- Allamanis, Miltiadis, Brockschmidt, Marc, and Khademi, Mahmoud. 2017. Learning to represent programs with graphs. arXiv preprint arXiv:1711.00740.Google Scholar
- Allamanis, Miltiadis, and Sutton, Charles. 2013. Mining source code repositories at massive scale using language modeling. In Proceedings of the 10th Working Conference on Mining Software Repositories. 207--216.Google ScholarDigital Library
- Amodio, Matthew, Chaudhuri, Swarat, and Reps, Thomas. 2017. Neural Attribute Machines for Program Generation. arXiv preprint arXiv:1705.09231.Google Scholar
- [5] Bahdanau, Dzmitry, Cho, Kyunghyun, and Bengio, Yoshua. 2014. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473.Google Scholar
- Bhoopchand, Avishkar, Rocktäschel, Tim, Barr, Earl, et al. 2016. Learning Python code suggestion with a sparse pointer network. arXiv preprint arXiv:1611.08307.Google Scholar
- Bielik, Pavol, Raychev, Veselin, and Vechev, Martin. 2016. PHOG: probabilistic model for code. In International Conference on Machine Learning. 2933--2942.Google Scholar
- Bojanowski, Piotr, Grave, Edouard, Joulin, Armand, et al. 2016. Enriching word vectors with subword information. arXiv preprint arXiv:1607.04606.Google Scholar
- Corley, Christopher S, Damevski, Kostadin, and Kraft, Nicholas A. 2015. Exploring the use of deep learning for feature location. In 2015 IEEE International Conference on Software Maintenance and Evolution (ICSME). 556--560.Google ScholarDigital Library
- Hindle, Abram, Barr, Earl T, Su, Zhendong, et al. 2012. On the naturalness of software. In Software Engineering (ICSE), 2012 34th International Conference on. 837--847.Google ScholarCross Ref
- Hochreiter, Sepp, and Schmidhuber, Jürgen. 1997. Long short-term memory. Neural computation, 9(8), 1735--1780.Google Scholar
- Hou, Daqing, and Pletcher, David M. 2011. An evaluation of the strategies of sorting, filtering, and grouping API methods for code completion. In Software Maintenance (ICSM), 2011 27th IEEE International Conference on. 233--242.Google ScholarDigital Library
- Li, Jian, Wang, Yue, King, Irwin, et al. 2017. Code Completion with Neural Attention and Pointer Networks. arXiv preprint arXiv:1711.09573.Google Scholar
- Li, Zhenmin, and Zhou, Yuanyuan. 2005. PR-Miner: automatically extracting implicit programming rules and detecting violations in large software code. In ACM SIGSOFT Software Engineering Notes. 306--315.Google Scholar
- Liu, Chang, Wang, Xin, Shin, Richard, et al. 2016. Neural Code Completion.Google Scholar
- Luong, Minh Thang, Pham, Hieu, and Manning, Christopher D. 2015. Effective Approaches to Attention-based Neural Machine Translation. Computer Science.Google Scholar
- Maddison, Chris, and Tarlow, Daniel. 2014. Structured generative models of natural source code. In International Conference on Machine Learning. 649--657.Google Scholar
- Mikolov, Tomas, Chen, Kai, Corrado, Greg, et al. 2013. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781.Google Scholar
- Mikolov, Tomas, Sutskever, Ilya, Chen, Kai, et al. 2013. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems. 3111--3119.Google Scholar
- Mou, Lili, Li, Ge, Liu, Yuxuan, et al. 2014. Building program vector representations for deep learning. arXiv preprint arXiv:1409.3358.Google Scholar
- Nguyen, Anh Tuan, Nguyen, Tung Thanh, Nguyen, Hoan Anh, et al. 2012. Graph-based pattern-oriented, context-sensitive source code completion. In Software Engineering (ICSE), 2012 34th International Conference on. 69--79.Google ScholarCross Ref
- Nguyen, Tam The, Pham, Hung Viet, Vu, Phong Minh, et al. 2015. Recommending API usages for mobile apps with hidden markov model. In 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE). 795--800.Google ScholarDigital Library
- Nguyen, Trong Duc, Nguyen, Anh Tuan, and Nguyen, Tien N. 2016. Mapping API elements for code migration with vector representations. In Software Engineering Companion (ICSE-C), IEEE/ACM International Conference on. 756--758.Google Scholar
- Nguyen, Trong Duc, Nguyen, Anh Tuan, Phan, Hung Dang, et al. 2017. Exploring API embedding for API usages and applications. In Software Engineering (ICSE), 2017 IEEE/ACM 39th International Conference on. 438--449.Google ScholarDigital Library
- Proksch, Sebastian, Lerch, Johannes, and Mezini, Mira. 2015. Intelligent code completion with Bayesian networks. ACM Transactions on Software Engineering and Methodology (TOSEM), 25(1), 3.Google ScholarDigital Library
- Raychev, Veselin, Vechev, Martin, and Krause, Andreas. 2015. Predicting program properties from big code. In ACM SIGPLAN Notices. 111--124.Google Scholar
- Raychev, Veselin, Vechev, Martin, and Yahav, Eran. 2014. Code completion with statistical language models. In Acm Sigplan Notices. 419--428.Google Scholar
- Thung, Ferdian, Wang, Shaowei, Lo, David, et al. 2013. Automatic recommendation of API methods from feature requests. In Proceedings of the 28th IEEE/ACM International Conference on Automated Software Engineering. 290--300.Google ScholarDigital Library
- Vaswani, Ashish, Shazeer, Noam, Parmar, Niki, et al. 2017. Attention is all you need. In Advances in Neural Information Processing Systems. 5998--6008.Google Scholar
- White, Martin, Vendome, Christopher, Linares-Vásquez, Mario, et al. 2015. Toward deep learning software repositories. In Proceedings of the 12th Working Conference on Mining Software Repositories. 334--345.Google ScholarDigital Library
- Xie, Tao, and Pei, Jian. 2006. MAPO: Mining API usages from open source repositories. In Proceedings of the 2006 international workshop on Mining software repositories. 54--57.Google ScholarDigital Library
Index Terms
- Deep Learning Based Code Completion Models for Programming Codes
Recommendations
A rule-based and object-oriented AI programming language
This paper presents a new artificial intelligence programming language called ROOP which is machine independent and has the ability to deal with logical programming in general. ROOP is a multiple paradigm language which combines rule-based, procedure, ...
SF1: introduction to ada
SIGAda '08: Proceedings of the 2008 ACM annual international conference on SIGAda annual international conferenceLevel - Beginner, but attendees should have some experience with a high-level programming language.
This tutorial is designed for those who have some familiarity with a programming language, but who are new to Ada. In the morning, we will discuss the ...
SF1: introduction to ada
Level - Beginner, but attendees should have some experience with a high-level programming language.
This tutorial is designed for those who have some familiarity with a programming language, but who are new to Ada. In the morning, we will discuss the ...
Comments