Abstract
Code completion is an integral component of modern integrated development environments, as it not only facilitates the software development process but also enhances the quality of software products. By leveraging large-scale codes to learn the probability distribution among code token units, deep learning methods have demonstrated significant improvements in the accuracy of token unit recommendations. However, the effectiveness of code completion with deep learning techniques is hindered by information loss. To alleviate the above problem, we proposed a code language model which combines the pointer network and Transformer-XL network to overcome the limitations of existing approaches in code completion. The proposed model takes as input the original code fragment and its corresponding abstract syntax tree and leverages the Transformer-XL model as the basis model for capturing long-term dependencies. Furthermore, we integrate a pointer network as a local component to predict the out-of-vocabulary words. The proposed method is evaluated on real PY150 and JS150 datasets. The comparative experimental results demonstrate the effectiveness of our model in improving the accuracy of the code completion task at the token unit level.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Izadi, M., Gismondi, R., Gousios, G.: CodeFill: multi-token code completion by jointly learning from structure and naming sequences. In: Proceedings of the 44th International Conference on Software Engineering, pp. 401–412 (2022)
Yang, Y., Xiang, C.: Improve language modelling for code completion by tree language model with tree encoding of context (S), pp. 675–777 (2019)
Fang, L., Huang, Z., Zhou, Y., Chen., T.: Adaptive code completion with meta-learning. In: Proceedings of the 12th Asia-Pacific Symposium on Internetware, pp. 116–125 (2020)
Popov, A., Orekhov, D., Litvinov, D.: Time-efficient code completion model for the R programming language. In: Proceedings of the 1st Workshop on Natural Language Processing for Programming (NLP4Prog 2021), pp. 34–39 (2021)
Kyaw, H.H.S., Funabiki, N., Kuribayashi, M.: An implementation of offline answering function for code completion problem in PLAS. In: 2021 IEEE 3rd Global Conference on Life Sciences and Technologies (LifeTech), pp. 162–165 (2021)
Raychev, V., Vechev, M., Yahav, E.: Code completion with statistical language models. In: Proceedings of the 35th ACM SIGPLAN Conference on Programming Language Design and Implementation, pp. 419–428 (2014)
Robbes, R., Lanza, M.: How program history can improve code completion. In: 2008 23rd IEEE/ACM International Conference on Automated Software Engineering, pp. 317–326 (2008)
Proksch, S., Lerch, J., Mezini, M.: Intelligent code completion with bayesian networks. ACM Trans. Softw. Eng. Methodol. 25(1), 1–31 (2015)
Lee, Y.Y., Harwell, S., Khurshid, S.: Temporal code completion and navigation. In: 2013 35th International Conference on Software Engineering (ICSE), pp. 1181–1184 (2013)
Nguyen, A.T., Nguyen, H.A., Nguyen, T.T.: GraPacc: a graph-based pattern-oriented, context-sensitive code completion tool. In: 2012 34th International Conference on Software Engineering, pp. 1407–1410 (2012)
Omori, T., Kuwabara, H., Maruyama, K.: A study on repetitiveness of code completion operations. In: 2012 28th IEEE International Conference on Software Maintenance (ICSM), pp. 584–587 (2012)
Zhang, X., Liu, J., Shi, M.: A parallel deep learning-based code clone detection model. J. Parallel Distrib. Comput. 181, 104747 (2023)
Guo, D., Lu, S., Duan, N.: UnixCoder: unified cross-modal pre-training for code representation. arXiv preprint arXiv:2203.03850 (2022)
Shi, J., Yang, Z., He, J., Xu, B., Lo, D.: Can identifier splitting improve open-vocabulary language model of code? In: 2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER), pp. 1134–1138 (2022)
Hindle, A., Barr, E.T., Gabel, M.: On the naturalness of software. Commun. ACM 59(5), 122–131 (2016)
Tu, Z., Su, Z., Devanbu, P.: On the localness of software. In: Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering, pp. 269–280 (2014)
Franks, C., Tu, Z., Devanbu, P.: CACHECA: a cache language model based code suggestion tool. In: 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering, vol. 2, pp. 705–708 (2015)
Henkel, J., Lahiri, S.K., Liblit, B.: Code vectors: understanding programs through embedded abstracted symbolic traces. In: Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pp. 163–174 (2018)
Vinyals, O., Fortunato, M., Jaitly, N.: Pointer networks. In: Advances in Neural Information Processing Systems, vol. 28 (2015)
Bhoopchand, A., Rocktäschel, T., Barr, E.: Learning python code suggestion with a sparse pointer network. arXiv preprint arXiv:1611.08307 (2016)
Li, J., Wang, Y., Lyu, M.R.: Code completion with neural attention and pointer networks. arXiv preprint arXiv:1711.09573 (2017)
Yang, H., Kuang, L.: CCMC: code completion with a memory mechanism and a copy mechanism. In: Evaluation and Assessment in Software Engineering, pp. 129–138 (2021)
Tay, Y., Dehghani, M., Bahri, D., Metzler, D.: Efficient transformers: a survey. ACM Comput. Surv. 55(6), 1–28 (2022)
Dowdell, T., Zhang, H.: Language modelling for source code with transformer-XL. arXiv preprint arXiv:2007.15813 (2020)
Liu, C., Wang, X., Shin, R., Gonzalez, J.E., Song, D.: Neural code completion (2016)
Kim, S., Zhao, J., Tian, Y., Chandra, S.: Code prediction by feeding trees to transformers. In: 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE), pp. 150–162 (2021)
Liu, F., Li, G., Wei, B., Xia, X., Fu, Z., Jin, Z.: A self-attentional neural architecture for code completion with multi-task learning. In: Proceedings of the 28th International Conference on Program Comprehension, pp. 37–47 (2020)
Acknowledgements
This work was supported by the National Natural Science Foundation of China (61872139).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Zhang, X., Liu, J., Long, T., Hu, H. (2024). A Code Completion Approach Combining Pointer Network and Transformer-XL Network. In: Gao, H., Wang, X., Voros, N. (eds) Collaborative Computing: Networking, Applications and Worksharing. CollaborateCom 2023. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 561. Springer, Cham. https://doi.org/10.1007/978-3-031-54521-4_17
Download citation
DOI: https://doi.org/10.1007/978-3-031-54521-4_17
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-54520-7
Online ISBN: 978-3-031-54521-4
eBook Packages: Computer ScienceComputer Science (R0)