Abstract
One assignment of a front-end engineer is to transform the User Interface (UI) design drafts to arrangements with program compatibilities. To furnish the customer requirement, engineers usually need to consider various factors for different terminal devices such as a wide range of screen sizes. Such process usually contains a lot of tedious work and much of them is repetitive which can be done by machine. In this paper, we propose Visual Compiler, a tool trying to bridge the gap between UI design draft and front-end code. We combine expert heuristics and techniques in computer vision to improve the readability and usability of generated code, making the generation process more intelligently. In practice, users just need to input the design drafts obtained from UI designers, then our method can produce clean and well-organized front-end code. Compared with existing approaches, Visual Compiler is able to reduce the size of output files by approximately 80% while maintaining the same appearance as in the original draft. We hope our tool can help UI designers and front-end developers collaborate more efficiently.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473 (2014)
Beltramelli, T.: pix2code: generating code from a graphical user interface screenshot. In: Proceedings of the ACM SIGCHI Symposium on Engineering Interactive Computing Systems, p. 3. ACM (2018)
Chen, C., Su, T., Meng, G., Xing, Z., Liu, Y.: From UI design image to GUI skeleton: a neural machine translator to bootstrap mobile GUI implementation. In: Proceedings of the 40th International Conference on Software Engineering, pp. 665–676. ACM (2018)
He, K., Zhang,, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)
Lin, T.-Y., Dollár, P., Girshick, R., He, K., Hariharan, B., Belongie, S.: Feature pyramid networks for object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2117–2125 (2017)
Nguyen, T.A., Csallner, C.: Reverse engineering mobile application user interfaces with REMAUI. In: 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE), pp. 248–259. IEEE (2015)
Ren, S., He., K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Advances in Neural Information Processing Systems, pp. 91–99 (2015)
Robinson, A.: Sketch2Code: generating a website from a paper mockup. arXiv preprint arXiv:1905.13750 (2019)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. In: Advances in Neural Information Processing Systems, pp. 3104–3112 (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Pan, J. et al. (2020). Visual Compiler: Towards Translating Digital UI Design Draft to Front-End Code Automatically. In: Yamamoto, S., Mori, H. (eds) Human Interface and the Management of Information. Interacting with Information. HCII 2020. Lecture Notes in Computer Science(), vol 12185. Springer, Cham. https://doi.org/10.1007/978-3-030-50017-7_28
Download citation
DOI: https://doi.org/10.1007/978-3-030-50017-7_28
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-50016-0
Online ISBN: 978-3-030-50017-7
eBook Packages: Computer ScienceComputer Science (R0)