Abstract
In recent years, we have witnessed the spread of computer graphics techniques, used as a background map for movies and video games. Nevertheless, when creating 3D models with conventional computer graphics software, it is necessary for the user to manually change the placement and size. This requires expertise of computer graphics architecture and operations, which is time demanding. Applying Artificial Intelligence (AI) to games is currently an established research field. Starting from such premises, in this paper MONstEr (dEEp lEArNiNG GENErAtiON AssEt) a system for the automatic generation of virtual asset for videogames is presented. MONstEr exploits the principle of Deep Learning (DL) and in particular Generative models to automatically design new assets for videogames. The DL pipeline is the core of this system and it is based on a Deep Convolutional Generative Adversarial Network followed by Pixel2Mesh architecture for the 3D models generation. The approach was applied and tested on a newly collected dataset of images, “GameAssetDataset” which comprises characters representation extracted thanks a web crawler algorithm specifically developed for its acquisition. MONstEr expedites the implementation of solutions for new gamining environments, requiring only a small intervention in the 3D construction to insert the object in the game scene.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Browne, C.B.: Automatic generation and evaluation of recombination games. Ph.D. thesis, Queensland University of Technology (2008)
Gadelha, M., Maji, S., Wang, R.: 3D shape induction from 2D views of multiple objects. In: 2017 International Conference on 3D Vision (3DV), pp. 402–411. IEEE (2017)
Gonzalez, A., Guzdial, M., Ramos, F.: Generating gameplay-relevant art assets with transfer learning. arXiv preprint arXiv:2010.01681 (2020)
Goodfellow, I., et al.: Generative adversarial nets. Adv. Neural Inf. Process. Syst. 27 (2014)
Hendrikx, M., Meijer, S., Van Der Velden, J., Iosup, A.: Procedural content generation for games: a survey. ACM Trans. Multi. Comput. Commun. Appl. (TOMM) 9(1), 1–22 (2013)
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., Hochreiter, S.: Gans trained by a two time-scale update rule converge to a local nash equilibrium. Adv. Neural Inf. Process. Syst. 30 (2017)
Karp, R., Swiderska-Chadaj, Z.: Automatic generation of graphical game assets using GAN. In: 2021 7th International Conference on Computer Technology Applications, pp. 7–12 (2021)
Kingma, D.P., Welling, M.: Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 (2013)
Kingma, D.P., Welling, M.: An introduction to variational autoencoders. arXiv preprint arXiv:1906.02691 (2019)
Liu, Z., Luo, P., Wang, X., Tang, X.: Deep learning face attributes in the wild. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3730–3738 (2015)
Mathieu, M., Couprie, C., LeCun, Y.: Deep multi-scale video prediction beyond mean square error. arXiv preprint arXiv:1511.05440 (2015)
Maturana, D., Scherer, S.: Voxnet: a 3D convolutional neural network for real-time object recognition. In: 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 922–928. IEEE (2015)
Nigam, A.: Web crawling algorithms. Int. J. Comput. Sci. Artif. Intell. 4(3), 63 (2014)
Radford, A., Metz, L., Chintala, S.: Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434 (2015)
Serpa, Y.R., Rodrigues, M.A.F.: Towards machine-learning assisted asset generation for games: a study on pixel art sprite sheets. In: 2019 18th Brazilian Symposium on Computer Games and Digital Entertainment (SBGames), pp. 182–191. IEEE (2019)
Summerville, A., et al.: Procedural content generation via machine learning (PCGML). IEEE Trans. Games 10(3), 257–270 (2018)
Togelius, J., Yannakakis, G.N., Stanley, K.O., Browne, C.: Search-based procedural content generation: a taxonomy and survey. IEEE Trans. Comput. Intell. AI Games 3(3), 172–186 (2011)
Torrado, R.R., Khalifa, A., Green, M.C., Justesen, N., Risi, S., Togelius, J.: Bootstrapping conditional GANS for video game level generation. In: 2020 IEEE Conference on Games (CoG), pp. 41–48. IEEE (2020)
Wang, N., Zhang, Y., Li, Z., Fu, Y., Liu, W., Jiang, Y.G.: Pixel2mesh: generating 3D mesh models from single RGB images. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 52–67 (2018)
Wu, J., Zhang, C., Xue, T., Freeman, W.T., Tenenbaum, J.B.: Learning a probabilistic latent space of object shapes via 3D generative-adversarial modeling. In: Proceedings of the 30th International Conference on Neural Information Processing Systems, pp. 82–90 (2016)
Wu, Z., et al.: 3D shapenets: a deep representation for volumetric shapes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1912–1920 (2015)
Yang, L.C., Chou, S.Y., Yang, Y.H.: Midinet: a convolutional generative adversarial network for symbolic-domain music generation. arXiv preprint arXiv:1703.10847 (2017)
Yu, F., Seff, A., Zhang, Y., Song, S., Funkhouser, T., Xiao, J.: LSUN: construction of a large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv:1506.03365 (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Brocchini, M. et al. (2022). MONstEr: A Deep Learning-Based System for the Automatic Generation of Gaming Assets. In: Mazzeo, P.L., Frontoni, E., Sclaroff, S., Distante, C. (eds) Image Analysis and Processing. ICIAP 2022 Workshops. ICIAP 2022. Lecture Notes in Computer Science, vol 13373. Springer, Cham. https://doi.org/10.1007/978-3-031-13321-3_25
Download citation
DOI: https://doi.org/10.1007/978-3-031-13321-3_25
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-13320-6
Online ISBN: 978-3-031-13321-3
eBook Packages: Computer ScienceComputer Science (R0)