Abstract
This paper presents and evaluates co-creative drawing scenarios in which a user is asked to provide a small hand-drawn pattern which then is interactively extended with the support of a trained neural model. We show that it is possible to use one-shot trained Transformer Neural Networks to generate stroke-based images and that these trained models can successfully be used for design assisting tasks.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Carlier, A., Danelljan, M., Alahi, A., Timofte, R.: Deepsvg: A hierarchical generative network for vector graphics animation (2020)
Carroll, E.A., Latulipe, C.: The creativity support index. In: CHI ’09 Extended Abstracts on Human Factors in Computing Systems. p. 4009–4014. CHI EA ’09, Association for Computing Machinery, New York, NY, USA (2009). https://doi.org/10.1145/1520340.1520609
Davis, N., Hsiao, C.P., Yashraj Singh, K., Li, L., Magerko, B.: Empirically studying participatory sense-making in abstract drawing with a co-creative cognitive agent. In: Proceedings of the 21st International Conference on Intelligent User Interfaces. p. 196–207. IUI ’16, Association for Computing Machinery, New York, NY, USA (2016). https://doi.org/10.1145/2856767.2856795
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
Fan, J.E., Dinculescu, M., Ha, D.: Collabdraw: An environment for collaborative sketching with an artificial agent. In: Proceedings of the 2019 on Creativity and Cognition. p. 556–561. Association for Computing Machinery, New York, NY, USA (2019). https://doi.org/10.1145/3325480.3326578
Fish, J., Scrivener, S.: Amplifying the mind’s eye: sketching and visual cognition. Leonardo 23(1), 117–126 (1990)
Ha, D., Eck, D.: A neural representation of sketch drawings. arXiv preprint arXiv:1704.03477 (2017)
Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Levy, O., Goldberg, Y.: Neural word embedding as implicit matrix factorization. In: Advances in neural information processing systems. pp. 2177–2185 (2014)
Lin, Y., Guo, J., Chen, Y., Yao, C., Ying, F.: It is your turn: Collaborative ideation with a co-creative robot through sketch. In: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. p. 1–14. CHI ’20, Association for Computing Machinery, New York, NY, USA (2020). https://doi.org/10.1145/3313831.3376258
Macedo, L., Cardoso, A.: Assessing creativity: the importance of unexpected novelty. Structure 1(C2), C3 (2002)
Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I.: Language models are unsupervised multitask learners. OpenAI Blog 1(8), 9 (2019)
Salomon, D.: Curves and surfaces for computer graphics. Springer Science & Business Media (2007)
Schneider, P.J.: An algorithm for automatically fitting digitized curves. In: Graphics gems. pp. 612–626. Academic Press Professional, Inc. (1990)
Serban, I.V., Sordoni, A., Lowe, R., Charlin, L., Pineau, J., Courville, A., Bengio, Y.: A hierarchical latent variable encoder-decoder model for generating dialogues. In: Thirty-First AAAI Conference on Artificial Intelligence (2017)
Shaham, T.R., Dekel, T., Michaeli, T.: Singan: Learning a generative model from a single natural image. In: Proceedings of the IEEE International Conference on Computer Vision. pp. 4570–4580 (2019)
Shneiderman, B.: Creativity support tools: Accelerating discovery and innovation. Communications of the ACM 50(12), 20–32 (2007)
Suwa, M., Gero, J.S., Purcell, T.: The roles of sketches in early conceptual design processes. In: Proceedings of Twentieth Annual Meeting of the Cognitive Science Society. pp. 1043–1048. Lawrence Erlbaum Hillsdale, New Jersey (1998)
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. In: Advances in neural information processing systems. pp. 5998–6008 (2017)
Voita, E., Talbot, D., Moiseev, F., Sennrich, R., Titov, I.: Analyzing multi-head self-attention: Specialized heads do the heavy lifting, the rest can be pruned. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. pp. 5797–5808. Association for Computational Linguistics, Florence, Italy (Jul 2019). https://doi.org/10.18653/v1/P19-1580, https://www.aclweb.org/anthology/P19-1580
Wang, J., He, Y., Tian, H., Cai, H.: Retrieving 3d cad model by freehand sketches for design reuse. Advanced Engineering Informatics 22(3), 385–392 (2008)
Wieluch, S., Schwenker, F.: Strokecoder: Path-based image generation from single examples using transformers (2020)
Xu, P., Joshi, C.K., Bresson, X.: Multi-graph transformer for free-hand sketch recognition. arXiv preprint arXiv:1912.11258 (2019)
Xu, P., Song, Z., Yin, Q., Song, Y.Z., Wang, L.: Deep self-supervised representation learning for free-hand sketch. arXiv preprint arXiv:2002.00867 (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Wieluch, S., Schwenker, F. (2021). Co-creative Drawing with One-Shot Generative Models. In: Romero, J., Martins, T., Rodríguez-Fernández, N. (eds) Artificial Intelligence in Music, Sound, Art and Design. EvoMUSART 2021. Lecture Notes in Computer Science(), vol 12693. Springer, Cham. https://doi.org/10.1007/978-3-030-72914-1_31
Download citation
DOI: https://doi.org/10.1007/978-3-030-72914-1_31
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-72913-4
Online ISBN: 978-3-030-72914-1
eBook Packages: Computer ScienceComputer Science (R0)