skip to main content
research-article

Sketchar: Supporting Character Design and Illustration Prototyping Using Generative AI

Published: 15 October 2024 Publication History

Abstract

Character design in games involves interdisciplinary collaborations, typically between designers who create the narrative content, and illustrators who realize the design vision. However, traditional workflows face challenges in communication due to the differing backgrounds of illustrators and designers, the latter with limited artistic abilities. To overcome these challenges, we created Sketchar, a Generative AI (GenAI) tool that allows designers to prototype game characters and generate images based on conceptual input, providing visual outcomes that can give immediate feedback and enhance communication with illustrators' next step in the design cycle. We conducted a mixed-method study to evaluate the interaction between game designers and Sketchar. We showed that the reference images generated in co-creating with Sketchar fostered refinement of design details and can be incorporated into real-world workflows. Moreover, designers without artistic backgrounds found the Sketchar workflow to be more expressive and worthwhile. This research demonstrates the potential of GenAI in enhancing interdisciplinary collaboration in the game industry, enabling designers to interact beyond their own limited expertise.

Supplemental Material

MP4 File
Supplemental video
ZIP File
1. The .srt file is the subtitle for the final video. 2. The .mp4 file is another video for demonstrating our system Sketchar.

References

[1]
[n. d.]. Lexica. https://lexica.art/
[2]
AlgorithmWatch. 2024. Image Generators Reinforce Stereotypes, Lack Diversity: A Study by AlgorithmWatch. AlgorithmWatch (2024). https://algorithmwatch.org/en/image-generators-stereotypes-diversity/ Accessed: 2024-05- 26.
[3]
Janna Anderson and Lee Rainie. 2023. As AI spreads, experts predict the best and worst changes in digital life by 2035. (2023).
[4]
Tyler Angert, Miroslav Ivan Suzara, Jenny Han, Christopher Lawrence Pondoc, and Hariharan Subramonyam. 2023. Spellburst: A Node-based Interface for Exploratory Creative Coding with Natural Language Prompts. arXiv preprint arXiv:2308.03921 (2023).
[5]
Rahul Arora, Ishan Darolia, Vinay P Namboodiri, Karan Singh, and Adrien Bousseau. 2017. Sketchsoup: Exploratory ideation using design sketches. In Computer Graphics Forum, Vol. 36. Wiley Online Library, 302--312.
[6]
Bloomberg. 2023. Generative AI and Bias: A Deep Dive by Bloomberg. Bloomberg (2023). https://www.bloomberg. com/graphics/2023-generative-ai-bias/ Accessed: 2024-05--26.
[7]
Ali Borji. 2023. Generated Faces in the Wild: Quantitative Comparison of Stable Diffusion, Midjourney and DALL-E 2. arXiv:2210.00586 [cs.CV]
[8]
Josiah D Boucher, Gillian Smith, and Yunus Dogan Telliel. 2024. Is Resistance Futile?: Early Career Game Developers, Generative AI, and Ethical Skepticism. In Proceedings of the CHI Conference on Human Factors in Computing Systems. 1--13.
[9]
Gwern Branwen. 2020. GPT-3 creative fiction. (2020).
[10]
Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative research in psychology 3, 2 (2006), 77--101.
[11]
Alice Cai, Steven R Rick, Jennifer L Heyman, Yanxia Zhang, Alexandre Filipowicz, Matthew Hong, Matt Klenk, and Thomas Malone. 2023. DesignAID: Using Generative AI and Semantic Diversity for Design Inspiration. In Proceedings of The ACM Collective Intelligence Conference. 1--11.
[12]
Cassiano Canheti, Flávio Andalo, and Milton Luiz Horn Vieira. 2019. Case study: Game character creation process. In Advances in Human Factors in Wearable Technologies and Game Design: Proceedings of the AHFE 2018 International Conferences on Human Factors and Wearable Technologies, and Human Factors in Game Design and Virtual Environments, Held on July 21--25, 2018, in Loews Sapphire Falls Resort at Universal Studios, Orlando, Florida, USA 9. Springer, 343--354.
[13]
Erin Cherry and Celine Latulipe. 2014. Quantifying the creativity support of digital tools through the creativity support index. ACM Transactions on Computer-Human Interaction (TOCHI) 21, 4 (2014), 1--25.
[14]
John Joon Young Chung. 2022. Artistic user expressions in AI-powered creativity support tools. In Adjunct Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology. 1--4.
[15]
John Joon Young Chung and Eytan Adar. 2023. Artinter: AI-powered Boundary Objects for Commissioning Visual Arts. (2023).
[16]
John Joon Young Chung, Shiqing He, and Eytan Adar. 2022. Artist support networks: Implications for future creativity support tools. In Designing Interactive Systems Conference. 232--246.
[17]
John Joon Young Chung, Wooseok Kim, Kang Min Yoo, Hwaran Lee, and Eytan Adar. [n. d.]. TaleBrush: Sketching Stories with Generative Pretrained Language Models. In CHI Conference on Human Factors in Computing Systems (New Orleans LA USA, 2022-04--29). ACM, 1--19. https://doi.org/10.1145/3491102.3501819
[18]
Niklas Deckers, Maik Fröbe, Johannes Kiesel, Gianluca Pandolfo, Christopher Schröder, Benno Stein, and Martin Potthast. 2023. The Infinite Index: Information Retrieval on Generative Text-To-Image Models. In Proceedings of the 2023 Conference on Human Information Interaction and Retrieval. 172--186.
[19]
Ferda Gül Aydin Emekligil and Ilkay Öksüz. 2022. Game Character Generation with Generative Adversarial Networks. In 2022 30th Signal Processing and Communications Applications Conference (SIU). 1--4. https://doi.org/10.1109/SIU55565. 2022.9864747
[20]
Peter Fernandez. 2022. Technology behind text to image generators. Library Hi Tech News 39, 10 (2022), 1--4.
[21]
Elise Fogelström. 2013. Investigation of shapes and colours as elements of character design.
[22]
Forbes Middle East. 2023. ChatGPT Reportedly Blocked on Chinese Social Media Apps as Beijing Claims AI is Used to Spread Propaganda. https://www.forbesmiddleeast.com/innovation/artificial-intelligence-machine-learning/chatgptreportedly- blocked-on-chinese-social-media-appsas-beijing-claims-ai-is-used-to-spread-propaganda Accessed: 2024- 05--31.
[23]
Fiona Fui-Hoon Nah, Ruilin Zheng, Jingyuan Cai, Keng Siau, and Langtao Chen. 2023. Generative AI and ChatGPT: Applications, challenges, and AI-human collaboration., 277--304 pages.
[24]
Lilian Guo and Anand Bhojan. 2022. Converting Nebulous Ideas to Reality--A Deep Learning Tool for Conditional Synthesis of Character Designs. In International Conference on Computer, Communication, and Signal Processing. Springer, 75--89.
[25]
Ariel Han and Zhenyao Cai. 2023. Design implications of generative AI systems for visual storytelling for young learners. In Proceedings of the 22nd Annual ACM Interaction Design and Children Conference. 470--474.
[26]
Yuanning Han, Ziyi Qiu, Jiale Cheng, and RAY LC. 2024. When Teams Embrace AI: Human Collaboration Strategies in Generative Prompting in a Creative Design Task. In Proceedings of the CHI Conference on Human Factors in Computing Systems. 1--14.
[27]
Seunghoon Hong, Dingdong Yang, Jongwook Choi, and Honglak Lee. 2018. Inferring semantic layout for hierarchical text-to-image synthesis. In Proceedings of the IEEE conference on computer vision and pattern recognition. 7986--7994.
[28]
John Howard. 2019. Artificial intelligence: Implications for the future of work. American journal of industrial medicine 62, 11 (2019), 917--926.
[29]
Forrest Huang and John F Canny. 2019. Sketchforme: Composing sketched scenes from text descriptions for interactive applications. In Proceedings of the 32nd annual ACM symposium on user interface software and technology. 209--220.
[30]
Forrest Huang, Eldon Schoop, David Ha, and John Canny. 2020. Scones: towards conversational authoring of sketches. In Proceedings of the 25th International Conference on Intelligent User Interfaces. 313--323.
[31]
Angel Hsing-Chi Hwang. [n. d.]. Too Late to be Creative? AI-Empowered Tools in Creative Processes. In Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems (New York, NY, USA, 2022-04--28) (CHI EA '22). Association for Computing Machinery, 1--9. https://doi.org/10.1145/3491101.3503549
[32]
Nanna Inie, Jeanette Falk, and Steve Tanimoto. 2023. Designing participatory ai: Creative professionals? worries and expectations about generative ai. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1--8.
[33]
Daphne Ippolito, Ann Yuan, Andy Coenen, and Sehmon Burnam. 2022. Creative writing with an ai-powered writing assistant: Perspectives from professional writers. arXiv preprint arXiv:2211.05030 (2022).
[34]
Youngseung Jeon, Seungwan Jin, Patrick C. Shih, and Kyungsik Han. [n. d.]. FashionQ: An AI-Driven Creativity Support Tool for Facilitating Ideation in Fashion Design. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (New York, NY, USA, 2021-05-07) (CHI '21). Association for Computing Machinery, 1--18. https://doi.org/10.1145/3411764.3445093
[35]
Hyung-Kwon Ko, Subin An, Gwanmo Park, Seung Kwon Kim, Daesik Kim, Bohyoung Kim, Jaemin Jo, and Jinwook Seo. 2022. We-toon: A Communication Support System between Writers and Artists in Collaborative Webtoon Sketch Revision. In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology. 1--14.
[36]
Hyung-Kwon Ko, Gwanmo Park, Hyeon Jeon, Jaemin Jo, Juho Kim, and Jinwook Seo. 2023. Large-scale text-to-image generation models for visual artists? creative works. In Proceedings of the 28th International Conference on Intelligent User Interfaces. 919--933.
[37]
Marianne Krawczyk and Jeannie Novak. 2006. Game development essentials: Game story & character development. (No Title) (2006).
[38]
Max Kreminski and Chris Martens. 2022. Unmet creativity support needs in computationally supported creative writing. In Proceedings of the First Workshop on Intelligent and Interactive Writing Assistants (In2Writing 2022). 74--82.
[39]
Hagung Kuntjara, Betha Almanfaluthi, et al. 2017. Character design in games analysis of character design theory. Journal of Games, Game Art, and Gamification 2, 2 (2017).
[40]
Elisa Kwon, Vivek Rao, and Kosa Goucher-Lambert. 2023. Understanding inspiration: Insights into how designers discover inspirational stimuli using an AI-enabled platform. Design Studies 88 (2023), 101202.
[41]
Petri Lankoski. 2004. Character design fundamentals for role-playing games. Beyond Role and Play (2004), 139--148.
[42]
Jonathan Lazar, Jinjuan Heidi Feng, and Harry Hochheiser. 2017. Research methods in human-computer interaction. Morgan Kaufmann.
[43]
RAY LC and Yuying Tang. 2023. Speculative Design with Generative AI: Applying Stable Diffusion and ChatGPT to imagining climate change futures. In Proceedings of the 11th International Conference on Digital and Interactive Arts. 1--8.
[44]
Josiah Lebowitz and Chris Klug. 2011. Interactive storytelling for video games: A player-centered approach to creating memorable characters and stories. Taylor & Francis.
[45]
Sola Lee, Suin Jun, Mihye Park, Sui Park, and Jongin Choi. 2021. Character Generation and Storytelling using Modeling Tool and Game Engine. In Proceedings of the Korean Institute of Information and Commucation Sciences Conference. The Korea Institute of Information and Commucation Engineering, 620--622.
[46]
Jie Li, Hancheng Cao, Laura Lin, Youyang Hou, Ruihao Zhu, and Abdallah El Ali. 2024. User experience design professionals? perceptions of generative artificial intelligence. In Proceedings of the CHI Conference on Human Factors in Computing Systems. 1--18.
[47]
Eliott Lilly. 2017. The Big BadWorld of Concept Art for Video Games: How to Start Your Career as a Concept Artist. Design Studio Press.
[48]
Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. 2023. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. Comput. Surveys 55, 9 (2023), 1--35.
[49]
Elman Mansimov, Emilio Parisotto, Jimmy Lei Ba, and Ruslan Salakhutdinov. 2015. Generating images from captions with attention. arXiv preprint arXiv:1511.02793 (2015).
[50]
Gary Marcus, Ernest Davis, and Scott Aaronson. 2022. A very preliminary analysis of DALL-E 2. arXiv preprint arXiv:2204.13807 (2022).
[51]
Margaret Mark and Carol S Pearson. 2001. O herói e o fora-da-lei: como construir marcas extraordinárias usando o poder dos arquétipos. São Paulo: Pensamento (2001).
[52]
Dalila Martins, Leonardo Pereira, Ana Patrícia Oliveira, and Nelson Zagalo. 2021. Character Design: The Case of Characters in a Hybrid Serious Game Called FlavourGame. In International Conference on Design and Digital Communication. Springer, 183--194.
[53]
Piotr Mirowski, Kory W Mathewson, Jaylen Pittman, and Richard Evans. 2023. Co-Writing Screenplays and Theatre Scripts with Language Models: Evaluation by Industry Professionals. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1--34.
[54]
Michael Muller, Lydia B Chilton, Anna Kantosalo, Charles Patrick Martin, and Greg Walsh. [n. d.]. GenAICHI: Generative AI and HCI. In CHI Conference on Human Factors in Computing Systems Extended Abstracts (New Orleans LA USA, 2022-04--27). ACM, 1--7. https://doi.org/10.1145/3491101.3503719
[55]
John Naughton. 2023. The ChatGPT bot is causing panic now--but it'll soon be as mundane a tool as Excel. The Guardian 7 (2023).
[56]
Kongdee Nop, Jintapitak Manissaward, and Tangto Oattarapon. 2019. Development of character design frameworks using game engine: unreal engine. In 2019 joint international conference on digital arts, media and technology with ECTI northern section conference on electrical, electronics, computer and telecommunications engineering (ECTI DAMT-NCON). IEEE, 54--59.
[57]
Changhoon Oh, Jungwoo Song, Jinhan Choi, Seonghyeon Kim, Sungwoo Lee, and Bongwon Suh. 2018. I lead, you help but only with enough details: Understanding user experience of co-creation with artificial intelligence. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1--13.
[58]
Jonas Oppenlaender. 2022. A taxonomy of prompt modifiers for text-to-image generation. arXiv preprint arXiv:2204.13988 2 (2022).
[59]
Hua Xuan Qin, Shan Jin, Ze Gao, Mingming Fan, and Pan Hui. 2024. CharacterMeet: Supporting CreativeWriters? Entire Story Character Construction Processes Through Conversation with LLM-Powered Chatbot Avatars. In Proceedings of the CHI Conference on Human Factors in Computing Systems. 1--19.
[60]
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learning. PMLR, 8748--8763.
[61]
Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever.
[62]
. Zero-Shot Text-to-Image Generation. In Proceedings of the 38th International Conference on Machine Learning (2021-07-01). PMLR, 8821--8831. https://proceedings.mlr.press/v139/ramesh21a.html ISSN: 2640--3498.
[63]
Oliver Schmitt and Daniel Buschek. 2021. Characterchat: Supporting the creation of fictional characters through conversation and progressive manifestation with a chatbot. In Creativity and Cognition. 1--10.
[64]
Tianyang Shi, Zhengxia Zou, Xinhui Song, Zheng Song, Changjian Gu, Changjie Fan, and Yi Yuan. 2020. Neutral face game character auto-creation via pokerface-gan. In Proceedings of the 28th ACM International Conference on Multimedia. 3201--3209.
[65]
Chris Solarski. 2012. Drawing basics and video game art: Classic to cutting-edge art techniques for winning video game design. Watson-Guptill.
[66]
Angie Spoto, Natalia Oleynik, Sebastian Deterding, and Jon Hook. 2017. Library of Mixed-Initiative Creative Interfaces.
[67]
Luke Stark and Kate Crawford. 2019. The work of art in the age of artificial intelligence: What artists can teach us about the ethics of data practice. Surveillance & Society 17, 3/4 (2019), 442--455.
[68]
Sangho Suh, Jian Zhao, and Edith Law. 2022. Codetoon: Story ideation, auto comic generation, and structure mapping for code-driven storytelling. In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology. 1--16.
[69]
Qirui Sun, Qiaoyang Luo, Yunyi Ni, and Haipeng Mi. 2024. Text2AC: A Framework for Game-Ready 2D Agent Character (AC) Generation from Natural Language. In Extended Abstracts of the CHI Conference on Human Factors in Computing Systems. 1--7.
[70]
Yuqian Sun, Xuran Ni, Haozhen Feng, Ray LC, Chang Hee Lee, and Ali Asadipour. 2022. Bringing stories to life in 1001 nights: A co-creative text adventure game using a story generation model. In International Conference on Interactive Digital Storytelling. Springer, 651--672.
[71]
P Tavares and PM Teixeira. 2011. Drawing and project: tradition and innovation. In Proceedings of Conferência Designa.
[72]
Mathias Peter Verheijden and Mathias Funk. 2023. Collaborative Diffusion: Boosting Designerly Co-Creation with Generative AI. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1--8.
[73]
Jesse Voimala. 2023. Creating a Video Game Character Design Workflow: a literature review on the theories and methods used by professionals in video game character design. (2023).
[74]
Xingzhi Wang, Nabil Anwer, Yun Dai, and Ang Liu. 2023. ChatGPT for design, manufacturing, and education. Procedia CIRP 119 (2023), 7--14.
[75]
Daijin Yang, Yanpeng Zhou, Zhiyuan Zhang, and Toby Jia-Jun Li. [n. d.]. AI as an Active Writer: Interaction strategies with generated text in human-AI collaborative fiction writing. ([n. d.]).
[76]
Han Zhang, Tao Xu, Hongsheng Li, Shaoting Zhang, Xiaogang Wang, Xiaolei Huang, and Dimitris N Metaxas. 2017. Stackgan: Text to photo-realistic image synthesis with stacked generative adversarial networks. In Proceedings of the IEEE international conference on computer vision. 5907--5915.
[77]
Zheng Zhang, Jie Gao, Ranjodh Singh Dhaliwal, and Toby Jia-Jun Li. 2023. VISAR: A Human-AI Argumentative Writing Assistant with Visual Programming and Rapid Draft Prototyping. arXiv preprint arXiv:2304.07810 (2023).
[78]
Suifang Zhou, Latisha Besariani Hendra, Qinshi Zhang, Jussi Holopainen, and RAY LC. 2024. Eternagram: Probing Player Attitudes Towards Climate Change Using a ChatGPT-driven Text-based Adventure. In Proceedings of the CHI Conference on Human Factors in Computing Systems. 1--23.

Index Terms

  1. Sketchar: Supporting Character Design and Illustration Prototyping Using Generative AI

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image Proceedings of the ACM on Human-Computer Interaction
    Proceedings of the ACM on Human-Computer Interaction  Volume 8, Issue CHI PLAY
    CHI PLAY
    October 2024
    1726 pages
    EISSN:2573-0142
    DOI:10.1145/3700823
    Issue’s Table of Contents
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 15 October 2024
    Published in PACMHCI Volume 8, Issue CHI PLAY

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. collaboration
    2. creativity support

    Qualifiers

    • Research-article

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • 0
      Total Citations
    • 256
      Total Downloads
    • Downloads (Last 12 months)256
    • Downloads (Last 6 weeks)66
    Reflects downloads up to 18 Feb 2025

    Other Metrics

    Citations

    View Options

    Login options

    Full Access

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media