skip to main content
10.1145/3325480.3326578acmconferencesArticle/Chapter ViewAbstractPublication Pagesc-n-cConference Proceedingsconference-collections
poster

collabdraw: An Environment for Collaborative Sketching with an Artificial Agent

Published: 13 June 2019 Publication History

Abstract

Sketching is one of the most accessible techniques for communicating our ideas quickly and for collaborating in real time. Here we present a web-based environment for collaborative sketching of everyday visual concepts. We explore the integration of an artificial agent, instantiated as a recurrent neural network, who is both cooperative and responsive to actions performed by its human collaborator. To evaluate the quality of the sketches produced in this environment, we conducted an experimental user study and found that sketches produced collaboratively carried as much semantically relevant information as those produced by humans on their own. Further control analyses suggest that the semantic information in these sketches were indeed the product of collaboration, rather than attributable to the contributions of the human or the artificial agent alone. Taken together, our findings attest to the potential of systems enabling real-time collaboration between humans and machines to create novel and meaningful content.

References

[1]
Nicholas Davis, Chih-PIn Hsiao, Kunwar Yashraj Singh, Lisa Li, and Brian Magerko. 2016b. Empirically studying participatory sense-making in abstract drawing with a co-creative cognitive agent. In Proceedings of the 21st International Conference on Intelligent User Interfaces. ACM, 196--207.
[2]
Nicholas Mark Davis, Chih-Pin Hsiao, Kunwar Yashraj Singh, and Brian Magerko. 2016a. Co-creative drawing agent with object recognition. In Twelfth Artificial Intelligence and Interactive Digital Entertainment Conference .
[3]
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Imagenet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition. Ieee, 248--255.
[4]
Judith E. Fan, Daniel L. K. Yamins, and Nicholas B. Turk-Browne. 2018. Common Object Representations for Visual Production and Recognition. Cognitive Science (2018).
[5]
David Ha and Douglas Eck. 2017. A neural representation of sketch drawings. arXiv preprint arXiv:1704.03477 (2017).
[6]
Jonas Jongejan, Henry Rowley, Takashi Kawashima, Jongmin Kim, and Nick Fox-Gieg. 2017. Google QuickDraw. (2017). https://quickdraw.withgoogle.com/
[7]
Pegah Karimi, Nicholas Davis, Kazjon Grace, and Mary Lou Maher. 2018. Deep Learning for Identifying Potential Conceptual Shifts for Co-creative Drawing. CoRR (2018). http://arxiv.org/abs/1801.00723
[8]
Bria Long, Judith E Fan, and Michael C Frank. 2018. Drawings as a window into developmental changes in object representations. In Proceedings of the 40th Annual Meeting of the Cognitive Science Society.
[9]
Changhoon Oh, Jungwoo Song, Jinhan Choi, Seonghyeon Kim, Sungwoo Lee, and Bongwon Suh. 2018. I lead, you help but only with enough details: Understanding user experience of co-creation with artificial intelligence. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 649.
[10]
Adam Roberts, Curtis Hawthorne, and Ian Simon. 2018. Magenta. js: A Javascript API for augmenting creativity with deep learning . (2018).
[11]
Patsorn Sangkloy, Nathan Burnell, Cusuh Ham, and James Hays. 2016. The sketchy database: learning to retrieve badly drawn bunnies. ACM Transactions on Graphics (TOG) 35, 4 (2016), 119.
[12]
Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).
[13]
Daniel Smilkov, Nikhil Thorat, Yannick Assogba, Ann Yuan, Nick Kreeger, Ping Yu, Kangyi Zhang, Shanqing Cai, Eric Nielsen, David Soergel, and others. 2019. TensorFlow. js: Machine Learning for the Web and Beyond. arXiv preprint arXiv:1901.05350 (2019).
[14]
Qian Yu, Yongxin Yang, Feng Liu, Yi-Zhe Song, Tao Xiang, and Timothy M Hospedales. 2017. Sketch-a-net: A deep neural network that beats humans. International Journal of Computer Vision 122, 3 (2017), 411--425.
[15]
Daniel Zalewski. 2015. Life Lines. The New Yorker (2015).

Cited By

View all
  • (2024)VRCopilot: Authoring 3D Layouts with Generative AI Models in VRProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676451(1-13)Online publication date: 13-Oct-2024
  • (2024)Generative AI in User Experience Design and Research: How Do UX Practitioners, Teams, and Companies Use GenAI in Industry?Proceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3660720(1579-1593)Online publication date: 1-Jul-2024
  • (2024)Reasoning and Planning with Large Language Models in Code DevelopmentProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671452(6480-6490)Online publication date: 25-Aug-2024
  • Show More Cited By

Index Terms

  1. collabdraw: An Environment for Collaborative Sketching with an Artificial Agent

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      C&C '19: Proceedings of the 2019 Conference on Creativity and Cognition
      June 2019
      745 pages
      ISBN:9781450359177
      DOI:10.1145/3325480
      Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 13 June 2019

      Check for updates

      Author Tags

      1. artificial intelligence
      2. cooperation
      3. drawing

      Qualifiers

      • Poster

      Conference

      C&C '19
      Sponsor:
      C&C '19: Creativity and Cognition
      June 23 - 26, 2019
      CA, San Diego, USA

      Acceptance Rates

      C&C '19 Paper Acceptance Rate 30 of 101 submissions, 30%;
      Overall Acceptance Rate 108 of 371 submissions, 29%

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)128
      • Downloads (Last 6 weeks)9
      Reflects downloads up to 08 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)VRCopilot: Authoring 3D Layouts with Generative AI Models in VRProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676451(1-13)Online publication date: 13-Oct-2024
      • (2024)Generative AI in User Experience Design and Research: How Do UX Practitioners, Teams, and Companies Use GenAI in Industry?Proceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3660720(1579-1593)Online publication date: 1-Jul-2024
      • (2024)Reasoning and Planning with Large Language Models in Code DevelopmentProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671452(6480-6490)Online publication date: 25-Aug-2024
      • (2024)Perceptions of Interaction Dynamics in Co-Creative AI: A Comparative Study of Interaction Modalities in DrawctoProceedings of the 16th Conference on Creativity & Cognition10.1145/3635636.3656202(102-116)Online publication date: 23-Jun-2024
      • (2024)Evaluating Human-AI Partnership for LLM-based Code MigrationExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650896(1-8)Online publication date: 11-May-2024
      • (2024)Doodlebot: An Educational Robot for Creativity and AI LiteracyProceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3610977.3634950(772-780)Online publication date: 11-Mar-2024
      • (2024)A Collaborative, Interactive and Context-Aware Drawing Agent for Co-Creative DesignIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.329385330:8(5525-5537)Online publication date: Aug-2024
      • (2024)Exploring post-human futures with design fiction among undergraduate students: implications for future-oriented design educationThe Design Journal10.1080/14606925.2024.2433357(1-24)Online publication date: 28-Nov-2024
      • (2024)User Preferences on a Generative AI User Interface Through a Choice ExperimentInternational Journal of Human–Computer Interaction10.1080/10447318.2024.2400379(1-12)Online publication date: 12-Sep-2024
      • (2024)“Journey of Finding the Best Query”: Understanding the User Experience of AI Image Generation SystemInternational Journal of Human–Computer Interaction10.1080/10447318.2024.230767041:2(951-969)Online publication date: 2-Feb-2024
      • Show More Cited By

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media