Zijian Ding
ABSTRACT
Technological progress has persistently shaped the dynamics of human-machine interactions in task execution. In response to the advancements in Generative AI, this paper outlines a detailed study plan that investigates various human-AI interaction modalities across a range of tasks, characterized by differing levels of creativity and complexity. This exploration aims to inform and contribute to the development of Graphical User Interfaces (GUIs) that effectively integrate with and enhance the capabilities of Generative AI systems. The study comprises three parts: exploring fixed-scope tasks through news headline generation, delving into atomic creative tasks with analogy generation, and investigating complex tasks via data visualization. Future work aims to extend this exploration to linearize complex data analysis results into narratives understandable to a broader audience, thereby enhancing the interpretability of AI-generated content.
- Tyler Angert, Miroslav Suzara, Jenny Han, Christopher Pondoc, and Hariharan Subramonyam. 2023. Spellburst: A Node-based Interface for Exploratory Creative Coding with Natural Language Prompts. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology. ACM, San Francisco CA USA, 1–22. https://doi.org/10.1145/3586183.3606719Google Scholar
Digital Library
- Stephen Brade, Bryan Wang, Mauricio Sousa, Sageev Oore, and Tovi Grossman. 2023. Promptify: Text-to-Image Generation through Interactive Prompt Exploration with Large Language Models. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology. ACM, San Francisco CA USA, 1–14. https://doi.org/10.1145/3586183.3606725Google ScholarDigital Library
- Joel Chan, Zijian Ding, Eesh Kamrah, and Mark Fuge. 2024. Formulating or Fixating: Effects of Examples on Problem Solving Vary as a Function of Example Presentation Interface Design. http://arxiv.org/abs/2401.11022 arXiv:2401.11022 [cs].Google Scholar
- Joel Chan, Pao Siangliulue, Denisa Qori McDonald, Ruixue Liu, Reza Moradinezhad, Safa Aman, Erin T. Solovey, Krzysztof Z. Gajos, and Steven P. Dow. 2017. Semantically Far Inspirations Considered Harmful?: Accounting for Cognitive States in Collaborative Ideation. In Proceedings of the 2017 ACM SIGCHI Conference on Creativity and Cognition(C&C ’17). ACM, New York, NY, USA, 93–105. https://doi.org/10.1145/3059454.3059455Google ScholarDigital Library
- Ruijia Cheng, Alison Smith-Renner, Ke Zhang, Joel Tetreault, and Alejandro Jaimes-Larrarte. 2022. Mapping the Design Space of Human-AI Interaction in Text Summarization. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, Seattle, United States, 431–455. https://doi.org/10.18653/v1/2022.naacl-main.33Google ScholarCross Ref
- John Joon Young Chung and Eytan Adar. 2023. PromptPaint: Steering Text-to-Image Generation Through Paint Medium-like Interactions. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology. ACM, San Francisco CA USA, 1–17. https://doi.org/10.1145/3586183.3606777Google ScholarDigital Library
- Elizabeth Clark, Tal August, Sofia Serrano, Nikita Haduong, Suchin Gururangan, and Noah A. Smith. 2021. All That’s ’Human’ Is Not Gold: Evaluating Human Evaluation of Generated Text. arXiv:2107.00061 [cs] (July 2021). http://arxiv.org/abs/2107.00061 00008 arXiv: 2107.00061.Google Scholar
- Elizabeth Clark, Anne Spencer Ross, Chenhao Tan, Yangfeng Ji, and Noah A. Smith. 2018. Creative Writing with a Machine in the Loop: Case Studies on Slogans and Stories. In 23rd International Conference on Intelligent User Interfaces. ACM, Tokyo Japan, 329–340. https://doi.org/10.1145/3172944.3172983Google ScholarDigital Library
- Zijian Ding and Joel Chan. 2023. Mapping the Design Space of Interactions in Human-AI Text Co-creation Tasks. http://arxiv.org/abs/2303.06430 arXiv:2303.06430 [cs].Google Scholar
- Zijian Ding, Jiawen Kang, Tinky Oi Ting Ho, Ka Ho Wong, Helene H Fung, Helen Meng, and Xiaojuan Ma. 2022. TalkTive: A Conversational Agent Using Backchannels to Engage Older Adults in Neurocognitive Disorders Screening. In CHI Conference on Human Factors in Computing Systems. ACM, New Orleans LA USA, 1–19. https://doi.org/10.1145/3491102.3502005Google ScholarDigital Library
- Zijian Ding, Alison Smith-Renner, Wenjuan Zhang, Joel R. Tetreault, and Alejandro Jaimes. 2023. Harnessing the Power of LLMs: Evaluating Human-AI Text Co-Creation through the Lens of News Headline Generation. http://arxiv.org/abs/2310.10706 arXiv:2310.10706 [cs].Google Scholar
- Zijian Ding, Arvind Srinivasan, Stephen MacNeil, and Joel Chan. 2023. Fluid Transformers and Creative Analogies: Exploring Large Language Models’ Capacity for Augmenting Cross-Domain Analogical Creativity. arXiv preprint arXiv:2302.12832 (2023).Google Scholar
- Yao Dou, Maxwell Forbes, Rik Koncel-Kedziorski, Noah Smith, and Yejin Choi. 2022. Is GPT-3 Text Indistinguishable from Human Text? Scarecrow: A Framework for Scrutinizing Machine Text. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Dublin, Ireland, 7250–7274. https://doi.org/10.18653/v1/2022.acl-long.501Google ScholarCross Ref
- Karl Duncker. 1945. On problem-solving.Psychological Monographs 58, 5 (1945), i–113. https://doi.org/10.1037/h0093599Google ScholarCross Ref
- Katy Ilonka Gero, Vivian Liu, and Lydia Chilton. 2022. Sparks: Inspiration for Science Writing using Language Models. In Designing Interactive Systems Conference. ACM, Virtual Event Australia, 1002–1019. https://doi.org/10.1145/3532106.3533533Google ScholarDigital Library
- Tanya Goyal, Junyi Jessy Li, and Greg Durrett. 2022. News Summarization and Evaluation in the Era of GPT-3. http://arxiv.org/abs/2209.12356 arXiv:2209.12356 [cs].Google Scholar
- Søren Knudsen, Mikkel Rønne Jakobsen, and Kasper Hornbæk. 2012. An exploratory study of how abundant display space may support data analysis. In Proceedings of the 7th Nordic Conference on Human-Computer Interaction: Making Sense Through Design. ACM, Copenhagen Denmark, 558–567. https://doi.org/10.1145/2399016.2399102Google ScholarDigital Library
- Mina Lee, Megha Srivastava, Amelia Hardy, John Thickstun, Esin Durmus, Ashwin Paranjape, Ines Gerard-Ursin, Xiang Lisa Li, Faisal Ladhak, Frieda Rong, Rose E. Wang, Minae Kwon, Joon Sung Park, Hancheng Cao, Tony Lee, Rishi Bommasani, Michael Bernstein, and Percy Liang. 2022. Evaluating Human-Language Model Interaction. http://arxiv.org/abs/2212.09746 arXiv:2212.09746 [cs] version: 2.Google Scholar
- Zhicheng Lin. 2023. Why and how to embrace AI such as ChatGPT in your academic life. preprint. PsyArXiv. https://doi.org/10.31234/osf.io/sdx3jGoogle ScholarCross Ref
- Stephen MacNeil, Zijian Ding, Ashley Boone, Anthony Bryce Grubbs, and Steven P. Dow. 2021. Finding Place in a Design Space: Challenges for Supporting Community Design Efforts at Scale. Proceedings of the ACM on Human-Computer Interaction 5, CSCW1 (April 2021), 1–30. https://doi.org/10.1145/3449246Google ScholarDigital Library
- Stephen MacNeil, Zijian Ding, Kexin Quan, Ziheng Huang, Kenneth Chen, and Steven P. Dow. 2021. ProbMap: Automatically constructing design galleries through feature extraction and semantic clustering. In The Adjunct Publication of the 34th Annual ACM Symposium on User Interface Software and Technology. ACM, Virtual Event USA, 134–136. https://doi.org/10.1145/3474349.3480203Google ScholarDigital Library
- Stephen MacNeil, Zijian Ding, Kexin Quan, Thomas j Parashos, Yajie Sun, and Steven P. Dow. 2021. Framing Creative Work: Helping Novices Frame Better Problems through Interactive Scaffolding. In Creativity and Cognition(C&C ’21). Association for Computing Machinery, New York, NY, USA, 1–10. https://doi.org/10.1145/3450741.3465261Google ScholarDigital Library
- Stephen MacNeil, Ziheng Huang, Kenneth Chen, Zijian Ding, Alex Yu, Kendall Nakai, and Steven P. Dow. 2023. Freeform Templates: Combining Freeform Curation with Structured Templates. In Creativity and Cognition. 478–488. https://doi.org/10.1145/3591196.3593337 arXiv:2305.00937 [cs].Google ScholarDigital Library
- Stephen MacNeil, Andrew Tran, Dan Mogil, Seth Bernstein, Erin Ross, and Ziheng Huang. 2022. Generating Diverse Code Explanations using the GPT-3 Large Language Model. In Proceedings of the 2022 ACM Conference on International Computing Education Research - Volume 2. ACM, Lugano and Virtual Event Switzerland, 37–39. https://doi.org/10.1145/3501709.3544280Google ScholarDigital Library
- Srishti Palani, Zijian Ding, Stephen MacNeil, and Steven P. Dow. 2021. The "Active Search" Hypothesis: How Search Strategies Relate to Creative Learning. In Proceedings of the 2021 Conference on Human Information Interaction and Retrieval. ACM, Canberra ACT Australia, 325–329. https://doi.org/10.1145/3406522.3446046Google ScholarDigital Library
- Srishti Palani, Zijian Ding, Austin Nguyen, Andrew Chuang, Stephen MacNeil, and Steven P. Dow. 2021. CoNotate: Suggesting Queries Based on Notes Promotes Knowledge Discovery. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems(CHI ’21). Association for Computing Machinery, New York, NY, USA, 1–14. https://doi.org/10.1145/3411764.3445618Google ScholarDigital Library
- R. Keith Sawyer. 2012. Explaining creativity: the science of human innovation (2nd ed.). Oxford University Press, New York.Google Scholar
- Nikhil Singh, Guillermo Bernal, Daria Savchenko, and Elena L. Glassman. 2022. Where to Hide a Stolen Elephant: Leaps in Creative Writing with Multimodal Machine Intelligence. ACM Transactions on Computer-Human Interaction (Feb. 2022), 3511599. https://doi.org/10.1145/3511599Google ScholarDigital Library
- Zihan Yan, Chunxu Yang, Qihao Liang, and Xiang ’Anthony’ Chen. 2023. XCreation: A Graph-based Crossmodal Generative Creativity Support Tool. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology. ACM, San Francisco CA USA, 1–15. https://doi.org/10.1145/3586183.3606826Google ScholarDigital Library
- Ann Yuan, Andy Coenen, Emily Reif, and Daphne Ippolito. 2022. Wordcraft: Story Writing With Large Language Models. In 27th International Conference on Intelligent User Interfaces. ACM, Helsinki Finland, 841–852. https://doi.org/10.1145/3490099.3511105Google ScholarDigital Library
- Q. Zhu and J. Luo. 2022. Generative Pre-Trained Transformer for Design Concept Generation: An Exploration. Proceedings of the Design Society 2 (May 2022), 1825–1834. https://doi.org/10.1017/pds.2022.185 Publisher: Cambridge University Press.Google ScholarCross Ref
Recommendations
AI Creativity and the Human-AI Co-creation Model
Human-Computer Interaction. Theory, Methods and ToolsAbstractArtificial intelligence (AI) is bringing new possibilities to numerous fields. There have been a lot of discussions about the development of AI technologies and the challenges caused by AI such as job replacement and ethical issues. However, it’s ...
Comments