skip to main content
10.1145/3613905.3650947acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
Work in Progress

The Social Construction of Generative AI Prompts

Published: 11 May 2024 Publication History

Abstract

As text-to-image AI tools grow in capability and widespread use, research has focused on studying individualistic user prompt crafting strategies. Recognizing that technologies are socially constructed, this paper examines prompt engineering through a social lens. We propose reframing prompt engineering as a socio-cultural practice shaped by collective knowledge building. Through qualitative analysis of 19 semi-structured interviews with members of the MidJourney community, a text-to-image generative AI tool, we identify four socio-engagement themes: proprietary/solitary, derivative, collaborative, and provocative prompting. These themes reveal a space of social engagement modes based on personal values and motivations from individual exploration to influencing the prompt community and highlight a fine line between being inspired by others’ prompts and maintaining creative ownership. We argue that understanding distinct social engagement preferences can inform the design of AI tools to facilitate transparent prompt reuse mechanisms, integrate collaborative features, or preserve ethical concerns about prompt sharing.

Supplemental Material

MP4 File
Talk Video
Transcript for: Talk Video

References

[1]
2021. EZ Charts - Diffusion Parameter Studies. https://docs.google.com/document/d/1ORymHm0Te18qKiHnhcdgGp-WSt8ZkLZvow3raiu2DVU/edit?usp=sharing
[2]
2022. The DALL·E 2 Prompt Book. https://dallery.gallery/the-dalle-2-prompt-book/
[3]
2022. Midjourney Founder David Holz On The Impact Of AI On Art, Imagination And The Creative Economy. https://www.forbes.com/sites/robsalkowitz/2022/09/16/midjourney-founder-david-holz-on-the-impact-of-ai-on-art-imagination-and-the-creative-economy/?sh=3ca9cabf2d2b
[4]
2024. A.I. Generated Content. https://www.facebook.com/groups/a.i.generatedcontent/
[5]
2024. chatGPT for Teachers. https://www.facebook.com/groups/703007927897194/
[6]
Saleem Alhabash and Mengyan Ma. 2017. A tale of four platforms: Motivations and uses of Facebook, Twitter, Instagram, and Snapchat among college students?Social media+ society 3, 1 (2017), 2056305117691544.
[7]
Saleema Amershi, Dan Weld, Mihaela Vorvoreanu, Adam Fourney, Besmira Nushi, Penny Collisson, Jina Suh, Shamsi Iqbal, Paul N Bennett, Kori Inkpen, 2019. Guidelines for human-AI interaction. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–13.
[8]
Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative research in psychology 3, 2 (2006), 77–101.
[9]
Minsuk Chang, Stefania Druga, Alexander J. Fiannaca, Pedro Vergani, Chinmay Kulkarni, Carrie J Cai, and Michael Terry. 2023. The Prompt Artists. In Proceedings of the 15th Conference on Creativity and Cognition (Virtual Event, USA) (C&C ’23). Association for Computing Machinery, New York, NY, USA, 75–87. https://doi.org/10.1145/3591196.3593515
[10]
Ruijia Cheng, Ruotong Wang, Thomas Zimmermann, and Denae Ford. 2022. " It would work for me too": How Online Communities Shape Software Developers’ Trust in AI-Powered Code Generation Tools. arXiv preprint arXiv:2212.03491 (2022).
[11]
Arthur Cropley. 2006. Creativity: A social approach. Roeper Review 28, 3 (2006), 125–130.
[12]
Paul Denny, Viraj Kumar, and Nasser Giacaman. 2022. Conversing with Copilot: Exploring Prompt Engineering for Solving CS1 Problems Using Natural Language. arXiv preprint arXiv:2210.15157 (2022).
[13]
Deborah G Douglas. 2012. The Social Construction of Technological Systems, anniversary edition: New Directions in the Sociology and History of Technology. MIT press.
[14]
Harmeet G. 2022. Disco Diffusion 70+ Artist Studies. https://weirdwonderfulai.art/resources/disco-diffusion-70-plus-artist-studies/
[15]
Vlad Petre Glăveanu and Vlad Petre Glăveanu. 2014. Distributed creativity: what is it?Springer.
[16]
Edwin Hutchins. 2000. Distributed cognition. International Encyclopedia of the Social and Behavioral Sciences. Elsevier Science 138 (2000), 1–10.
[17]
Farnaz Jahanbakhsh and David R Karger. 2024. A Browser Extension for in-place Signaling and Assessment of Misinformation. arXiv preprint arXiv:2403.11485 (2024).
[18]
Ellen Jiang, Kristen Olson, Edwin Toh, Alejandra Molina, Aaron Donsbach, Michael Terry, and Carrie J Cai. 2022. PromptMaker: Prompt-based Prototyping with Large Language Models. In CHI Conference on Human Factors in Computing Systems Extended Abstracts. 1–8.
[19]
Ellen Jiang, Edwin Toh, Alejandra Molina, Kristen Olson, Claire Kayacik, Aaron Donsbach, Carrie J Cai, and Michael Terry. 2022. Discovering the Syntax and Strategies of Natural Language Programming with Generative Language Models. In CHI Conference on Human Factors in Computing Systems. 1–19.
[20]
Alexa Kayman. 2023. Midjourney. https://research.contrary.com/reports/midjourney
[21]
Hyung-Kwon Ko, Gwanmo Park, Hyeon Jeon, Jaemin Jo, Juho Kim, and Jinwook Seo. 2023. Large-scale text-to-image generation models for visual artists’ creative works. In Proceedings of the 28th International Conference on Intelligent User Interfaces. 919–933.
[22]
Michael Xieyang Liu, Advait Sarkar, Carina Negreanu, Benjamin Zorn, Jack Williams, Neil Toronto, and Andrew D Gordon. 2023. “What It Wants Me To Say”: Bridging the Abstraction Gap Between End-User Programmers and Code-Generating Large Language Models. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–31.
[23]
Vivian Liu and Lydia B Chilton. 2022. Design Guidelines for Prompt Engineering Text-to-Image Generative Models. In CHI Conference on Human Factors in Computing Systems. 1–23.
[24]
Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, and Pontus Stenetorp. 2021. Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity. arXiv preprint arXiv:2104.08786 (2021).
[25]
Atefeh Mahdavi Goloujeh, Anne Sullivan, and Brian Magerko. 2024. Is It AI or Is It Me? Understanding Users’ Prompt Journey with Text-to-Image Generative AI Tools. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems.
[26]
Midjourney. 2022. Midjoruney AI. https://www.midjourney.com/home/
[27]
Midjourney Midjourney. 2022. Midjoruney AI. https://www.midjourney.com/home/#about
[28]
Joe O’Connor and Jacob Andreas. 2021. What Context Features Can Transformer Language Models Use?arXiv preprint arXiv:2106.08367 (2021).
[29]
OpenAI. 2022. Dall·E 2. https://openai.com/dall-e-2/
[30]
Jonas Oppenlaender. 2022. The creativity of text-to-image generation. In Proceedings of the 25th International Academic Mindtrek Conference. 192–202.
[31]
Jonas Oppenlaender. 2022. Prompt Engineering for Text-Based Generative Art. arXiv preprint arXiv:2204.13988 (2022).
[32]
Paul Paulus. 2000. Groups, teams, and creativity: The creative potential of idea-generating groups. Applied psychology 49, 2 (2000), 237–262.
[33]
Mitchel Resnick, John Maloney, Andrés Monroy-Hernández, Natalie Rusk, Evelyn Eastmond, Karen Brennan, Amon Millner, Eric Rosenbaum, Jay Silver, Brian Silverman, 2009. Scratch: programming for all. Commun. ACM 52, 11 (2009), 60–67.
[34]
Laria Reynolds and Kyle McDonell. 2021. Prompt programming for large language models: Beyond the few-shot paradigm. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems. 1–7.
[35]
Kevin Roose. 2022. A Coming-Out Party for Generative A.I., Silicon Valley’s New Craze. https://www.nytimes.com/2022/10/21/technology/generative-ai.html
[36]
Téo Sanchez. 2023. Examining the Text-to-Image Community of Practice: Why and How do People Prompt Generative AIs?. In Proceedings of the 15th Conference on Creativity and Cognition. 43–61.
[37]
Victor Sanh, Albert Webson, Colin Raffel, Stephen H Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Teven Le Scao, Arun Raja, 2021. Multitask prompted training enables zero-shot task generalization. arXiv preprint arXiv:2110.08207 (2021).
[38]
Ethan Smith. 2022. A Traveler’s Guide to the Latent Space. https://sweet-hall-e72.notion.site/A-Traveler-s-Guide-to-the-Latent-Space-85efba7e5e6a40e5bd3cae980f30235f
[39]
stability.ai. 2022. Stable Diffusion. https://stability.ai/blog/stable-diffusion-public-release
[40]
Lucy Suchman, Jeanette Blomberg, Julian E Orr, and Randall Trigg. 2017. Reconstructing technologies as social practice. In The Anthropology of Organisations. Routledge, 431–447.
[41]
Jakob Tholander and Martin Jonsson. 2023. Design Ideation with AI-Sketching, Thinking and Talking with Generative Machine Learning Models. In Proceedings of the 2023 ACM Designing Interactive Systems Conference. 1930–1940.
[42]
David R Thomas. 2003. A general inductive approach for qualitative data analysis. (2003).
[43]
Andy Warr and Eamonn O’Neill. 2005. Understanding design as a social creative process. In Proceedings of the 5th Conference on Creativity & Cognition. 118–127.
[44]
Sam Witteveen and Martin Andrews. 2022. Investigating Prompt Engineering in Diffusion Models. arXiv preprint arXiv:2211.15462 (2022).
[45]
JD Zamfirescu-Pereira, Richmond Y Wong, Bjoern Hartmann, and Qian Yang. 2023. Why Johnny can’t prompt: how non-AI experts try (and fail) to design LLM prompts. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21.

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
CHI EA '24: Extended Abstracts of the CHI Conference on Human Factors in Computing Systems
May 2024
4761 pages
ISBN:9798400703317
DOI:10.1145/3613905
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 May 2024

Check for updates

Author Tags

  1. Communities of Practice
  2. Generative AI
  3. Prompt Engineering
  4. Text-to-Image

Qualifiers

  • Work in progress
  • Research
  • Refereed limited

Conference

CHI '24

Acceptance Rates

Overall Acceptance Rate 6,164 of 23,696 submissions, 26%

Upcoming Conference

CHI 2025
ACM CHI Conference on Human Factors in Computing Systems
April 26 - May 1, 2025
Yokohama , Japan

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 719
    Total Downloads
  • Downloads (Last 12 months)719
  • Downloads (Last 6 weeks)78
Reflects downloads up to 20 Jan 2025

Other Metrics

Citations

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Full Text

View this article in Full Text.

Full Text

HTML Format

View this article in HTML Format.

HTML Format

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media