skip to main content
10.1145/3613904.3641899acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
research-article

ABScribe: Rapid Exploration & Organization of Multiple Writing Variations in Human-AI Co-Writing Tasks using Large Language Models

Published: 11 May 2024 Publication History

Abstract

Exploring alternative ideas by rewriting text is integral to the writing process. State-of-the-art Large Language Models (LLMs) can simplify writing variation generation. However, current interfaces pose challenges for simultaneous consideration of multiple variations: creating new variations without overwriting text can be difficult, and pasting them sequentially can clutter documents, increasing workload and disrupting writers’ flow. To tackle this, we present ABScribe, an interface that supports rapid, yet visually structured, exploration and organization of writing variations in human-AI co-writing tasks. With ABScribe, users can swiftly modify variations using LLM prompts, which are auto-converted into reusable buttons. Variations are stored adjacently within text fields for rapid in-place comparisons using mouse-over interactions on a popup toolbar. Our user study with 12 writers shows that ABScribe significantly reduces task workload (d = 1.20, p < 0.001), enhances user perceptions of the revision process (d = 2.41, p < 0.001) compared to a popular baseline workflow, and provides insights into how writers explore variations using LLMs.

Supplemental Material

MP4 File - Video Preview
Video Preview
MP4 File - Video Presentation
Video Presentation
Transcript for: Video Presentation
MP4 File - Video Figure
The abscribe_demo.mp4 file contains a 5-minute demo of the ABScribe prototype interface.
CSV File - Dataset
The task_workload.csv file displays the result of the NASA TLX survey administered to the participants. [1] The first column includes the identification numbers of each of the participants (w1, w2, ...). [2] The second column indicates the study condition (abscribe or baseline). [3] The remaining columns show the rating, tally, and weight for each scale of the NASA TLX, with each individual rating constituting its own columns. [4] The last column is labelled overall and shows each participant's overall score for the survey.
CSV File - Dataset
The revision_process.csv file details each participant's results of the Likert scale post-task questionnaire. [1] The first column includes the identification numbers of each participant (w1, w2, ...). [2] The second column indicates what version of the software (either abscribe or baseline) the participant was using when giving their ratings. [3] The remaining columns detail the ratings for each measure for a total of eleven measures. The names of the columns for these measures are based on the corresponding names introduced in the paper. [4] 1 = Strongly Disagree, 7 = Strongly Agree. Higher scores correspond to more positive perceptions for the given study condition (abscribe or baseline). For M11, the scores are reversed to account for the negative valence of wording.

References

[1]
William A Barrett and Alan S Cheney. 2002. Object-based image editing. In Proceedings of the 29th annual conference on Computer graphics and interactive techniques. Association for Computing Machinery, New York, NY, USA, 777–784.
[2]
Nancy Baym, Limor Shifman, Christopher Persaud, and Kelly Wagman. 2019. INTELLIGENT FAILURES: CLIPPY MEMES AND THE LIMITS OF DIGITAL ASSISTANTS. AoIR Selected Papers of Internet Research 2019 (Oct. 2019), 1–5. https://doi.org/10.5210/spir.v2019i0.10923
[3]
Michel Beaudouin-Lafon and Wendy E. Mackay. 2000. Reification, Polymorphism and Reuse: Three Principles for Designing Visual Interfaces. In Proceedings of the Working Conference on Advanced Visual Interfaces (Palermo, Italy) (AVI ’00). Association for Computing Machinery, New York, NY, USA, 102–109. https://doi.org/10.1145/345513.345267
[4]
Oloff C. Biermann, Ning F. Ma, and Dongwook Yoon. 2022. From Tool to Companion: Storywriters Want AI Writers to Respect Their Personal Values and Writing Strategies. In Proceedings of the 2022 ACM Designing Interactive Systems Conference (Virtual Event, Australia) (DIS ’22). Association for Computing Machinery, New York, NY, USA, 1209–1227. https://doi.org/10.1145/3532106.3533506
[5]
Sara Bly. 2007. Fundamentals in HCI: Learning the Value of Consistency and User Models. In HCI Remixed: Reflections on Works That Have Influenced the HCI Community. The MIT Press, Cambridge, MA, USA. https://doi.org/10.7551/mitpress/7455.003.0012 arXiv:https://direct.mit.edu/book/chapter-pdf/2290056/9780262256070_cah.pdf
[6]
Stephen Brade, Bryan Wang, Mauricio Sousa, Sageev Oore, and Tovi Grossman. 2023. Promptify: Text-to-Image Generation through Interactive Prompt Exploration with Large Language Models. arxiv:2304.09337 [cs.HC]
[7]
Virginia Braun and Victoria Clarke. 2019. Reflecting on reflexive thematic analysis. Qualitative research in sport, exercise and health 11, 4 (2019), 589–597.
[8]
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, 2020. Language models are few-shot learners. Advances in neural information processing systems 33 (2020), 1877–1901.
[9]
Daniel Buschek, Martin Zürn, and Malin Eiband. 2021. The impact of multiple parallel phrase suggestions on email input and composition behaviour of native and non-native english writers. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, 1–13.
[10]
Bill Buxton. 2010. Sketching user experiences: getting the design right and the right design. Morgan kaufmann, Boston.
[11]
Hsiang-Ting Chen, Li-Yi Wei, and Chun-Fa Chang. 2011. Nonlinear revision control for images. ACM Transactions on Graphics (TOG) 30, 4 (2011), 1–10.
[12]
N Ann Chenoweth. 1987. The need to teach rewriting. ELT journal 41, 1 (1987), 25–29.
[13]
Yin Ling Cheung. 2016. Teaching Writing. Springer International Publishing, Cham, 179–194. https://doi.org/10.1007/978-3-319-38834-2_13
[14]
John Joon Young Chung, Wooseok Kim, Kang Min Yoo, Hwaran Lee, Eytan Adar, and Minsuk Chang. 2022. TaleBrush: Sketching stories with generative pretrained language models. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, 1–19.
[15]
Robert Dale. 2021. GPT-3: What’s it good for?Natural Language Engineering 27, 1 (2021), 113–118.
[16]
Hai Dang, Sven Goller, Florian Lehmann, and Daniel Buschek. 2023. Choice over control: How users write with large language models using diegetic and non-diegetic prompting. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, 1–17.
[17]
Joost FC de Winter and Dimitra Dodou. 2010. Five-point likert items: t test versus Mann-Whitney-Wilcoxon (Addendum added October 2012). Practical Assessment, Research, and Evaluation 15, 1 (2010), 11.
[18]
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv:1810.04805http://arxiv.org/abs/1810.04805
[19]
Steven P Dow, Alana Glassco, Jonathan Kass, Melissa Schwarz, Daniel L Schwartz, and Scott R Klemmer. 2010. Parallel prototyping leads to better design results, more divergence, and increased self-efficacy. ACM Transactions on Computer-Human Interaction (TOCHI) 17, 4 (2010), 1–24.
[20]
Wanyu Du, Vipul Raheja, Dhruv Kumar, Zae Myung Kim, Melissa Lopez, and Dongyeop Kang. 2022. Understanding Iterative Revision from Human-Written Text. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Dublin, Ireland, 3573–3590. https://doi.org/10.18653/v1/2022.acl-long.250
[21]
Robin Emsley. 2023. ChatGPT: these are not hallucinations–they’re fabrications and falsifications. Schizophrenia 9, 1 (2023), 52.
[22]
Lester Faigley and Stephen Witte. 1981. Analyzing revision. College composition and communication 32, 4 (1981), 400–414.
[23]
Jill Fitzgerald. 1987. Research on revision in writing. Review of educational research 57, 4 (1987), 481–506.
[24]
Linda Flower and John R Hayes. 1981. A cognitive process theory of writing. College composition and communication 32, 4 (1981), 365–387.
[25]
Linda Flower and John R. Hayes. 1981. A Cognitive Process Theory of Writing. College Composition and Communication 32, 4 (1981), 365–387. http://www.jstor.org/stable/356600
[26]
Katy Ilonka Gero, Vivian Liu, and Lydia Chilton. 2022. Sparks: Inspiration for Science Writing using Language Models. In Proceedings of the 2022 ACM Designing Interactive Systems Conference (ustralia) (DIS ’22). Association for Computing Machinery, New York, NY, USA, 1002–1019. https://doi.org/10.1145/3532106.3533533
[27]
Katy Ilonka Gero, Tao Long, and Lydia B Chilton. 2023. Social dynamics of AI support in creative writing. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, 1–15.
[28]
Louie Giray. 2023. Prompt Engineering with ChatGPT: A Guide for Academic Writers. Annals of Biomedical Engineering 51 (06 2023), 3. https://doi.org/10.1007/s10439-023-03272-4
[29]
Natalie Goldberg. 2016. Writing down the bones: Freeing the writer within. Shambhala Publications, 2129 13th St, Boulder, Colorado.
[30]
Gabriela Goldschmidt. 2011. Avoiding design fixation: transformation and abstraction in mapping from source to target. The Journal of creative behavior 45, 2 (2011), 92–100.
[31]
Steven M Goodman, Erin Buehler, Patrick Clary, Andy Coenen, Aaron Donsbach, Tiffanie N Horne, Michal Lahav, Robert MacDonald, Rain Breaw Michaels, Ajit Narayanan, 2022. Lampost: Design and evaluation of an ai-assisted email writing prototype for adults with dyslexia. In Proceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility. Association for Computing Machinery, New York, NY, USA, 1–18.
[32]
Google. 2023. PaLM 2 Technical Report. https://ai.google/static/documents/palm2techreport.pdf.
[33]
Katja Grace, John Salvatier, Allan Dafoe, Baobao Zhang, and Owain Evans. 2018. When will AI exceed human performance? Evidence from AI experts. Journal of Artificial Intelligence Research 62 (2018), 729–754.
[34]
Saul Greenberg and Bill Buxton. 2008. Usability Evaluation Considered Harmful (Some of the Time). In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Florence, Italy) (CHI ’08). Association for Computing Machinery, New York, NY, USA, 111–120. https://doi.org/10.1145/1357054.1357074
[35]
Han L Han. 2020. Designing Representations for Digital Documents. In Adjunct Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology. Association for Computing Machinery, New York, NY, USA, 174–178.
[36]
Han L Han, Miguel A Renom, Wendy E Mackay, and Michel Beaudouin-Lafon. 2020. Textlets: Supporting constraints and consistency in text documents. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, 1–13.
[37]
Han L. Han, Miguel A. Renom, Wendy E. Mackay, and Michel Beaudouin-Lafon. 2020. Textlets: Supporting Constraints and Consistency in Text Documents. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3313831.3376804
[38]
Joseph Harris. 2017. Rewriting: How to do things with texts. University Press of Colorado, Boulder, CO, USA.
[39]
Sandra G. Hart. 2006. Nasa-Task Load Index (NASA-TLX); 20 Years Later. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 50, 9 (2006), 904–908. https://doi.org/10.1177/154193120605000909 arXiv:https://doi.org/10.1177/154193120605000909
[40]
Iikka Hauhio, Anna Kantosalo, Simo Linkola, and Hannu Toivonen. 2023. The Spectrum of Unpredictability and its Relation to Creative Autonomy. In Proceedings of the 14th International Conference on Computational Creativity. Association for Computational Creativity (ACC), Spain, 148–152. https://computationalcreativity.net/iccc23/ International Conference on Computational Creativity, ICCC ; Conference date: 19-06-2023 Through 23-06-2023.
[41]
Andrew Head, Fred Hohman, Titus Barik, Steven M. Drucker, and Robert DeLine. 2019. Managing Messes in Computational Notebooks. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3290605.3300500
[42]
Ernest Hemingway. 2014. Moveable feast: the restored edition. Simon and Schuster, New York, NY, USA.
[43]
Roland K Huff. 1983. Teaching revision: A model of the drafting process. College English 45, 8 (1983), 800–816.
[44]
David G Jansson and Steven M Smith. 1991. Design fixation. Design studies 12, 1 (1991), 3–11.
[45]
Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. Survey of hallucination in natural language generation. Comput. Surveys 55, 12 (2023), 1–38.
[46]
Ellen Jiang, Kristen Olson, Edwin Toh, Alejandra Molina, Aaron Donsbach, Michael Terry, and Carrie J Cai. 2022. PromptMaker: Prompt-based Prototyping with Large Language Models. In Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI EA ’22). Association for Computing Machinery, New York, NY, USA, Article 35, 8 pages. https://doi.org/10.1145/3491101.3503564
[47]
Tae Soo Kim, Yoonjoo Lee, Minsuk Chang, and Juho Kim. 2023. Cells, Generators, and Lenses: Design Framework for Object-Oriented Interaction with Large Language Models. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology (San Francisco, CA, USA) (UIST ’23). Association for Computing Machinery, New York, NY, USA, Article 4, 18 pages. https://doi.org/10.1145/3586183.3606833
[48]
Yewon Kim, Mina Lee, Donghwi Kim, and Sung-Ju Lee. 2023. Towards Explainable AI Writing Assistants for Non-native English Speakers.
[49]
Barbara Kroll and Joy Reid. 1994. Guidelines for designing writing prompts: Clarifications, caveats, and cautions. Journal of Second Language Writing 3, 3 (1994), 231–255.
[50]
Mina Lee, Percy Liang, and Qian Yang. 2022. Coauthor: Designing a human-ai collaborative writing dataset for exploring language model capabilities. In Proceedings of the 2022 CHI conference on human factors in computing systems. Association for Computing Machinery, New York, NY, USA, 1–19.
[51]
Mina Lee, Megha Srivastava, Amelia Hardy, John Thickstun, Esin Durmus, Ashwin Paranjape, Ines Gerard-Ursin, Xiang Lisa Li, Faisal Ladhak, Frieda Rong, 2022. Evaluating human-language model interaction.
[52]
Carrie S Leverenz. 2014. Design thinking and the wicked problem of teaching writing. Computers and Composition 33 (2014), 1–12.
[53]
Clayton Lewis and Cathleen Wharton. 1997. Chapter 30 - Cognitive Walkthroughs. In Handbook of Human-Computer Interaction (Second Edition) (second edition ed.), Marting G. Helander, Thomas K. Landauer, and Prasad V. Prabhu (Eds.). North-Holland, Amsterdam, 717–732. https://doi.org/10.1016/B978-044481862-1.50096-0
[54]
Vivian Liu. 2023. Beyond Text-to-Image: Multimodal Prompts to Explore Generative AI. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI EA ’23). Association for Computing Machinery, New York, NY, USA, Article 482, 6 pages. https://doi.org/10.1145/3544549.3577043
[55]
Vivian Liu and Lydia B Chilton. 2022. Design Guidelines for Prompt Engineering Text-to-Image Generative Models. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI ’22). Association for Computing Machinery, New York, NY, USA, Article 384, 23 pages. https://doi.org/10.1145/3491102.3501825
[56]
Tao Long, Dorothy Zhang, Grace Li, Batool Taraif, Samia Menon, Kynnedy Simone Smith, Sitong Wang, Katy Ilonka Gero, and Lydia B Chilton. 2023. Tweetorial Hooks: Generative AI Tools to Motivate Science on Social Media.
[57]
Charles A MacArthur. 2018. Evaluation and revision. Best practices in writing instruction 287 (2018), 23 pages.
[58]
Stephen MacNeil, Andrew Tran, Juho Leinonen, Paul Denny, Joanne Kim, Arto Hellas, Seth Bernstein, and Sami Sarsa. 2023. Automatically Generating CS Learning Materials with Large Language Models. In Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 2 (Toronto ON, Canada) (SIGCSE 2023). Association for Computing Machinery, New York, NY, USA, 1176. https://doi.org/10.1145/3545947.3569630
[59]
Thomas Mahatody, Mouldi Sagar, and Christophe Kolski. 2010. State of the art on the cognitive walkthrough method, its variants and evolutions. Intl. Journal of Human–Computer Interaction 26, 8 (2010), 741–785.
[60]
George A. McCulley. 1985. Writing Quality, Coherence, and Cohesion. Research in the Teaching of English 19, 3 (1985), 269–282. https://www.jstor.org/stable/40171050 Publisher: National Council of Teachers of English.
[61]
Deborah McCutchen, Paul Teske, and Catherine Bankston. 2009. Writing and cognition: Implications of the cognitive architecture for learning to write and writing to learn. In Handbook of research on writing. Routledge, Milton Park, Abingdon, Oxfordshire, United Kingdom, 554–578.
[62]
John McPhee. 2017. Draft No. 4: On the writing process. Farrar, Straus and Giroux, 18 West 18th Street New York, NY US.
[63]
Selina Meyer, David Elsweiler, Bernd Ludwig, Marcos Fernandez-Pichel, and David E Losada. 2022. Do we still need human assessors? prompt-based gpt-3 user simulation in conversational ai. In Proceedings of the 4th Conference on Conversational User Interfaces. Association for Computing Machinery, New York, NY, USA, 1–6.
[64]
Holland Michel. 2020. A.: The black box, unlocked: predictability and understandability in military AI. Ginebra, United Nations Institute for Disarmament Research.
[65]
Microsoft. 2013. Reinventing search with a new AI-powered Microsoft Bing and Edge, your copilot for the web. https://blogs.microsoft.com/blog/2023/02/07/reinventing-search-with-a-new-ai-powered-microsoft-bing-and-edge-your-copilot-for-the-web/.
[66]
Bonan Min, Hayley Ross, Elior Sulem, Amir Pouran Ben Veyseh, Thien Huu Nguyen, Oscar Sainz, Eneko Agirre, Ilana Heintz, and Dan Roth. 2023. Recent advances in natural language processing via large pre-trained language models: A survey. Comput. Surveys 56, 2 (2023), 1–40.
[67]
Piotr Mirowski, Kory W. Mathewson, Jaylen Pittman, and Richard Evans. 2023. Co-Writing Screenplays and Theatre Scripts with Language Models: Evaluation by Industry Professionals. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI ’23). Association for Computing Machinery, New York, NY, USA, Article 355, 34 pages. https://doi.org/10.1145/3544548.3581225
[68]
OpenAI. 2022. Introducing ChatGPT. https://openai.com/blog/chatgpt.
[69]
OpenAI. 2023. GPT-4 Technical Report. arxiv:2303.08774 [cs.CL]
[70]
Jonas Oppenlaender. 2022. The Creativity of Text-to-Image Generation. In Proceedings of the 25th International Academic Mindtrek Conference (Tampere, Finland) (Academic Mindtrek ’22). Association for Computing Machinery, New York, NY, USA, 192–202. https://doi.org/10.1145/3569219.3569352
[71]
Bryan O’Sullivan. 2009. Making sense of revision-control systems. Commun. ACM 52, 9 (2009), 56–62.
[72]
Sundar Pichai. 2023. An important next step on our AI journey. https://blog.google/technology/ai/bard-google-ai-search-updates/.
[73]
Sundar Pichai and Demis Hassabis. 2023. Introducing Gemini: Our largest and most capable AI model. https://blog.google/technology/ai/google-gemini-ai/#sundar-note
[74]
John Plaice and William W Wadge. 1993. A new approach to version control. IEEE transactions on Software Engineering 19, 3 (1993), 268–276.
[75]
James P. Purdy. 2014. What Can Design Thinking Offer Writing Studies?College Composition and Communication 65, 4 (2014), 612–641. http://www.jstor.org/stable/43490875
[76]
Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, 2018. Improving language understanding by generative pre-training.
[77]
Sarah Ransdell, C. Michael Levy, and Ronald T. Kellogg. 2002. The structure of writing processes as revealed by secondary task demands. L1-Educational Studies in Language and Literature 2, 2 (May 2002), 141–163. https://doi.org/10.1023/A:1020851300668
[78]
Horst Rittel. 1967. Wicked problems. Management Science,(December 1967) 4, 14 (1967), B141–B142.
[79]
Michele Salvagno, Fabio Silvio Taccone, and Alberto Giovanni Gerli. 2023. Artificial intelligence hallucinations. Critical Care 27, 1 (2023), 1–2.
[80]
Reinhard Sefelin, Manfred Tscheligi, and Verena Giller. 2003. Paper prototyping - what is it good for? a comparison of paper- and computer-based low-fidelity prototyping. In CHI ’03 Extended Abstracts on Human Factors in Computing Systems (Ft. Lauderdale, Florida, USA) (CHI EA ’03). Association for Computing Machinery, New York, NY, USA, 778–779. https://doi.org/10.1145/765891.765986
[81]
Anthony Seow. 2002. The writing process and process writing. Methodology in language teaching: An anthology of current practice 315 (2002), 320.
[82]
Hanieh Shakeri, Carman Neustaedter, and Steve DiPaola. 2021. Saga: Collaborative storytelling with gpt-3. In Companion Publication of the 2021 Conference on Computer Supported Cooperative Work and Social Computing. Association for Computing Machinery, New York, NY, USA, 163–166.
[83]
Mike Sharples. 2002. How we write: Writing as creative design. Routledge, Milton Park, Abingdon, Oxfordshire, United Kingdom.
[84]
Nikhil Singh, Guillermo Bernal, Daria Savchenko, and Elena L Glassman. 2023. Where to hide a stolen elephant: Leaps in creative writing with multimodal machine intelligence. ACM Transactions on Computer-Human Interaction 30, 5 (2023), 1–57.
[85]
Andrew L Smith, Felix Greaves, and Trishan Panch. 2023. Hallucination or Confabulation? Neuroanatomy as metaphor in Large Language Models. PLOS Digital Health 2, 11 (2023), e0000388.
[86]
Carolyn Snyder. 2003. Paper Prototyping: The Fast and Easy Way to Design and Refine User Interfaces. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA.
[87]
Nancy Sommers. 1980. Revision strategies of student writers and experienced adult writers. College composition and communication 31, 4 (1980), 378–388.
[88]
Walter F. Tichy. 1982. Design, implementation, and evaluation of a Revision Control System. In Proceedings of the 6th International Conference on Software Engineering (Tokyo, Japan) (ICSE ’82). IEEE Computer Society Press, Washington, DC, USA, 58–67.
[89]
Maryam Tohidi, William Buxton, Ronald Baecker, and Abigail Sellen. 2006. Getting the Right Design and the Design Right. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Montréal, Québec, Canada) (CHI ’06). Association for Computing Machinery, New York, NY, USA, 1243–1252. https://doi.org/10.1145/1124772.1124960
[90]
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, 2023. Llama 2: Open foundation and fine-tuned chat models.
[91]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017. Attention is All you Need. In Advances in Neural Information Processing Systems, I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.). Vol. 30. Curran Associates, Inc., Long Beach, CA. https://proceedings.neurips.cc/paper_files/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf
[92]
Miriam Walker, Leila Takayama, and James A. Landay. 2002. High-Fidelity or Low-Fidelity, Paper or Computer? Choosing Attributes when Testing Web Prototypes. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 46, 5 (2002), 661–665. https://doi.org/10.1177/154193120204600513 arXiv:https://doi.org/10.1177/154193120204600513
[93]
Yunlong Wang, Shuyuan Shen, and Brian Y Lim. 2023. RePrompt: Automatic Prompt Editing to Refine AI-Generative Art Towards Precise Expressions. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems(CHI ’23). ACM, Singapore, 1–29. https://doi.org/10.1145/3544548.3581402
[94]
Tongshuang Wu, Ellen Jiang, Aaron Donsbach, Jeff Gray, Alejandra Molina, Michael Terry, and Carrie J Cai. 2022. Promptchainer: Chaining large language model prompts through visual programming. In CHI Conference on Human Factors in Computing Systems Extended Abstracts. Association for Computing Machinery, New York, NY, USA, 1–10.
[95]
Roman V Yampolskiy. 2019. Unpredictability of AI.
[96]
Roman V Yampolskiy. 2020. Unpredictability of AI: On the impossibility of accurately predicting all actions of a smarter agent. Journal of Artificial Intelligence and Consciousness 7, 01 (2020), 109–118.
[97]
Ann Yuan, Andy Coenen, Emily Reif, and Daphne Ippolito. 2022. Wordcraft: Story Writing With Large Language Models. In 27th International Conference on Intelligent User Interfaces (Helsinki, Finland) (IUI ’22). Association for Computing Machinery, New York, NY, USA, 841–852. https://doi.org/10.1145/3490099.3511105
[98]
J.D. Zamfirescu-Pereira, Richmond Y. Wong, Bjoern Hartmann, and Qian Yang. 2023. Why Johnny Can’t Prompt: How Non-AI Experts Try (and Fail) to Design LLM Prompts. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI ’23). Association for Computing Machinery, New York, NY, USA, Article 437, 21 pages. https://doi.org/10.1145/3544548.3581388
[99]
Lei Zhang, Ashutosh Agrawal, Steve Oney, and Anhong Guo. 2023. VRGit: A Version Control System for Collaborative Content Creation in Virtual Reality. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI ’23). Association for Computing Machinery, New York, NY, USA, Article 36, 14 pages. https://doi.org/10.1145/3544548.3581136
[100]
Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba. 2022. Large language models are human-level prompt engineers.
[101]
William Knowlton Zinsser. 2001. On writing well: The classic guide to writing nonfiction. Quill/A Harper Collins Books, 195 Broadway New York, NY, USA.

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
CHI '24: Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems
May 2024
18961 pages
ISBN:9798400703300
DOI:10.1145/3613904
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 May 2024

Permissions

Request permissions for this article.

Check for updates

Badges

Author Tags

  1. datasets
  2. gaze detection
  3. neural networks
  4. text tagging

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Funding Sources

Conference

CHI '24

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025
ACM CHI Conference on Human Factors in Computing Systems
April 26 - May 1, 2025
Yokohama , Japan

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 1,496
    Total Downloads
  • Downloads (Last 12 months)1,496
  • Downloads (Last 6 weeks)117
Reflects downloads up to 20 Jan 2025

Other Metrics

Citations

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Full Text

View this article in Full Text.

Full Text

HTML Format

View this article in HTML Format.

HTML Format

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media