skip to main content
10.1145/3544548.3581525acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
research-article

Colaroid: A Literate Programming Approach for Authoring Explorable Multi-Stage Tutorials

Published: 19 April 2023 Publication History

Abstract

Multi-stage programming tutorials are key learning resources for programmers, using progressive incremental steps to teach them how to build larger software systems. A good multi-stage tutorial describes the code clearly, explains the rationale and code changes for each step, and allows readers to experiment as they work through the tutorial. In practice, it is time-consuming for authors to create tutorials with these attributes. In this paper, we introduce Colaroid, an interactive authoring tool for creating high quality multi-stage tutorials. Colaroid tutorials are augmented computational notebooks, where snippets and outputs represent a snapshot of a project, with source code differences highlighted, complete source code context for each snippet, and the ability to load and tinker with any stage of the project in a linked IDE. In two laboratory studies, we found Colaroid makes it easy to create multi-stage tutorials, while offering advantages to readers compared to video and web-based tutorials.

Supplementary Material

MP4 File (3544548.3581525-video-figure.mp4)
Video Figure
MP4 File (3544548.3581525-talk-video.mp4)
Pre-recorded Video Presentation

References

[1]
2022. CodeTour. https://marketplace.visualstudio.com/items?itemName=vsls-contrib.codetour
[2]
Lawrence Bergman, Vittorio Castelli, Tessa Lau, and Daniel Oblinger. 2005. DocWizards: a system for authoring follow-me documentation wizards. In Proceedings of the 18th annual ACM symposium on User interface software and technology. 191–200.
[3]
Raymond PL Buse and Westley R Weimer. 2010. Automatically documenting program changes. In Proceedings of the IEEE/ACM international conference on Automated software engineering. 33–42.
[4]
Paul Cairns and Jeremy Gow. 2005. Literate proving: presenting and documenting formal proofs. In International Conference on Mathematical Knowledge Management. Springer, 159–173.
[5]
Souti Chattopadhyay, Ishita Prasad, Austin Z Henley, Anita Sarma, and Titus Barik. 2020. What’s wrong with computational notebooks? Pain points, needs, and design opportunities. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–12.
[6]
Charles H Chen and Philip J Guo. 2019. Improv: Teaching programming at scale via live coding. In Proceedings of the Sixth (2019) ACM Conference on Learning@ Scale. 1–10.
[7]
Pei-Yu Chi, Sally Ahn, Amanda Ren, Mira Dontcheva, Wilmot Li, and Björn Hartmann. 2012. MixT: automatic generation of step-by-step mixed media tutorials. In Proceedings of the 25th annual ACM symposium on User interface software and technology. 93–102.
[8]
Fulvio Corno, Luigi De Russis, and Juan Pablo Sáenz. 2019. Towards computational notebooks for IoT development. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. 1–6.
[9]
Barthélémy Dagenais and Martin P Robillard. 2010. Creating and evolving developer documentation: understanding the decisions of open source contributors. In Proceedings of the eighteenth ACM SIGSOFT international symposium on Foundations of software engineering. 127–136.
[10]
Alan Davies, Frances Hooley, Peter Causey-Freeman, Iliada Eleftheriou, and Georgina Moulton. 2020. Using interactive digital notebooks for bioscience and informatics education. PLoS computational biology 16, 11 (2020), e1008326.
[11]
Eve 2020. Eve: Programming designed for humans. http://witheve.com/
[12]
Travis Faas, Lynn Dombrowski, Alyson Young, and Andrew D Miller. 2018. Watch me code: Programming mentorship communities on twitch. tv. Proceedings of the ACM on Human-Computer Interaction 2, CSCW(2018), 1–18.
[13]
Shiry Ginosar, Luis Fernando De Pombo, Maneesh Agrawala, and Bjorn Hartmann. 2013. Authoring multi-stage code examples with editable code histories. In Proceedings of the 26th annual ACM symposium on User interface software and technology. 485–494.
[14]
Mitchell Gordon and Philip J Guo. 2015. Codepourri: Creating visual coding tutorials using a volunteer crowd of learners. In 2015 IEEE symposium on visual languages and human-centric computing (VL/HCC). IEEE, 13–21.
[15]
Floraine Grabler, Maneesh Agrawala, Wilmot Li, Mira Dontcheva, and Takeo Igarashi. 2009. Generating photo manipulation tutorials by demonstration. In ACM SIGGRAPH 2009 papers. 1–9.
[16]
Tovi Grossman, Justin Matejka, and George Fitzmaurice. 2010. Chronicle: capture, exploration, and playback of document workflow histories. In Proceedings of the 23nd annual ACM symposium on User interface software and technology. 143–152.
[17]
Andrew Head, Elena L Glassman, Björn Hartmann, and Marti A Hearst. 2018. Interactive extraction of examples from existing code. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1–12.
[18]
Andrew Head, Fred Hohman, Titus Barik, Steven M Drucker, and Robert DeLine. 2019. Managing messes in computational notebooks. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–12.
[19]
Andrew Head, Jason Jiang, James Smith, Marti A Hearst, and Björn Hartmann. 2020. Composing flexibly-organized step-by-step tutorials from linked source code, snippets, and outputs. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–12.
[20]
Fred Hohman, Kanit Wongsuphasawat, Mary Beth Kery, and Kayur Patel. 2020. Understanding and visualizing data iteration in machine learning. In Proceedings of the 2020 CHI conference on human factors in computing systems. 1–13.
[21]
Mike Horn, Amartya Banerjee, and Matthew Brucker. 2022. TunePad Playbooks: Designing Computational Notebooks for Creative Music Coding. In CHI Conference on Human Factors in Computing Systems. 1–12.
[22]
Mary Beth Kery, Bonnie E John, Patrick O’Flaherty, Amber Horvath, and Brad A Myers. 2019. Towards effective foraging by data scientists to find past analysis choices. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–13.
[23]
Mary Beth Kery, Marissa Radensky, Mahima Arya, Bonnie E John, and Brad A Myers. 2018. The story in the notebook: Exploratory data science using a literate programming tool. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1–11.
[24]
Ada S Kim and Amy J Ko. 2017. A pedagogical analysis of online coding tutorials. In Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education. 321–326.
[25]
Youngtaek Kim, Jaeyoung Kim, Hyeon Jeon, Young-Ho Kim, Hyunjoo Song, Bohyoung Kim, and Jinwook Seo. 2020. Githru: visual analytics for understanding software development history through git metadata analysis. IEEE Transactions on Visualization and Computer Graphics 27, 2(2020), 656–666.
[26]
Donald Ervin Knuth. 1984. Literate programming. The computer journal 27, 2 (1984), 97–111.
[27]
Kenneth R Koedinger, Elizabeth A McLaughlin, Julianna Zhuxin Jia, and Norman L Bier. 2016. Is the doer effect a causal relationship? How can we tell and why it’s important. In Proceedings of the sixth international conference on learning analytics & knowledge. 388–397.
[28]
Rebecca Krosnick, Fraser Anderson, Justin Matejka, Steve Oney, Walter S. Lasecki, Tovi Grossman, and George Fitzmaurice. 2021. Think-Aloud Computing: Supporting Rich and Low-Effort Knowledge Capture. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–13.
[29]
Sean Kross and Philip J Guo. 2019. Practitioners teaching data science in industry and academia: Expectations, workflows, and challenges. In Proceedings of the 2019 CHI conference on human factors in computing systems. 1–14.
[30]
Mario Linares-Vásquez, Luis Fernando Cortés-Coy, Jairo Aponte, and Denys Poshyvanyk. 2015. Changescribe: A tool for automatically generating commit messages. In 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering, Vol. 2. IEEE, 709–712.
[31]
Mark Mahoney. 2018. Storyteller: a tool for creating worked examples. Journal of Computing Sciences in Colleges 34, 1 (2018), 137–144.
[32]
Brad A Myers. 1991. Separating application code from toolkits: Eliminating the spaghetti of call-backs. In Proceedings of the 4th annual ACM symposium on User interface software and technology. 211–220.
[33]
Alok Mysore and Philip J Guo. 2017. Torta: Generating mixed-media gui and command-line app tutorials using operating-system-wide activity tracing. In Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology. 703–714.
[34]
Seyed Mehdi Nasehi, Jonathan Sillito, Frank Maurer, and Chris Burns. 2012. What makes a good code example?: A study of programming Q&A in StackOverflow. In 2012 28th IEEE International Conference on Software Maintenance (ICSM). IEEE, 25–34.
[35]
Observable 2020. Observable: the magic notebook for exploring data and thinking with code.https://observablehq.com/
[36]
Steve Oney, Christopher Brooks, and Paul Resnick. 2018. Creating guided code explanations with chat. codes. Proceedings of the ACM on Human-Computer Interaction 2, CSCW(2018), 1–20.
[37]
Chris Parnin, Christoph Treude, and Margaret-Anne Storey. 2013. Blogging developer knowledge: Motivations, challenges, and future directions. In 2013 21st International Conference on Program Comprehension (ICPC). IEEE, 211–214.
[38]
Jeffrey M Perkel. 2018. Why Jupyter is data scientists’ computational notebook of choice. Nature 563, 7732 (2018), 145–147.
[39]
Clément Pit-Claudel. 2020. Untangling mechanized proofs. In Proceedings of the 13th ACM SIGPLAN International Conference on Software Language Engineering. 155–174.
[40]
Suporn Pongnumkul, Mira Dontcheva, Wilmot Li, Jue Wang, Lubomir Bourdev, Shai Avidan, and Michael F Cohen. 2011. Pause-and-play: automatically linking screencast video tutorials with applications. In Proceedings of the 24th annual ACM symposium on User interface software and technology. 135–144.
[41]
Roman Rädle, Midas Nouwens, Kristian Antonsen, James R Eagan, and Clemens N Klokmose. 2017. Codestrates: Literate computing with webstrates. In Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology. 715–725.
[42]
Ernst Z Rothkopf and MJ Billington. 1979. Goal-guided learning from text: inferring a descriptive processing model from inspection times and eye movements.Journal of educational psychology 71, 3 (1979), 310.
[43]
Adam Carl Rule. 2018. Design and use of computational notebooks. University of California, San Diego.
[44]
Huascar Sanchez, Jim Whitehead, and Martin Schäf. 2016. Multistaging to understand: Distilling the essence of java code examples. In 2016 IEEE 24th International Conference on Program Comprehension (ICPC). IEEE, 1–10.
[45]
Anselm Strauss and Juliet Corbin. 1994. Grounded theory methodology: An overview.(1994).
[46]
Brad Victor. 2011. Explorable Explanations. (2011). http://worrydream.com/ExplorableExplanations/
[47]
April Yi Wang, Will Epperson, Robert A DeLine, and Steven M Drucker. 2022. Diff in the Loop: Supporting Data Comparison in Exploratory Data Analysis. In CHI Conference on Human Factors in Computing Systems. 1–10.
[48]
April Yi Wang, Anant Mittal, Christopher Brooks, and Steve Oney. 2019. How data scientists use computational notebooks for real-time collaboration. Proceedings of the ACM on Human-Computer Interaction 3, CSCW(2019), 1–30.
[49]
April Yi Wang, Zihan Wu, Christopher Brooks, and Steve Oney. 2020. Callisto: Capturing the" Why" by Connecting Conversations with Computational Narratives. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–13.
[50]
Nathaniel Weinman, Steven M Drucker, Titus Barik, and Robert DeLine. 2021. Fork It: Supporting stateful alternatives in computational notebooks. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–12.
[51]
Wikipedia contributors. 2022. Instant camera — Wikipedia, The Free Encyclopedia. https://en.wikipedia.org/w/index.php?title=Instant_camera&oldid=1109305997 [Online; accessed 15-September-2022].
[52]
Stephen Wolfram. 2003. The mathematica book. Vol. 1. Wolfram Research, Inc.
[53]
Yihui Xie. 2018. knitr: a comprehensive tool for reproducible research in R. In Implementing reproducible research. Chapman and Hall/CRC, 3–31.
[54]
Tom Yeh, Tsung-Hsiang Chang, and Robert C Miller. 2009. Sikuli: using GUI screenshots for search and automation. In Proceedings of the 22nd annual ACM symposium on User interface software and technology. 183–192.

Cited By

View all
  • (2024)Do LLMs Meet the Needs of Software Tutorial Writers? Opportunities and Design ImplicationsProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3660692(1760-1773)Online publication date: 1-Jul-2024
  • (2024)Explorable Explainable AI: Improving AI Understanding for Community Health Workers in IndiaProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642733(1-21)Online publication date: 11-May-2024
  • (2023)Support for Long-Form Documentation Authoring and Maintenance2023 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC)10.1109/VL-HCC57772.2023.00020(109-114)Online publication date: 3-Oct-2023

Index Terms

  1. Colaroid: A Literate Programming Approach for Authoring Explorable Multi-Stage Tutorials

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    CHI '23: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems
    April 2023
    14911 pages
    ISBN:9781450394215
    DOI:10.1145/3544548
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 19 April 2023

    Permissions

    Request permissions for this article.

    Check for updates

    Badges

    • Honorable Mention

    Author Tags

    1. computational notebooks
    2. instruction
    3. programming
    4. tutorials

    Qualifiers

    • Research-article
    • Research
    • Refereed limited

    Funding Sources

    • NSF

    Conference

    CHI '23
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

    Upcoming Conference

    CHI 2025
    ACM CHI Conference on Human Factors in Computing Systems
    April 26 - May 1, 2025
    Yokohama , Japan

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)349
    • Downloads (Last 6 weeks)18
    Reflects downloads up to 16 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Do LLMs Meet the Needs of Software Tutorial Writers? Opportunities and Design ImplicationsProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3660692(1760-1773)Online publication date: 1-Jul-2024
    • (2024)Explorable Explainable AI: Improving AI Understanding for Community Health Workers in IndiaProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642733(1-21)Online publication date: 11-May-2024
    • (2023)Support for Long-Form Documentation Authoring and Maintenance2023 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC)10.1109/VL-HCC57772.2023.00020(109-114)Online publication date: 3-Oct-2023

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Full Text

    View this article in Full Text.

    Full Text

    HTML Format

    View this article in HTML Format.

    HTML Format

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media