Yihao Zhu
ABSTRACT
Nowadays, novice users often turn to digital tutorials for guidance in software. However, searching and utilizing the tutorial remains a challenge due to the request for proper problem articulation, extensive searches and mind-intensive follow-through. We introduce "Docent", a system designed to bridge this knowledge-seeking gap. Powered by Large Language Models (LLMs), Docent takes vague user input and recent digital operation contexts to reason, seek, and present the most relevant tutorials in-situ. We assume that Docent smooths the user experience and facilitates learning of the software.
- Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners. In Proceedings of the 34th International Conference on Neural Information Processing Systems(NIPS’20). Curran Associates Inc., Red Hook, NY, USA, 1877–1901.Google Scholar
Digital Library
- Pei-Yu Chi, Bongshin Lee, and Steven M. Drucker. 2014. DemoWiz: re-performing software demonstrations for a live presentation. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, Toronto Ontario Canada, 1581–1590. https://doi.org/10.1145/2556288.2557254Google ScholarDigital Library
- Pramod Chundury, Mehmet Adil Yalçin, Jonathan Crabtree, Anup Mahurkar, Lisa M Shulman, and Niklas Elmqvist. 2023. Contextual in situ help for visual data interfaces. Information Visualization 22, 1 (Jan. 2023), 69–84. https://doi.org/10.1177/14738716221120064Google ScholarCross Ref
- C. Ailie Fraser, Tricia J. Ngoon, Mira Dontcheva, and Scott Klemmer. 2019. RePlay: Contextually Presenting Learning Videos Across Software Applications. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, Glasgow Scotland Uk, 1–13. https://doi.org/10.1145/3290605.3300527Google ScholarDigital Library
- Ben Lafreniere, Andrea Bunt, Matthew Lount, and Michael Terry. [n. d.]. “Looks cool, I’ll try this later!”: Understanding the Faces and Uses of Online Tutorials. ([n. d.]).Google Scholar
- Tessa Lau, Clemens Drews, and Jeffrey Nichols. [n. d.]. Interpreting Written How-To Instructions. ([n. d.]).Google Scholar
- Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013).Google Scholar
- Alok Mysore and Philip J. Guo. 2017. Torta: Generating Mixed-Media GUI and Command-Line App Tutorials Using Operating-System-Wide Activity Tracing. In Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology. ACM, Québec City QC Canada, 703–714. https://doi.org/10.1145/3126594.3126628Google ScholarDigital Library
- Cuong Nguyen and Feng Liu. 2015. Making Software Tutorial Video Responsive. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM, Seoul Republic of Korea, 1565–1568. https://doi.org/10.1145/2702123.2702209Google ScholarDigital Library
- Srishti Palani, Zijian Ding, Stephen MacNeil, and Steven P. Dow. 2021. The "Active Search" Hypothesis: How Search Strategies Relate to Creative Learning. In Proceedings of the 2021 Conference on Human Information Interaction and Retrieval. ACM, Canberra ACT Australia, 325–329. https://doi.org/10.1145/3406522.3446046Google ScholarDigital Library
- Luca Ponzanelli, Gabriele Bavota, Andrea Mocci, Massimiliano Di Penta, Rocco Oliveto, Mir Hasan, Barbara Russo, Sonia Haiduc, and Michele Lanza. 2016. Too long; didn’t watch!: extracting relevant fragments from software development video tutorials. In Proceedings of the 38th International Conference on Software Engineering. ACM, Austin Texas, 261–272. https://doi.org/10.1145/2884781.2884824Google ScholarDigital Library
- Luca Ponzanelli, Gabriele Bavota, Andrea Mocci, Massimiliano Di Penta, Rocco Oliveto, Barbara Russo, Sonia Haiduc, and Michele Lanza. 2016. CodeTube: extracting relevant fragments from software development video tutorials. In Proceedings of the 38th International Conference on Software Engineering Companion. ACM, Austin Texas, 645–648. https://doi.org/10.1145/2889160.2889172Google ScholarDigital Library
- Soo Young Rieh, Kevyn Collins-Thompson, Preben Hansen, and Hye-Jung Lee. 2016. Towards searching as a learning process: A review of current perspectives and future directions. Journal of Information Science 42, 1 (2016), 19–34.Google ScholarDigital Library
- Mingyuan Zhong, Gang Li, Peggy Chi, and Yang Li. 2021. HelpViz: Automatic Generation of Contextual Visual Mobile Tutorials from Text-Based Instructions. In The 34th Annual ACM Symposium on User Interface Software and Technology. ACM, Virtual Event USA, 1144–1153. https://doi.org/10.1145/3472749.3474812Google ScholarDigital Library
Index Terms
- Docent: Digital Operation-Centric Elicitation of Novice-friendly Tutorials
Recommendations
MixT: automatic generation of step-by-step mixed media tutorials
UIST '12: Proceedings of the 25th annual ACM symposium on User interface software and technologyUsers of complex software applications often learn concepts and skills through step-by-step tutorials. Today, these tutorials are published in two dominant forms: static tutorials composed of images and text that are easy to scan, but cannot effectively ...
Docable: evaluating the executability of software tutorials
ESEC/FSE 2020: Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software EngineeringThe typical software tutorial includes step-by-step instructions for installing developer tools, editing files and code, and running commands. When these software tutorials are not executable, either due to missing instructions, ambiguous steps, or ...
Torta: Generating Mixed-Media GUI and Command-Line App Tutorials Using Operating-System-Wide Activity Tracing
UIST '17: Proceedings of the 30th Annual ACM Symposium on User Interface Software and TechnologyTutorials are vital for helping people perform complex software-based tasks in domains such as programming, data science, system administration, and computational research. However, it is tedious to create detailed step-by-step tutorials for tasks that ...
Comments