skip to main content
10.1145/3643834.3661596acmconferencesArticle/Chapter ViewAbstractPublication PagesdisConference Proceedingsconference-collections
research-article

Does the Medium Matter? An Exploration of Voice-Interaction for Self-Explanations

Published: 01 July 2024 Publication History

Abstract

This research evaluates voice-based self-explanations as a pedagogical tool in preparation for lectures, assesses user preferences between voice and text, and derives design insights. We report two studies: Study 1, a quasi-experimental field study, with 247 participants divided into voice-based (N = 83), text-based (N = 81), and choice (N = 83) conditions. Study 2 uses semi-structured interviews (N = 16) to explore perceptions of the interaction paradigms in-depth. Results from the first study revealed a general preference for text, though voice users produced longer responses and more topic-related keywords. Over time, the preference for voice increased among students, from 10% to 46%, when given a choice. Study 2 suggested that factors like social presence contribute to hesitance toward voice-based explanations, with a cognitive load, self-confidence, and performance anxiety also influencing medium preferences. Our findings highlight design recommendations and demonstrate the potential of voice-based self-explanations in educational settings, indicating that mixed interfaces might better meet diverse needs.

References

[1]
Shaaron Ainsworth and Sarah Burcham. 2007. The impact of text coherence on learning by self-explanation. Learning and instruction 17, 3 (2007), 286–303.
[2]
Yuya Asano, Jaemarie Solyst, and Joseph Jay Williams. 2020. Characterizing and influencing students’ tendency to write self-explanations in online homework. In Proceedings of the Tenth International Conference on Learning Analytics & Knowledge. 448–453.
[3]
Kirsten Berthold, Tessa HS Eysink, and Alexander Renkl. 2009. Assisting self-explanation prompts are more effective than open prompts when learning with multiple representations. Instructional Science 37 (2009), 345–363.
[4]
Kiran Bisra, Qing Liu, John C Nesbit, Farimah Salimi, and Philip H Winne. 2018. Inducing self-explanation: A meta-analysis. Educational Psychology Review 30, 3 (2018), 703–725.
[5]
Virginia Braun and Victoria Clarke. 2012. Thematic analysis.American Psychological Association.
[6]
Paul Brna, Richard Cox, and Judith Good. 2001. Learning to think and communicate with diagrams: 14 questions to consider. Artificial Intelligence Review 15, 1 (2001), 115–134.
[7]
Ian Bruce. 2008. Academic writing and genre: A systematic analysis. Bloomsbury Publishing.
[8]
Wallace Chafe. 1982. Integration and involvement in speaking, writing, and oral literature. Spoken and written language: Exploring orality and literacy (1982), 35–54.
[9]
Ying-Chih Chen, Soonhye Park, and Brian Hand. 2016. Examining the use of talk and writing for students’ development of scientific conceptual knowledge through constructing and critiquing arguments. Cognition and Instruction 34, 2 (2016), 100–147.
[10]
Alan Cheville and Brianna Healey Derr. 2016. Using videos to elicit self-explanations of emergent electromagnetic concepts. In 2016 ASEE Annual Conference & Exposition.
[11]
Michelene TH Chi. 1997. Quantifying qualitative analyses of verbal data: A practical guide. The journal of the learning sciences 6, 3 (1997), 271–315.
[12]
Michelene TH Chi. 2013. Self-explaining expository texts: The dual processes of generating inferences and repairing mental models. In Advances in instructional psychology. Routledge, 161–238.
[13]
Michelene TH Chi, Nicholas De Leeuw, Mei-Hung Chiu, and Christian LaVancher. 1994. Eliciting self-explanations improves understanding. Cognitive science 18, 3 (1994), 439–477.
[14]
Michelene TH Chi, Paul J Feltovich, and Robert Glaser. 1981. Categorization and representation of physics problems by experts and novices. Cognitive science 5, 2 (1981), 121–152.
[15]
Minyoung Cho. 2018. Task complexity, modality, and working memory in L2 task performance. System 72 (2018), 85–98.
[16]
Young Hoan Cho and David H Jonassen. 2012. Learning by self-explaining causal diagrams in high-school biology. Asia Pacific Education Review 13, 1 (2012), 171–184.
[17]
Herbert H Clark. 2002. Speaking in time. Speech communication 36, 1-2 (2002), 5–13.
[18]
Jacob Cohen. 2013. Statistical power analysis for the behavioral sciences. Academic press.
[19]
Cristina Conati and Kurt Vanlehn. 2000. Toward computer-based support of meta-cognitive skills: A computational framework to coach self-explanation. International Journal of Artificial Intelligence in Education 11 (2000), 389–415.
[20]
Richard Cox. 1999. Representation construction, externalised cognition and individual differences. Learning and instruction 9, 4 (1999), 343–363.
[21]
Fergus IM Craik and Robert S Lockhart. 1972. Levels of processing: A framework for memory research. Journal of verbal learning and verbal behavior 11, 6 (1972), 671–684.
[22]
Kathleen Cramer and Terry Wyberg. 2009. Efficacy of different concrete models for teaching the part-whole construct for fractions. Mathematical thinking and learning 11, 4 (2009), 226–257.
[23]
Björn B de Koning, Huib K Tabbers, Remy MJP Rikers, and Fred Paas. 2010. Learning by generating vs. receiving instructional explanations: Two approaches to enhance attention cueing in animations. Computers & Education 55, 2 (2010), 681–691.
[24]
Susan De La Paz and Steve Graham. 1997. Effects of dictation and advanced planning instruction on the composing of students with writing and learning problems.Journal of Educational Psychology 89, 2 (1997), 203.
[25]
Sidney K D’Mello, Nia Dowell, and Arthur Graesser. 2011. Does it really matter whether students’ contributions are spoken versus typed in an intelligent tutoring system with natural language?Journal of Experimental Psychology: Applied 17, 1 (2011), 1.
[26]
Sidney K D’Mello, Art Graesser, and Brandon King. 2010. Toward spoken human–computer tutorial dialogues. Human–Computer Interaction 25, 4 (2010), 289–323.
[27]
John Dunlosky, Katherine A Rawson, Elizabeth J Marsh, Mitchell J Nathan, and Daniel T Willingham. 2013. Improving students’ learning with effective learning techniques: Promising directions from cognitive and educational psychology. Psychological Science in the public interest 14, 1 (2013), 4–58.
[28]
Janet Emig. 2020. Writing as a mode of learning. In Landmark essays on writing across the curriculum. Routledge, 89–96.
[29]
Brent J Evans, Rachel B Baker, and Thomas S Dee. 2016. Persistence patterns in massive open online courses (MOOCs). The Journal of Higher Education 87, 2 (2016), 206–242.
[30]
Tatsushi Fukaya. 2013. Explanation generation, not explanation expectancy, improves metacomprehension accuracy. Metacognition and learning 8, 1 (2013), 1–18.
[31]
Peter Gerjets, Katharina Scheiter, and Richard Catrambone. 2006. Can learning from molar and modular worked examples be enhanced by providing instructional explanations and prompting self-explanations?Learning and Instruction 16, 2 (2006), 104–121.
[32]
Michail N Giannakos, John Krogstie, and Nikos Chrisochoides. 2014. Reviewing the flipped classroom research: reflections for computer science education. In Proceedings of the computer science education research conference. 23–29.
[33]
Joachim Grabowski. 2010. Speaking, writing, and memory span in children: Output modality affects cognitive performance. International Journal of Psychology 45, 1 (2010), 28–39.
[34]
James G Greeno. 1977. Process of understanding in problem solving. Cognitive theory 2 (1977).
[35]
Eric Hamilton, Richard Lesh, FRANK Lester, and Michael Brilleslyper. 2008. Model-eliciting activities (MEAs) as a bridge between engineering education research and mathematics education research.Advances in Engineering Education 1, 2 (2008), n2.
[36]
Robert GM Hausmann and MicheleneT H Chi. 2002. Can a computer interface support self-explaining. Cognitive Technology 7, 1 (2002), 4–14.
[37]
Robert GM Hausmann and Kurt VanLehn. 2010. The effect of self-explaining on robust learning. International Journal of Artificial Intelligence in Education 20, 4 (2010), 303–332.
[38]
Jarkko Hautala, Doris Luft Baker, Aleksi Keurulainen, Miia Ronimus, Ulla Richardson, and Ronald Cole. 2018. Early science learning with a virtual tutor through multimedia explanations and feedback on spoken questions. Educational Technology Research and Development 66, 2 (2018), 403–428.
[39]
Joan I Heller and James G Greeno. 1979. Information processing analyses of mathematical problem solving. Testing, teaching and learning. Washington, DC: National Institute of Education (1979).
[40]
Common Core State Standards Initiative 2010. National governors association center for best practices and council of chief state school officers. Retrieved December 11 (2010), 2012.
[41]
Leonie Jacob, Andreas Lachner, and Katharina Scheiter. 2020. Learning by explaining orally or in written form? Text complexity matters. Learning and Instruction 68 (2020), 101344.
[42]
Cheryl I Johnson and Richard E Mayer. 2010. Applying the self-explanation principle to multimedia learning in a computer-based game-like environment. Computers in Human Behavior 26, 6 (2010), 1246–1252.
[43]
Hyojung Jung, Seung Won Park, Hyun Song Kim, and Jongchan Park. 2022. The effects of the regulated learning-supported flipped classroom on student performance. Journal of Computing in Higher Education 34, 1 (2022), 132–153.
[44]
Anam Ahmad Khan, Sadia Nawaz, Joshua Newn, Ryan M Kelly, Jason M Lodge, James Bailey, and Eduardo Velloso. 2022. To type or to speak? The effect of input modality on text understanding during note-taking. In CHI Conference on Human Factors in Computing Systems. 1–15.
[45]
Karen A Kim, Amy J Fann, and Kimberly O Misa-Escalante. 2011. Engaging women in computer science and engineering: Promising practices for promoting gender equity in undergraduate research experiences. ACM Transactions on Computing Education (TOCE) 11, 2 (2011), 1–19.
[46]
Carlos Delgado Kloos, Carlos Alario-Hoyos, Pedro J Muñoz-Merino, Cristina Catalán Aguirre, and Nuria González Castro. 2019. Principles for the design of an educational voice assistant for learning Java. In International Conference on Sustainable ICT, Education, and Learning. Springer, 99–106.
[47]
Deanna Kuhn and Jared Katz. 2009. Are self-explanations always beneficial?Journal of Experimental Child Psychology 103, 3 (2009), 386–394.
[48]
Christopher A Kurby, Joseph P Magliano, Srikanth Dandotkar, James Woehrle, Sara Gilliam, and Danielle S McNamara. 2012. Changing how students process and comprehend texts with computer-based self-explanation training. Journal of Educational Computing Research 47, 4 (2012), 429–459.
[49]
Andreas Lachner, Iris Backfisch, Vincent Hoogerheide, Tamara van Gog, and Alexander Renkl. 2020. Timing matters! Explaining between study phases enhances students’ learning.Journal of Educational Psychology 112, 4 (2020), 841.
[50]
Andreas Lachner, Christian Burkhart, and Matthias Nückles. 2017. Mind the gap! Automated concept map feedback supports students in writing cohesive explanations.Journal of Experimental Psychology: Applied 23, 1 (2017), 29.
[51]
Andreas Lachner, Kim-Tek Ly, and Matthias Nückles. 2018. Providing written or oral explanations? Differential effects of the modality of explaining on students’ conceptual learning and transfer. The Journal of Experimental Education 86, 3 (2018), 344–361.
[52]
Linda Lebie, Jonathan A Rhoades, and Joseph E McGrath. 1995. Interaction process in computer-mediated and face-to-face groups. Computer Supported Cooperative Work (CSCW) 4, 2 (1995), 127–152.
[53]
R Lesh, G Carmona, and T Moore. 2009. Six sigma learning gains and long term retention of understandings and attitudes related to models & modeling. Mediterranean Journal for Research in Mathematics Education 9, 1 (2009), 19–54.
[54]
Diane J Litman, Carolyn P Rosé, Kate Forbes-Riley, Kurt VanLehn, Dumisizwe Bhembe, and Scott Silliman. 2006. Spoken versus typed human and computer dialogue tutoring. International Journal of Artificial Intelligence in Education 16, 2 (2006), 145–170.
[55]
Kate Lockwood and Rachel Esselstein. 2013. The inverted classroom and the CS curriculum. In Proceeding of the 44th ACM technical symposium on Computer science education. 113–118.
[56]
Mary Lou Maher, Celine Latulipe, Heather Lipford, and Audrey Rorrer. 2015. Flipped classroom strategies for CS education. In Proceedings of the 46th ACM Technical Symposium on Computer Science Education. 218–223.
[57]
James C McCroskey. 2009. Communication apprehension: What have we learned in the last four decades. Human Communication 12, 2 (2009), 157–171.
[58]
Mary L McHugh. 2012. Interrater reliability: the kappa statistic. Biochemia medica 22, 3 (2012), 276–282.
[59]
Natalie B Milman. 2012. The flipped classroom strategy: What is it and how can it best be used?Distance learning 9, 3 (2012), 85.
[60]
Tamara J Moore, Ronald L Miller, Richard A Lesh, Micah S Stohlmann, and Young Rae Kim. 2013. Modeling in engineering: The role of representational fluency in students’ conceptual understanding. Journal of Engineering Education 102, 1 (2013), 141–178.
[61]
Roxana Moreno and Richard Mayer. 2007. Interactive multimodal learning environments. Educational psychology review 19, 3 (2007), 309–326.
[62]
Roxana Moreno and Richard E Mayer. 2005. Role of guidance, reflection, and interactivity in an agent-based multimedia game.Journal of educational psychology 97, 1 (2005), 117.
[63]
Jarrod Moss, Kenneth Kotovsky, and Jonathan Cagan. 2006. The role of functionality in the mental representations of engineering students: Some differences in the early stages of expertise. Cognitive Science 30, 1 (2006), 65–93.
[64]
Tenaha O’Reilly, Sonya Symons, and Heather MacLatchy-Gaudet. 1998. A comparison of self-explanation and elaborative interrogation. Contemporary Educational Psychology 23, 4 (1998), 434–445.
[65]
Harold Pashler, Patrice M Bain, Brian A Bottge, Arthur Graesser, Kenneth Koedinger, Mark McDaniel, and Janet Metcalfe. 2007. Organizing Instruction and Study to Improve Student Learning. IES Practice Guide. NCER 2007-2004.National Center for Education Research (2007).
[66]
Celeste Pilegard and Logan Fiorella. 2016. Helping students help themselves: Generative learning strategies improve middle school students’ self-regulation in a cognitive tutor. Computers in Human Behavior 65 (2016), 121–126.
[67]
Heather Pon-Barry, Brady Clark, Karl Schultz, Elizabeth Owen Bratt, Stanley Peters, and David Haley. 2005. Contextualizing reflective dialogue in a spoken conversational tutor. Journal of Educational Technology & Society 8, 4 (2005), 42–51.
[68]
Alexander Renkl. 1997. Learning from worked-out examples: A study on individual differences. Cognitive science 21, 1 (1997), 1–29.
[69]
Alexander Renkl and Robert K Atkinson. 2002. Learning from examples: Fostering self-explanations in computer-based learning environments. Interactive learning environments 10, 2 (2002), 105–119.
[70]
Alexander Renkl, Robert K Atkinson, and Uwe H Maier. 2000. From studying examples to solving problems: Fading worked-out solution steps helps learning. In Proceeding of the 22nd Annual Conference of the Cognitive Science Society. 393–398.
[71]
Rod D Roscoe and Michelene TH Chi. 2008. Tutor learning: The role of explaining and responding to questions. Instructional science 36, 4 (2008), 321–350.
[72]
Mary Ryan. 2011. Improving reflective writing in higher education: A social semiotic perspective. Teaching in Higher Education 16, 1 (2011), 99–111.
[73]
Alpaslan Sahin, Baki Cavlazoglu, and Yunus E Zeytuncu. 2015. Flipping a college calculus course: A case study. Journal of Educational Technology & Society 18, 3 (2015), 142–152.
[74]
Silke Schworm and Alexander Renkl. 2006. Computer-supported example-based learning: When instructional explanations reduce self-explanations. Computers & Education 46, 4 (2006), 426–445.
[75]
Naaz Sibia and Michael Liut. 2022. The Positive Effects of using Reflective Prompts in a Database Course. In 1st International Workshop on Data Systems Education (Philadelphia, PA, USA) (DataEd ’22). Association for Computing Machinery, New York, NY, USA, 32–37. https://doi.org/10.1145/3531072.3535323
[76]
Naaz Sibia, Angela Zavaleta Bernuy, Elexandra Tran, Jessica Jia-Ni Xu, Joseph Jay Williams, Andrew Petersen, and Michael Liut. 2024. Exploring Self-Explanations in a Flipped Database Course. In 3rd International Workshop on Data Systems Education (Santiago, Chile) (DataEd ’24). Association for Computing Machinery, New York, NY, USA, 7 pages.
[77]
Jerry Chih-Yuan Sun and Yu-Ting Wu. 2016. Analysis of learning achievement and teacher–student interactions in flipped and conventional classrooms. International Review of Research in Open and Distributed Learning 17, 1 (2016), 79–99.
[78]
Deborah Tannen. 1982. Oral and literate strategies in spoken and written narratives. Language (1982), 1–21.
[79]
Harriet R Tenenbaum, Louis Alfieri, Patricia J Brooks, and Guler Dunne. 2008. The effects of explanatory conversations on children’s emotion understanding. British Journal of Developmental Psychology 26, 2 (2008), 249–263.
[80]
David R Thomas. 2006. A general inductive approach for analyzing qualitative evaluation data. American journal of evaluation 27, 2 (2006), 237–246.
[81]
Louisa Tomas, Neus Snowy Evans, Tanya Doyle, and Keith Skamp. 2019. Are first year students ready for a flipped classroom? A case for a flipped learning continuum. International Journal of Educational Technology in Higher Education 16, 1 (2019), 1–22.
[82]
Peggy Van Meter, Maja Aleksic, Ana Schwartz, and Joanna Garner. 2006. Learner-generated drawing as a strategy for learning from content area text. Contemporary Educational Psychology 31, 2 (2006), 142–166.
[83]
Lev Semenovich Vygotsky and Michael Cole. 1978. Mind in society: Development of higher psychological processes. Harvard university press.
[84]
Thiemo Wambsganss, Rainer Winkler, Pascale Schmid, and Matthias Söllner. 2020. Designing a conversational agent as a formative course evaluation tool. (2020).
[85]
Muyu Wang, Naaz Sibia, Ilir Dema, Michael Liut, and Carlos Aníbal Suárez. 2021. Building a Better SQL Automarker for Database Courses. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3488042.3489970
[86]
Sheila Haley Wathen. 1997. Collaborative versus individualistic learning and the role of explanations. Ph. D. Dissertation. University of Pittsburgh.
[87]
Bernadette Wilkin. 1997. Learning from explanations: Diagrams can “inhibit” the self-explanation effect. Reasoning with diagrammatic representations II (1997), 136–143.
[88]
William Winn. 1991. Learning from maps and diagrams. Educational Psychology Review 3, 3 (1991), 211–247.
[89]
Merlin C Wittrock. 1989. Generative processes of comprehension. Educational psychologist 24, 4 (1989), 345–376.
[90]
Yunhan Wu, Daniel Rough, Anna Bleakley, Justin Edwards, Orla Cooney, Philip R Doyle, Leigh Clark, and Benjamin R Cowan. 2020. See what I’m saying? Comparing intelligent personal assistant use for native and non-native language speakers. In 22nd international conference on human-computer interaction with mobile devices and services. 1–9.
[91]
Meehyun Yoon, Janette Hill, and Dongho Kim. 2021. Designing supports for promoting self-regulated learning in the flipped classroom. Journal of Computing in Higher Education 33, 2 (2021), 398–418.
[92]
Qian Yu, Tonya Nguyen, Soravis Prakkamakul, and Niloufar Salehi. 2019. " I Almost Fell in Love with a Machine" Speaking with Computers Affects Self-disclosure. In Extended Abstracts of the 2019 CHI conference on human factors in computing systems. 1–6.
[93]
Angela Zavaleta Bernuy, Naaz Sibia, Pan Chen, Chloe Huang, Andrew Petersen, Joseph Jay Williams, and Michael Liut. 2023. VoiceEx: Voice Submission System for Interventions in Education. In Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 2 (Turku, Finland) (ITiCSE 2023). Association for Computing Machinery, New York, NY, USA, 585–586. https://doi.org/10.1145/3587103.3595284
[94]
Yi Zhang, Caixia Liu, Yana Xing, Zhongling Pi, and Jiumin Yang. 2023. How does drawing influence the effectiveness of oral self-explanation versus instructional explanation in video learning?British Journal of Educational Technology (2023).

Index Terms

  1. Does the Medium Matter? An Exploration of Voice-Interaction for Self-Explanations

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    DIS '24: Proceedings of the 2024 ACM Designing Interactive Systems Conference
    July 2024
    3616 pages
    ISBN:9798400705830
    DOI:10.1145/3643834
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 01 July 2024

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Active Learning
    2. Explanation Prompts
    3. Long-Term Memory
    4. Self-Explanations
    5. Student Performance
    6. Text Explanations
    7. Voice Explanations
    8. Voice-based Interaction

    Qualifiers

    • Research-article
    • Research
    • Refereed limited

    Funding Sources

    • NSERC Discovery grant
    • Office of Naval Research grant

    Conference

    DIS '24
    Sponsor:
    DIS '24: Designing Interactive Systems Conference
    July 1 - 5, 2024
    Copenhagen, Denmark

    Acceptance Rates

    Overall Acceptance Rate 1,158 of 4,684 submissions, 25%

    Upcoming Conference

    DIS '25
    Designing Interactive Systems Conference
    July 5 - 9, 2025
    Funchal , Portugal

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • 0
      Total Citations
    • 181
      Total Downloads
    • Downloads (Last 12 months)181
    • Downloads (Last 6 weeks)33
    Reflects downloads up to 16 Feb 2025

    Other Metrics

    Citations

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format.

    HTML Format

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media