skip to main content
10.1145/3613905.3650868acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
Work in Progress

Advancing Knowledge Together: Integrating Large Language Model-based Conversational AI in Small Group Collaborative Learning

Published: 11 May 2024 Publication History

Abstract

In today’s educational landscape, students learn collaboratively, where students benefit from both peer interactions and facilitator guidance. Prior research in Human-Computer Interaction (HCI) and Computer-Supported Collaborative Learning (CSCL) has explored chatbots and AI techniques to aid such collaboration. However, these methods often depend on predefined dialogues (which limits adaptability), are not based on collaborative learning theories, and do not fully recognize the learning context. In this paper, we introduce an Large Language Model (LLM)-powered conversational AI, designed to enhance small group learning through its advanced language understanding and generation capabilities. We detail the iterative design process, final design, and implementation. Our preliminary evaluation indicates that the bot performs as designed but points to considerations in the timing of interventions and bot’s role in discussions. The evaluation also reveals that learners perceive the bot’s tone and behavior as important for engagement. We discuss design implications for chatbot integration in collaborative learning and future research directions.

Supplemental Material

MP4 File
Talk Video
Transcript for: Talk Video

References

[1]
[n. d.]. Firebase. https://firebase.google.com/. Accessed: [Jan 22, 2023].
[2]
[n. d.]. Node.js. https://nodejs.org/. Accessed: [Jan 22, 2023].
[3]
[n. d.]. OpenAI. https://openai.com/. Accessed: [Jan 22, 2023].
[4]
[n. d.]. React - A JavaScript library for building user interfaces. https://reactjs.org/. Accessed: [Jan 22, 2023].
[5]
[n. d.]. Socket.IO. https://socket.io/. Accessed: [Jan 22, 2023].
[6]
David Adamson, Gregory Dyke, Hyeju Jang, and Carolyn Penstein Rosé. 2014. Towards an agile approach to adapting dynamic collaboration support to student needs. International Journal of Artificial Intelligence in Education 24 (2014), 92–124. https://doi.org/10.1007/s40593-013-0012-6
[7]
Douglas Barnes and Frankie Todd. 1977. Communication and learning in small groups.Routledge & Kegan Paul.
[8]
Ivo Benke, Ulrich Gnewuch, and Alexander Maedche. 2022. Understanding the impact of control levels over emotion-aware chatbots. Computers in Human Behavior 129 (2022), 107122. https://doi.org/10.1016/j.chb.2021.107122
[9]
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, 2020. Language models are few-shot learners. Advances in neural information processing systems 33 (2020), 1877–1901.
[10]
Jessy Ceha, Ken Jen Lee, Elizabeth Nilsen, Joslin Goh, and Edith Law. 2021. Can a humorous conversational agent enhance learning experience and outcomes?. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–14. https://doi.org/10.1145/3411764.3445068
[11]
Michelene TH Chi and Ruth Wylie. 2014. The ICAP framework: Linking cognitive engagement to active learning outcomes. Educational psychologist 49, 4 (2014), 219–243. https://doi.org/10.1080/00461520.2014.965823
[12]
Elizabeth G Cohen. 1994. Restructuring the classroom: Conditions for productive small groups. Review of educational research 64, 1 (1994), 1–35. https://doi.org/10.3102/00346543064001001
[13]
Nils Dahlbäck, Arne Jönsson, and Lars Ahrenberg. 1993. Wizard of Oz studies: why and how. In Proceedings of the 1st international conference on Intelligent user interfaces. 193–200. https://doi.org/10.1016/0950-7051(93)90017-N
[14]
Hyo Jin Do, Ha-Kyung Kong, Pooja Tetali, Jaewook Lee, and Brian P Bailey. 2023. To Err is AI: Imperfect Interventions and Repair in a Conversational Agent Facilitating Group Chat Discussions. Proceedings of the ACM on Human-Computer Interaction 7, CSCW1 (2023), 1–23. https://doi.org/10.1145/3579532
[15]
Arthur C Graesser, Nia Dowell, and Danielle Clewley. 2017. Assessing collaborative problem solving through conversational agents. Innovative assessment of collaboration (2017), 65–80. https://doi.org/doi.org/10.1007/978-3-319-33261-1_5
[16]
Arthur C Graesser, Nia Dowell, Andrew J Hampton, Anne M Lippert, Haiying Li, and David Williamson Shaffer. 2018. Building intelligent conversational tutors and mentors for team collaborative problem solving: Guidance from the 2015 Program for International Student Assessment. In Building intelligent tutoring systems for teams: What matters. Emerald Publishing Limited, 173–211. https://doi.org/10.1108/S1534-085620180000019012
[17]
Arthur C Graesser, Stephen M Fiore, Samuel Greiff, Jessica Andrews-Todd, Peter W Foltz, and Friedrich W Hesse. 2018. Advancing the science of collaborative problem solving. psychological science in the public interest 19, 2 (2018), 59–92. https://doi.org/doi.org/10.1177/152910061880824
[18]
Arthur C Graesser, Shulan Lu, George Tanner Jackson, Heather Hite Mitchell, Mathew Ventura, Andrew Olney, and Max M Louwerse. 2004. AutoTutor: A tutor with dialogue in natural language. Behavior Research Methods, Instruments, & Computers 36 (2004), 180–192. https://doi.org/doi.org/10.3758/BF03195563
[19]
Charlotte N Gunawardena, Constance A Lowe, and Terry Anderson. 1997. Analysis of a global online debate and the development of an interaction analysis model for examining social construction of knowledge in computer conferencing. Journal of educational computing research 17, 4 (1997), 397–431. https://doi.org/10.2190/7MQV-X9UJ-C7Q3-NRAG
[20]
Ariel Han, Xiaofei Zhou, Zhenyao Cai, Shenshen Han, Richard Ko, Seth Corrigan, and Kylie Peppler. 2024. Teachers, Parents, and Students’ Perspectives on Integrating Generative AI into Elementary Literacy Education. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems. To Appear. https://doi.org/10.1145/3613904.3642438
[21]
Weijiao Huang, Khe Foon Hew, and Donn Gonda. 2023. How to design an educational chatbot: Insights from experienced teachers and developers. In EdMedia+ Innovate Learning. Association for the Advancement of Computing in Education (AACE), 1339–1351.
[22]
Irving L Janis. 2008. Groupthink. IEEE Engineering Management Review 36, 1 (2008), 36.
[23]
Hyoungwook Jin, Seonghee Lee, Hyungyu Shin, and Juho Kim. 2023. " Teach AI How to Code": Using Large Language Models as Teachable Agents for Programming Education. arXiv preprint arXiv:2309.14534 (2023). https://doi.org/doi.org/10.48550/arXiv.2309.14534
[24]
Finn Kensing and Jeanette Blomberg. 1998. Participatory design: Issues and concerns. Computer supported cooperative work (CSCW) 7 (1998), 167–185. https://doi.org/doi.org/10.1023/A:1008689307411
[25]
Soomin Kim, Jinsu Eun, Joseph Seering, and Joonhwan Lee. 2021. Moderator chatbot for deliberative discussion: Effects of discussion structure and discussant facilitation. Proceedings of the ACM on Human-Computer Interaction 5, CSCW1 (2021), 1–26. https://doi.org/10.1145/3449161
[26]
Marjan Laal and Seyed Mohammad Ghodsi. 2012. Benefits of collaborative learning. Procedia-social and behavioral sciences 31 (2012), 486–490. https://doi.org/10.1016/j.sbspro.2011.12.091
[27]
Ken Jen Lee, Apoorva Chauhan, Joslin Goh, Elizabeth Nilsen, and Edith Law. 2021. Curiosity notebook: the design of a research platform for learning by teaching. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–26. https://doi.org/10.1145/3479538
[28]
Krittaya Leelawong and Gautam Biswas. 2008. Designing learning by teaching agents: The Betty’s Brain system. International Journal of Artificial Intelligence in Education 18, 3 (2008), 181–208. https://doi.org/10.5555/1454278.1454280
[29]
Arita L Liu and John C Nesbit. 2020. Dashboards for computer-supported collaborative learning. Machine learning paradigms: Advances in learning analytics (2020), 157–182. https://doi.org/10.1007/978-3-030-13743-4_9
[30]
Shuai Ma, Ying Lei, Xinru Wang, Chengbo Zheng, Chuhan Shi, Ming Yin, and Xiaojuan Ma. 2023. Who should i trust: Ai or myself? leveraging human and ai correctness likelihood to promote appropriate trust in ai-assisted decision-making. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–19. https://doi.org//doi.org/10.1145/3544548.3581058
[31]
Ha Nguyen. 2023. Role design considerations of conversational agents to facilitate discussion and systems thinking. Computers & Education 192 (2023), 104661. https://doi.org/10.1016/j.compedu.2022.104661
[32]
Ha Nguyen and Morgan Diederich. 2023. Facilitating knowledge construction in informal learning: A study of TikTok scientific, educational videos. Computers & Education 205 (2023), 104896. https://doi.org/10.1016/j.compedu.2023.104896
[33]
Nia Nixon, Lin Yiwen, and Snow Laura. [n. d.]. Catalyzing Equity in STEM Teams: Harnessing Generative AI for Inclusion and Diversity. Policy Insights from the Behavioral and Brain Sciences ([n. d.]). https://doi.org/doi.org/10.1177/23727322231220356
[34]
Theodore Panitz. 1999. Collaborative versus Cooperative Learning: A Comparison of the Two Concepts Which Will Help Us Understand the Underlying Nature of Interactive Learning. (1999).
[35]
Bertrand Schneider, Nia Dowell, and Kate Thompson. 2021. Collaboration analytics—current state and potential futures. Journal of Learning Analytics 8, 1 (2021), 1–12. https://doi.org/10.18608/JLA.2021.7447
[36]
Donghoon Shin, Soomin Kim, Ruoxi Shang, Joonhwan Lee, and Gary Hsieh. 2023. IntroBot: Exploring the Use of Chatbot-assisted Familiarization in Online Collaborative Groups. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. https://doi.org/10.1145/3544548.3580930
[37]
Seng Chee Tan, Alwyn Vwen Yen Lee, and Min Lee. 2022. A systematic review of artificial intelligence techniques for collaborative learning over the past two decades. Computers and Education: Artificial Intelligence (2022), 100097. https://doi.org/10.1016/j.caeai.2022.100097
[38]
Gaurav Singh Tomar, Sreecharan Sankaranarayanan, and Carolyn Penstein Rosé. 2016. Intelligent conversational agents as facilitators and coordinators for group work in distributed learning environments (MOOCs). In 2016 AAAI Spring Symposium Series.
[39]
Bruce W Tuckman. 1965. Developmental sequence in small groups.Psychological bulletin 63, 6 (1965), 384.
[40]
Kurt VanLehn. 2011. The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems. Educational psychologist 46, 4 (2011), 197–221. https://doi.org/10.1080/00461520.2011.611369
[41]
Noreen M Webb, Jonathan D Troper, and Randy Fall. 1995. Constructive activity and learning in collaborative small groups.Journal of educational psychology 87, 3 (1995), 406. https://doi.org/10.1037/0022-0663.87.3.406
[42]
Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, 2022. Emergent abilities of large language models. arXiv preprint arXiv:2206.07682 (2022).
[43]
Amy X Zhang and Justin Cranshaw. 2018. Making sense of group chat through collaborative tagging and summarization. Proceedings of the ACM on Human-Computer Interaction 2, CSCW (2018), 1–27. https://doi.org/10.1145/3274465
[44]
Jiawei Zhou, Yixuan Zhang, Qianni Luo, Andrea G Parker, and Munmun De Choudhury. 2023. Synthetic lies: Understanding ai-generated misinformation and evaluating algorithmic and human solutions. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–20. https://doi.org/doi.org/10.1145/3544548.3581318

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
CHI EA '24: Extended Abstracts of the CHI Conference on Human Factors in Computing Systems
May 2024
4761 pages
ISBN:9798400703317
DOI:10.1145/3613905
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 May 2024

Check for updates

Author Tags

  1. AI facilitator
  2. Collaborative Learning
  3. Human-AI Collaboration

Qualifiers

  • Work in progress
  • Research
  • Refereed limited

Conference

CHI '24

Acceptance Rates

Overall Acceptance Rate 6,164 of 23,696 submissions, 26%

Upcoming Conference

CHI 2025
ACM CHI Conference on Human Factors in Computing Systems
April 26 - May 1, 2025
Yokohama , Japan

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 1,802
    Total Downloads
  • Downloads (Last 12 months)1,802
  • Downloads (Last 6 weeks)274
Reflects downloads up to 22 Jan 2025

Other Metrics

Citations

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Full Text

View this article in Full Text.

Full Text

HTML Format

View this article in HTML Format.

HTML Format

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media